A computer-implemented method for obtaining texture data for a three-dimensional model is disclosed. The method is performed at a portable electronic device including a camera, one or more processors, and memory storing one or more programs. The method includes obtaining an image of one or more objects with the camera. The method also includes concurrently displaying at least a portion of the image with one or more objects in the three-dimensional model. The one or more objects in the three-dimensional model at least partially overlay the one or more objects in the image. The method furthermore includes storing at least one or more portions of the image as texture data for one or more of the one or more objects in the image.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method for obtaining texture data for a three-dimensional model, comprising: at a portable electronic device including a camera, one or more processors, and memory storing one or more programs, the one or more processors executing the one or more programs to perform the operations of: obtaining with the camera a live-video of real-world objects as the portable electronic device moves; obtaining a three-dimensional model including at least one virtual object; concurrently displaying at least a portion of the live-video in real-time and a representation of the three-dimensional model, wherein the representation of the three-dimensional model at least partially overlays the display of the portion of the live-video; and while concurrently displaying at least the portion of the live-video in real-time and the representation of the three-dimensional model: aligning the representation of one or more virtual objects in the three-dimensional model with one or more real-world objects in the live-video in real-time; storing an image of the one or more real-world objects captured by the camera in real-time as texture data for the one or more virtual objects in the three-dimensional model; concurrently displaying an overhead view of the three-dimensional model and displaying as part of the overhead view a depiction of a field of view of the camera; and subsequent to obtaining an image of objects with the camera: determining a recommended location of the camera for obtaining the texture data; and visually indicating in the overhead view the recommended location of the camera for obtaining the texture data.
2. The method of claim 1 , wherein the one or more virtual objects in the three-dimensional model correspond to the one or more real-world objects in the live-video, and aligning the representation of one or more virtual objects in the three-dimensional model with one or more real-world objects in the live video includes: associating a first set of markers placed in the three-dimensional model with a second set of markers placed in the image.
3. The method of claim 2 , wherein the device includes a touch-sensitive display and the method includes: receiving one or more touch gestures on the touch-sensitive display; and aligning the representation of the one or more virtual objects in the three-dimensional model in accordance with the one or more touch gestures.
4. The method of claim 3 , wherein aligning the one or more virtual objects includes one or more of: scaling the representation of the one or more virtual objects in the three-dimensional model; panning the representation of the one or more virtual objects in the three-dimensional model; and rotating the representation of the one or more virtual objects in the three-dimensional model.
5. The method of claim 2 , wherein the live-video includes one or more augmented reality markers and the method includes: aligning the representation of the one or more virtual objects in the three-dimensional model in accordance with respective locations of the one or more augmented reality markers.
6. The method of claim 5 , further comprising: calibrating one or more of a location and an orientation of the portable electronic device in accordance with at least one of the one or more augmented reality markers.
7. The method of claim 5 , wherein: information for the one or more augmented reality markers is received at the portable electronic device from a remote server, the one or more augmented reality markers received at the portable electronic device are selected and updated in accordance with a location of the portable electronic device as the portable electronic device moves.
8. The method of claim 1 , further comprising: obtaining a plurality of images at a respective location; and generating a respective three-dimensional model of the respective location based on the plurality of images, including identifying one or more interfaces of walls by using edge detection.
9. The method of claim 8 , wherein the device includes an orientation sensor and the method includes: obtaining the plurality of images at the respective location with orientation information for the plurality of images; and generating the respective three-dimensional model of the respective location based on the plurality of images and the orientation information.
10. The method of claim 1 , wherein the image is sent to a remote server as texture data for the one or more virtual objects in the three-dimensional model.
11. The method of claim 10 , wherein additional texture data is sent to the remote server from a plurality of portable electronic devices, distinct from the portable electronic device.
12. The method of claim 1 , further comprising: receiving an image rendered from information for the one or more virtual objects in the three-dimensional model and texture data for the one or more virtual objects in the three-dimensional model; and displaying at least a portion of the rendered image.
13. The method of claim 12 , further comprising: receiving a touch gesture on the touch-sensitive display; and adjusting the texture data in accordance with the touch gesture while maintaining the representation of the one or more virtual objects in the three-dimensional model, wherein adjusting the texture data includes one or more of: positioning the texture data onto the one or more virtual objects in the three-dimensional model; resizing the texture data; and cropping the texture data.
14. The method of claim 1 , further comprising: receiving information for camera placement, the information for camera placement based at least on a viewing angle and a field of view with respect to the one or more virtual objects.
15. The method of claim 1 , wherein a virtual object of the one or more virtual objects with missing texture data is displayed in a distinct color.
16. The method of claim 1 , including: concurrently displaying at least the portion of the live-video in real-time and a second representation of the three-dimensional model, wherein the second representation of the three-dimensional model at least partially overlays the display of the portion of the live-video, the image is applied as the texture data for the one or more virtual objects in the three-dimensional model.
17. A non-transitory computer readable storage medium storing one or more programs for execution by one or more processors of an electronic device with a camera, the one or more programs including instructions for: obtaining with the camera a live-video of real-world objects as the portable electronic device moves; obtaining a three-dimensional model including at least one virtual object; concurrently displaying at least a portion of the live-video in real-time and a representation of the three-dimensional model, wherein the representation of the three-dimensional model at least partially overlays the display of the portion of the live-video; and while concurrently displaying at least the portion of the live-video in real-time and the representation of the three-dimensional model: aligning the representation of one or more virtual objects in the three-dimensional model with one or more real-world objects in the live-video in real-time; storing an image of the one or more real-world objects captured by the camera in real-time as texture data for the one or more virtual objects in the three-dimensional model; concurrently displaying an overhead view of the three-dimensional model and displaying as part of the overhead view a depiction of a field of view of the camera; and subsequent to obtaining an image of objects with the camera: determining a recommended location of the camera for obtaining the texture data; and visually indicating in the overhead view the recommended location of the camera for obtaining the texture data.
18. The computer readable storage medium of claim 17 , wherein the one or more virtual objects in the three-dimensional model correspond to the one or more real-world objects in the live-video, and aligning the representation of one or more virtual objects in the three-dimensional model with one or more real-world objects in the live video includes: associating a first set of markers placed in the three-dimensional model with a second set of markers placed in the image.
19. An electronic device, comprising: a camera; one or more processors; and memory storing one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the operations of: obtaining with the camera a live-video of real-world objects as the portable electronic device moves; obtaining a three-dimensional model including at least one virtual object; concurrently displaying at least a portion of the live-video in real-time and a representation of the three-dimensional model, wherein the representation of the three-dimensional model at least partially overlays the display of the portion of the live-video; and while concurrently displaying at least the portion of the live-video in real-time and the representation of the three-dimensional model: aligning the representation of one or more virtual objects in the three-dimensional model with one or more real-world objects in the live-video in real-time; storing an image of the one or more real-world objects captured by the camera in real-time as texture data for the one or more virtual objects in the three-dimensional model; concurrently displaying an overhead view of the three-dimensional model and displaying as part of the overhead view a depiction of a field of view of the camera; and subsequent to obtaining an image of objects with the camera: determining a recommended location of the camera for obtaining the texture data; and visually indicating in the overhead view the recommended location of the camera for obtaining the texture data.
20. The device of claim 19 , wherein the one or more virtual objects in the three-dimensional model correspond to the one or more real-world objects in the live-video, and aligning the representation of one or more virtual objects in the three-dimensional model with one or more real-world objects in the live video includes: associating a first set of markers placed in the three-dimensional model with a second set of markers placed in the image.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
July 3, 2013
September 6, 2016
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.