A method and system are described. The method includes capturing a set of images from an array of cameras, each camera of the array of cameras having an overlapping field of view (FOV) with an adjacent camera of the array of cameras. The method further includes synchronously capturing a supplemental image from an additional camera, the additional camera having an at least partially overlapping FOV with every camera of the array of cameras. Supplemental information is extracted by comparing the supplemental image with the set of images. Portions of the set of images are stitched based in part on the supplemental information to produce a combined stitched image, the combined stitched image having a higher resolution than each image of the set of images.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method for capturing images, comprising: capturing a set of images from a plurality of cameras, each camera of the plurality of cameras having an overlapping field of view (FOV) with an adjacent camera of the plurality of cameras; synchronously capturing a supplemental image from an additional camera, the additional camera having an at least partially overlapping FOV with the plurality of cameras; extracting supplemental information by comparing the supplemental image with the set of images; and stitching portions of the set of images based in part on the supplemental information to produce a combined stitched image, the combined stitched image having a higher resolution than the set of images.
2. The method of claim 1 , wherein each camera in the plurality of cameras is rotated by an offset angle from a reference plane, and wherein combined FOVs of the plurality of cameras are greater than an individual FOV of each camera.
3. The method of claim 1 , wherein the additional camera is located between the plurality of cameras.
4. The method of claim 1 , wherein the additional camera is located external to the plurality of cameras.
5. The method of claim 1 , wherein the additional camera is a depth sensing camera.
6. The method of claim 1 , wherein the supplemental information includes depth information, and wherein stitching portions of the set of images based in part on the supplemental information comprises: performing image manipulation operations on the set of images to correct for a corresponding rotated FOV of each camera in the plurality of cameras; identifying, using the depth information, objects within portions of the set of images corresponding to the overlapping FOVs; aligning the set of images such that one or more of the identified objects are aligned; and stitching the aligned set of images to produce the combined stitched image.
7. The method of claim 6 , wherein the supplemental information additionally includes object tracking information that indicates a location of objects in a shared FOV of the plurality of cameras, and wherein stitching portions of the set of images based in part on the supplemental information further comprises: identifying, using the object tracking information, objects within portions of the set of images corresponding to overlapping FOVs of each camera of the plurality of cameras.
8. The method of claim 1 , wherein the supplemental information includes depth information, the method further comprising: generating a three-dimensional (3D) reconstruction of a scene captured by the plurality of cameras using the depth information to produce a 3D model of the scene and using the set of images to texture the 3D model of the scene to produce the 3D reconstruction of the scene.
9. The method of claim 1 , wherein the supplemental information includes depth information, the method further comprising: generating object tracking information based on the depth information, the object tracking information indicating locations of objects in a combined FOV of the plurality of cameras; and maintaining optical focus on an object in the combined FOV of the plurality of cameras over a plurality of captured images using the object tracking information.
10. The method of claim 1 , wherein the additional camera captures video for use as a live video feed.
11. A system comprising: a plurality of cameras, each camera of the plurality of cameras having an overlapping field of view (FOV) with an adjacent camera of the plurality of cameras, the plurality of cameras configured to capture a set of images; an additional camera having an at least partially overlapping FOV with the plurality of cameras, the additional camera configured to capture a supplemental image; and an image processor configured to stitch portions of the set of images based at least in part on supplemental information to produce a combined stitched image, the supplemental information extracted based on the supplemental image, the combined stitched image having a higher resolution than the set of images.
12. The system of claim 11 , wherein each camera in the plurality of cameras is rotated by an offset angle from a reference plane, and wherein combined FOVs of the plurality of cameras are greater than an individual FOV of each camera.
13. The system of claim 11 , wherein the additional camera is located between the plurality of cameras.
14. The system of claim 11 , wherein the additional camera is located external to the plurality of cameras.
15. The system of claim 11 , wherein the additional camera is a depth sensing camera.
16. The system of claim 11 , wherein the supplemental information includes depth information, and wherein the image processor is further configured to: perform image manipulation operations on the set of images to correct for a corresponding rotated FOV of each camera in the plurality of cameras; identify, using the depth information, objects within portions of the set of images corresponding to the overlapping FOVs; align the set of images such that one or more of the identified objects are aligned; and stitch the aligned set of images to produce the combined stitched image.
17. The system of claim 16 , wherein the supplemental information additionally includes object tracking information that indicates a location of objects in a shared FOV of the plurality of cameras, and wherein the image processor is further configured to: identify, using the object tracking information, objects within portions of the set of images corresponding to overlapping FOVs of each camera of the plurality of cameras.
18. The system of claim 11 , wherein the supplemental information includes depth information, and wherein the image processor is further configured to: generate a three-dimensional (3D) reconstruction of a scene captured by the plurality of cameras using the depth information to produce a 3D model of the scene and using the set of images to texture the 3D model of the scene to produce the 3D reconstruction of the scene.
19. The system of claim 11 , wherein the supplemental information includes depth information, and wherein the image processor is further configured to: generate object tracking information based on the depth information, the object tracking information indicating locations of objects in a combined FOV of the plurality of cameras; and maintain optical focus on an object in the combined FOV of the plurality of cameras over a plurality of captured images using the object tracking information.
20. A system comprising: at least one camera having an overlapping field of view (FOV) with at least one other camera, the at least one camera configured to capture a set of images and the at least one other camera configured to capture a supplemental image; and an image processor configured to stitch portions of the set of images based at least in part on supplemental information to produce a combined stitched image, the supplemental information extracted based on the supplemental image, the combined stitched image having a higher resolution than the set of images.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
July 11, 2019
August 4, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.