An apparatus for high-speed video is described herein. The apparatus includes a camera array, wherein each camera of the camera array is to capture an image at a different time offset resulting in a plurality of images. The apparatus also includes a controller to interleave the plurality of images in chronological order and a view synthesis unit to synthesize a camera view from a virtual camera for each image of the plurality of images. Additionally, the apparatus includes a post-processing unit to remove any remaining artifacts from the plurality of images.
Legal claims defining the scope of protection, as filed with the USPTO.
1. An apparatus for high-speed video, comprising: a camera array, wherein each camera of the camera array is to capture an image at a different time offset resulting in a plurality of images; a controller to interleave the plurality of images in chronological order; a processor to enable view synthesis to synthesize a camera view from a virtual camera for each image of the plurality of images; and the processor to enable post processing to remove any remaining artifacts from the plurality of images and to generate a video with an adaptive frame rate from the plurality of images, wherein the adaptive frame rate is enabled, in part, by a mask applied to each image from the virtual camera.
2. The apparatus of claim 1 , wherein post processing removes holes caused by synthesizing a camera view from a virtual camera.
3. The apparatus of claim 1 , wherein post processing removes the remaining artifacts due to holes filled incoherently during view synthesis.
4. The apparatus of claim 1 , wherein color mapping is used to reconcile color differences between each image.
5. The apparatus of claim 1 , wherein the resulting frame rate is N times larger than the frame rate of a single camera.
6. The apparatus of claim 1 , wherein each camera image has a different perspective compared to other camera images and synthesizing a camera view from each image results in a single view point.
7. The apparatus of claim 1 , wherein the camera array is arranged to enable high-speed video, depth applications, and post capture photography.
8. The apparatus of claim 1 , wherein a frame rate of a video rendered using the plurality of images is an adaptive frame rate, spatio-temporally varying in each frame, based on the amount of motion in regions of each image of the synthesized camera views.
9. The apparatus of claim 1 , wherein a perspective of the virtual camera is positioned at the center of the camera array.
10. The apparatus of claim 1 , wherein a perspective of the virtual camera is selected to be as close as possible to all cameras of the camera array.
11. The apparatus of claim 1 , wherein the view synthesis unit is to synthesize camera views via content aware warping.
12. A method for high-speed video from a camera array, comprising: capturing a plurality of frames using the camera array, wherein each camera of the camera array is to capture a frame at a different time offset; stacking the plurality of frames in chronological order; synthesizing a view from a single perspective for each frame of the plurality of frames; and processing the synthesized camera views to remove artifacts in the images to render a slow motion video with adaptive frame rate based on a mask generated for each synthesized view.
13. The method of claim 12 , wherein synthesizing the view includes hole-filling or inpainting for each frame.
14. The method of claim 12 , wherein synthesizing the view includes content aware warping.
15. The method of claim 12 , wherein synthesizing the view includes content aware warping, and wherein content aware warping comprises: obtaining a point correspondence between a frame and the single perspective; calculating a disparity to minimize a sum of squared differences between distortion terms; generating the synthesized view based on the disparity.
16. The method of claim 12 , wherein synthesizing the view includes content aware warping that synthesizes a view based on a dense disparity map or a sparse point correspondence.
17. A system for high speed video from a camera array, comprising: a display; a camera array; a memory that is to store instructions and that is communicatively coupled to the camera array and the display; and a processor communicatively coupled to the camera array, the display, and the memory, wherein when the processor is to execute the instructions, the processor is to: capture a plurality of frames using the camera array, wherein each camera of the camera array is to capture an image at a different time offset resulting in a plurality of images; interleave the plurality of images in an order of capture; synthesize a view from a single perspective for each image of the plurality of images; and process the synthesized views to remove artifacts in the images to render a slow motion video with adaptive frame rate based on a mask generated for each synthesized view.
18. The system of claim 17 , wherein processing the synthesized camera views removes holes caused by synthesizing a camera view from a virtual camera.
19. The system of claim 17 , wherein processing the synthesized camera views removes the remaining artifacts due to holes filled incoherently during view synthesis.
20. The system of claim 17 , wherein processing the synthesized camera views comprises color mapping that is to reconcile color differences between each image.
21. The system of claim 17 , wherein an effective frame rate is N times larger than a frame rate of a single camera of the camera array.
22. The system of claim 17 , wherein the camera array is arranged to enable high-speed video, depth image capture, and post-capture photography.
23. The system of claim 17 , wherein the frame rate of the synthesized camera views is an adaptive frame rate, spatio-temporally varying across the synthesized camera views, based on the amount of motion in regions of the synthesized camera views.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
June 30, 2016
October 26, 2021
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.