Various methods and systems are disclosed for near-infrared video compositing techniques and an associated immersive video environment. In an example, a system includes: a dual camera system having a visible light camera and an infrared camera to capture light from a field of view; and a display screen to reflect a video output projected from a display source. In this system configuration, the display screen is arranged relative to the dual camera system to allow infrared light and visible light from a field of view to travel through the display screen and reach the cameras of the dual camera system. In an example, the video output provides real-time video captured from the visible light camera, an information from the infrared camera is used to create a matte of an object in the field of view within the real-time video captured from the visible light camera.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A system, comprising: a dual camera system, comprising: a visible light camera, the visible light camera including a sensor and a lens to capture visible light from a field of view; an infrared camera, the infrared camera including a sensor and a lens to capture infrared light from the field of view; and a hot mirror oriented between the lens of the infrared camera and the lens of the visible light camera; wherein the lens of the visible light camera is aimed at the field of view, and wherein the lens of the infrared camera is aimed perpendicular to the field of view; and wherein the hot mirror is adapted to filter infrared light from the field of view of the visible light camera, and reflect infrared light towards the field of view of the infrared camera; and a display screen to reflect a video output projected from a display source, wherein the display screen is arranged relative to the dual camera system to allow the infrared light and the visible light from the field of view to travel through the display screen and reach the hot mirror of the dual camera system; wherein the video output includes real-time video captured from the visible light camera, and wherein information from the infrared camera is used to create a matte of an object in the field of view within the real-time video captured from the visible light camera.
2. The system of claim 1 , further comprising: a teleprompter housing, the teleprompter housing arranged to host the display screen, a teleprompter monitor, and a camera opening; wherein the teleprompter monitor projects the video output to the display screen; wherein the display screen is located between the camera opening and the field of view; and wherein the dual camera system is coupled to the teleprompter housing, wherein dual camera system is arranged relative to the camera opening to allow the infrared light and the visible light from the field of view to travel through the display screen and the camera opening, to reach the hot mirror of the dual camera system.
3. The system of claim 1 , further comprising: a visible light projector, the visible light projector arranged to project the video output on the display screen; wherein the display screen is adapted to reflect the video output towards the field of view, while allowing the infrared light and the visible light from the field of view to travel through the display screen to the hot mirror of the dual camera system.
4. The system of claim 1 , further comprising: a video processing system to generate the real-time video with compositing, the compositing to combine digital video of a first video stream with digital video of a second video stream, wherein the first video stream includes the matte of the object.
5. The system of claim 4 , wherein the video processing system is further to generate the real-time video with visual effects, the visual effects to apply a digital visual effect to the matte of the object.
6. The system of claim 5 , further comprising: a user interface device to receive input from a human user, wherein the input from the human user is used to control the compositing and the visual effects.
7. The system of claim 1 , further comprising: a visible light source including at least one visible light emitter arranged around the field of view, wherein the display source and the dual camera system are arranged relative to the visible light source such that the visible light source provides illumination of a first side of the object in the field of view.
8. The system of claim 7 , further comprising: an infrared light source including at least one infrared light emitter arranged behind the field of view, wherein the display source and the dual camera system are arranged relative to the infrared light source such that the infrared light source emits light towards the dual camera system and provides illumination of a second side of the object in the field of view.
9. The system of claim 8 , further comprising: a backdrop, the backdrop including a surface and the infrared light source, wherein the backdrop is arranged relative to the dual camera system such that the surface is visible in the field of view, wherein the infrared light source is arranged to emit infrared light through the surface into the field of view, and wherein the surface comprises a material that permits infrared light to pass through while absorbing at least a portion of the visible light from the field of view.
10. A video compositing system, comprising: a visible light source to emit visible light into a field of view in a first direction; an infrared light source to emit infrared light into the field of view in a second direction, the infrared light source included in a backdrop, wherein the first direction differs from the second direction; a dual camera system, including: a visible light camera, the visible light camera arranged to capture the visible light reflected from an object; an infrared camera, the infrared camera to capture the infrared light directed towards the dual camera system; a hot mirror disposed between the infrared camera and the visible light camera; wherein the hot mirror is adapted to filter infrared light from the field of view of the visible light camera, and reflect infrared light towards the field of view of the infrared camera; a video processing system adapted to receive visible video data from the visible light camera and infrared video data from the infrared camera, and to generate a video output; and a display screen arranged relative to the dual camera system, wherein the display screen reflects the video output provided from a display source while allowing the infrared light and the visible light to reach the dual camera system through the display screen.
11. The video compositing system of claim 10 , wherein the infrared light source is arranged to emit infrared light through the backdrop into the field of view, and wherein the backdrop comprises a material that permits infrared light to pass through while absorbing visible light.
12. The video compositing system of claim 10 , further comprising: a user interface device to receive input from a human user; wherein the video processing system is further to generate the video output with at least one visual effect; and wherein the input from the human user is used to control application of the at least one visual effect.
13. The video compositing system of claim 10 , further comprising: a teleprompter housing, the teleprompter housing including the display screen and a teleprompter display monitor, wherein the display screen is included within the teleprompter housing, wherein the teleprompter display monitor includes the display source, and wherein the display screen includes a reflective surface to reflect the video output from the teleprompter display monitor towards the field of view.
14. The video compositing system of claim 10 , further comprising: a visible light projector, the visible light projector arranged to project the video output to the display screen; wherein the display screen includes a microlens array material adapted to reflect the video output from the visible light projector towards the field of view, while allowing the visible light reflected from the object and the infrared light directed towards the dual camera system to be captured by the dual camera system.
15. A method for video compositing based on infrared video, comprising: capturing RGB video of visible light from a field of view using a visible light camera, wherein the visible light camera is arranged relative to a hot mirror, and wherein the hot mirror allows the visible light to travel through the hot mirror to the visible light camera; capturing infrared video of infrared light from the field of view using an infrared camera, wherein the infrared camera is arranged relative to the hot mirror, and wherein the hot mirror reflects the infrared light to the infrared camera, wherein the field of view is common to the visible light camera and the infrared camera; generating a matte of an object in the field of view from the RGB video captured by the visible light camera, based on removal of at least a portion of the RGB video using a silhouette of the object identified from the infrared video; and outputting a video including the matte to a display screen, wherein the display screen is arranged relative to the visible light camera and the infrared camera, to allow the infrared light and the visible light to reach the respective cameras through the display screen.
16. The method of claim 15 , wherein the display screen is provided from a reflective screen housed in a teleprompter, wherein the video is output from a teleprompter monitor on a first side of the reflective screen, and wherein the respective cameras capture the video through a second side of the reflective screen, wherein the reflective screen allows the infrared light and the visible light to reach the respective cameras while reflecting the video from the teleprompter monitor.
17. The method of claim 15 , wherein the display screen is provided in a reflective projection screen, wherein the video is output from a projector on a first side of the reflective projection screen, and wherein the respective cameras capture the video from a second side of the reflective projection screen, wherein the reflective projection screen allows the infrared light and the visible light to reach the respective cameras while reflecting the video output from the projector.
18. The method of claim 15 , further comprising: applying visual effects to at least a portion of the video.
19. The method of claim 18 , further comprising: receiving input from a human user, wherein the input from the human user is used to control the visual effects.
20. The method of claim 15 , further comprising: controlling the infrared light, wherein the infrared light is emitted into the field of view from a background unit, wherein the background unit includes a backdrop and a source of the infrared light, wherein the source of the infrared light is arranged to emit infrared light through the backdrop into the field of view towards the infrared camera; wherein the object blocks at least a portion of the emitted infrared light from the field of view captured by the infrared light camera.
21. The method of claim 20 , further comprising: controlling the visible light, wherein the visible light is emitted into the field of view from a lighting unit, wherein the lighting unit is arranged to emit visible light towards the object; wherein the object reflects at least a portion of the emitted visible light from the field of view captured by the visible light camera.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 22, 2017
April 23, 2019
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.