Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method comprising: obtaining a first plenoptic video data capturing a scene from a first plenoptic device; obtaining a common 3D reference system for spatially locating the first plenoptic device and a second plenoptic device, the second plenoptic device having captured a second plenoptic video data from the scene, and the first plenoptic video data and the second plenoptic video data having been temporally aligned; and refocusing a rendering of at least a part of said first plenoptic video data on at least one common point of interest based on the second plenoptic video data, the common 3D reference system, and said temporal alignment.
This invention relates to plenoptic video processing, specifically for refocusing video content captured from multiple plenoptic devices. The technology addresses the challenge of dynamically adjusting focus in plenoptic video recordings, which traditionally lack the ability to refocus after capture. Plenoptic cameras capture light field data, allowing for post-capture refocusing but typically only within a single device's field of view. The invention improves this by enabling refocusing across multiple plenoptic devices. The method involves obtaining a first plenoptic video from a first plenoptic device and a second plenoptic video from a second plenoptic device, both capturing the same scene. The videos are temporally aligned, meaning their timestamps correspond to the same moments in time. A common 3D reference system is used to spatially locate both devices, ensuring their positions and orientations are known relative to each other. Using this spatial and temporal alignment, the system refocuses a portion of the first plenoptic video on a common point of interest in the scene, leveraging data from the second plenoptic video. This allows for dynamic refocusing across multiple viewpoints, enhancing depth perception and rendering flexibility in plenoptic video applications. The approach is useful in scenarios requiring multi-view refocusing, such as virtual reality, surveillance, or 3D reconstruction.
2. The method according to claim 1 , wherein obtaining a common 3D reference system comprises using structure-from-motion for determining a first 3D pose for said first plenoptic device and a second 3D pose for said second plenoptic device.
This invention relates to 3D imaging systems using multiple plenoptic devices to capture and reconstruct scenes. The problem addressed is accurately aligning and combining data from different plenoptic devices to create a coherent 3D representation. Plenoptic devices capture light field data, but integrating multiple devices requires precise spatial and angular alignment to avoid distortions or misalignments in the final 3D model. The method involves using structure-from-motion (SfM) techniques to determine the 3D poses of each plenoptic device. SfM is a photogrammetric method that reconstructs 3D geometry from 2D images by analyzing feature points across multiple views. For the first plenoptic device, SfM calculates its 3D pose, including position and orientation, relative to the scene. Similarly, the second plenoptic device's 3D pose is determined using the same SfM process. By establishing these poses, the system can align the light field data from both devices into a common 3D reference system. This alignment ensures that the combined data accurately represents the scene's geometry and appearance, enabling high-quality 3D reconstruction. The method improves the accuracy and reliability of multi-device plenoptic imaging systems.
3. The method according to claim 1 , further comprising determining common refocusing parameters in said common 3D reference system from said at least one common point of interest, said common refocusing parameters being converted in a rendering refocusing plane of a 3D reference system associated with said first plenoptic device.
This invention relates to plenoptic imaging systems, which capture light field data to enable refocusing and depth estimation. The problem addressed is the difficulty of aligning and refocusing images from multiple plenoptic devices to create a coherent 3D representation. The solution involves determining common refocusing parameters in a unified 3D reference system based on at least one shared point of interest. These parameters are then converted into a rendering refocusing plane associated with a first plenoptic device, allowing consistent refocusing across multiple devices. The method ensures that refocusing operations are synchronized in a shared coordinate system, improving accuracy and usability in applications like 3D imaging, virtual reality, and computer vision. The approach leverages geometric transformations to map refocusing parameters between different plenoptic devices, ensuring compatibility and reducing computational overhead. This enables seamless integration of data from multiple plenoptic sensors, enhancing depth perception and image reconstruction in dynamic environments.
4. The method according to claim 3 , wherein said refocusing is recomputed when a distance between said rendering refocusing plane and a plane perpendicular to an optical axis of said first plenoptic device, exceeds a predetermined value.
A method for dynamic refocusing in plenoptic imaging systems addresses the challenge of maintaining accurate focus in scenes with varying depth. Plenoptic cameras capture light field data, enabling post-capture refocusing by adjusting the virtual refocusing plane. However, as the scene depth changes, the initial refocusing plane may no longer align with the optimal focus plane, degrading image quality. This method improves upon existing plenoptic imaging by automatically recomputing the refocusing plane when the distance between the current rendering refocusing plane and a plane perpendicular to the optical axis of the plenoptic device exceeds a predetermined threshold. The predetermined value ensures that refocusing adjustments are made only when necessary, balancing computational efficiency and image quality. The method involves continuously monitoring the distance between the refocusing plane and the perpendicular plane, triggering a recomputation when the threshold is crossed. This dynamic adjustment ensures that the refocused image remains sharp and clear, even as the scene depth varies. The technique is particularly useful in applications requiring real-time focus adjustments, such as surveillance, medical imaging, and augmented reality. By integrating this adaptive refocusing mechanism, the method enhances the practicality and performance of plenoptic imaging systems in dynamic environments.
5. The method according to claim 3 , wherein said determining of common refocusing plane parameters in said common 3D reference system comprises: determining of said at least one common point of interest by obtaining information representative of at least one point of interest associated with at least one plenoptic video data of interest capturing said scene and provided by a third plenoptic device; using respectively at least one transformation between said common 3D reference system and at least one 3D reference system, associated with said third plenoptic device, for converting said information into said common 3D reference system; and determining said common refocusing plane parameters from said information converted into said common 3D reference system.
This invention relates to plenoptic video systems, specifically methods for determining common refocusing plane parameters in a shared 3D reference system. The problem addressed is the need to accurately align and refocus plenoptic video data from multiple devices to a common point of interest within a scene. Plenoptic cameras capture light field data, allowing post-capture refocusing, but aligning refocusing planes across different devices remains challenging due to varying reference systems. The method involves determining at least one common point of interest in a scene by obtaining information about a point of interest from plenoptic video data captured by a third plenoptic device. This information is then converted into a common 3D reference system using a transformation between the common system and the third device's reference system. The common refocusing plane parameters are then derived from the converted information. This ensures that refocusing operations across multiple plenoptic devices are consistent and aligned to the same point in 3D space. The approach leverages transformations between reference systems to maintain accuracy and coherence in refocusing across different plenoptic video streams.
6. The method according to claim 5 , wherein said information is provided automatically by said third plenoptic device.
A plenoptic imaging system captures light field data to reconstruct three-dimensional scenes. A challenge in such systems is efficiently providing additional information, such as depth or calibration data, to enhance image processing. This invention addresses the problem by automatically supplying such information from a third plenoptic device within the system. The third plenoptic device operates independently to gather supplementary data, which is then integrated with the primary imaging process. This ensures accurate and real-time enhancement of the captured light field without manual intervention. The system may include multiple plenoptic devices working in tandem, where the third device specifically handles auxiliary data acquisition. By automating this process, the invention improves efficiency and reduces errors in depth estimation, scene reconstruction, and other light field applications. The solution is particularly useful in advanced imaging systems requiring high precision and dynamic data integration.
7. The method according to claim 5 , wherein said information is: representative of a single point of interest associated with a single plenoptic video data of interest; and provided, through a user interface, associated with a device rendering one of said plenoptic video data capturing said scene.
This invention relates to plenoptic video data processing, specifically methods for handling information related to points of interest within plenoptic video streams. Plenoptic video captures light field data, enabling post-capture refocusing and perspective adjustments. A key challenge is efficiently managing and displaying information about specific points of interest within these complex datasets. The method involves processing information that represents a single point of interest within a plenoptic video. This information is provided through a user interface associated with a device that renders the plenoptic video capturing the scene. The user interface allows users to interact with the plenoptic video data, such as selecting or annotating points of interest. The method ensures that the information is accurately linked to the corresponding plenoptic video data, enabling precise visualization and analysis of the selected point of interest. This approach enhances user experience by simplifying the identification and tracking of specific features within the plenoptic video stream. The technique is particularly useful in applications requiring detailed scene analysis, such as surveillance, medical imaging, or virtual reality.
8. The method according to claim 5 , wherein said information is: representative of at least two points of interest associated with at least one plenoptic video data of interest; and provided, through at least one user interface, associated with at least one device rendering one of said at least one plenoptic video data of interest.
This invention relates to plenoptic video data processing, specifically enhancing user interaction with plenoptic video content. Plenoptic video captures light field data, allowing for post-capture refocusing and perspective adjustments. The challenge addressed is efficiently providing users with relevant information about points of interest within plenoptic video data during playback or rendering. The method involves displaying information about at least two points of interest within a plenoptic video. These points of interest are associated with the plenoptic video data, which contains multi-dimensional light field information. The information is presented through a user interface on a device rendering the plenoptic video. The user interface may be part of the rendering device or a separate device, enabling users to interact with the plenoptic content more effectively. The points of interest could represent objects, regions, or events within the video, and the associated information may include metadata, annotations, or interactive controls. This approach enhances user engagement by providing contextual details during playback, improving navigation and understanding of the plenoptic video content. The method ensures that the information is dynamically accessible, aligning with the user's interaction with the rendered plenoptic video.
9. The method according to claim 5 , wherein said method comprises a temporal filtering of said information representative of at least one point of interest, said temporal filtering being applied over a part of said at least one plenoptic video data of interest.
This invention relates to processing plenoptic video data to enhance the representation of points of interest. Plenoptic video captures light field information, allowing for post-capture adjustments like refocusing and depth estimation. The challenge addressed is improving the clarity and relevance of specific points of interest within the video by applying temporal filtering to reduce noise or irrelevant data over time. The method involves selecting at least one point of interest within the plenoptic video data. Temporal filtering is then applied to the information representing this point, focusing on a specific segment of the video where the point is relevant. This filtering process helps isolate and refine the point of interest by reducing temporal variations that may obscure it. The filtering can be used to enhance tracking, reduce motion blur, or improve depth estimation for the selected point. The method may also include preprocessing steps to identify and extract the point of interest from the plenoptic data, such as depth mapping or light field analysis. The temporal filtering is dynamically adjusted based on the characteristics of the point and the surrounding video content, ensuring that the filtering is applied only where necessary to maintain computational efficiency and accuracy. The result is a more precise and stable representation of the point of interest over time, improving applications like object tracking, augmented reality, and video analysis.
10. The method according to claim 1 , wherein when at least two successive refocusings associated respectively with at least two rendering refocusing planes are performed at two distinct instants, said method comprises generating a sequence of frames whose respective focusing planes are located between said at least two rendering refocusing planes.
This invention relates to a method for generating a sequence of frames with intermediate focusing planes between two distinct refocusing planes in a refocusing imaging system. The method addresses the challenge of smoothly transitioning between different focal depths in refocusable imaging, ensuring visual continuity when shifting focus between two or more rendering planes. The method involves performing at least two successive refocusings, each associated with a distinct rendering refocusing plane, at different instants. Between these refocusings, the method generates a sequence of frames where the focusing planes are positioned between the two rendering refocusing planes. This intermediate frame generation ensures a gradual transition in focus, preventing abrupt changes and improving the visual experience in applications such as refocusable displays, computational photography, or 3D imaging. The method may be applied in systems where refocusing is achieved through optical, computational, or hybrid means, allowing for dynamic adjustment of focal depth. By interpolating focus between predefined planes, the technique enhances the realism and smoothness of refocused images or videos, making it suitable for medical imaging, virtual reality, or augmented reality applications. The invention improves upon prior art by providing a seamless transition between focal depths, addressing the problem of discontinuous focus shifts in refocusable imaging systems.
11. A device comprising a processor configured to: obtain a first plenoptic video data capturing a scene from a first plenoptic device; obtain a common 3D reference system for spatially locating the first plenoptic device and a second plenoptic device, the second plenoptic device having captured a second plenoptic video data from the scene, and the first plenoptic video data and the second plenoptic video data having been temporally aligned; and refocus a rendering of at least a part of said first plenoptic video data on at least one common point of interest based on the second plenoptic video data, the common 3D reference system, and said temporal alignment.
This invention relates to plenoptic video processing, specifically for enhancing depth perception and refocusing capabilities in multi-camera plenoptic systems. The problem addressed is the difficulty in accurately refocusing plenoptic video data captured from different devices while maintaining spatial and temporal coherence. The device includes a processor that obtains a first plenoptic video of a scene from a first plenoptic camera. It also acquires a second plenoptic video of the same scene from a second plenoptic camera, ensuring both videos are temporally aligned. A common 3D reference system is used to spatially locate both cameras, enabling precise coordination between their viewpoints. The processor then refocuses a portion of the first plenoptic video on a shared point of interest in the scene, leveraging data from the second plenoptic video, the 3D reference system, and the temporal alignment. This allows for dynamic adjustments in focus, depth, and perspective, improving the accuracy and flexibility of plenoptic video rendering. The system ensures that refocusing operations are consistent across multiple viewpoints, enhancing the overall quality and usability of the captured data.
12. The device according to claim 11 , wherein the processor is further configured to obtain the common 3D reference system using structure-from-motion for determining a first 3D pose for said first plenoptic device and a second 3D pose for said second plenoptic device.
This invention relates to a system for generating a 3D reference system using multiple plenoptic devices. The problem addressed is the need for accurate spatial alignment of data captured by different plenoptic cameras, which is essential for applications like 3D reconstruction, augmented reality, and robotics. The solution involves a device with a processor that obtains a common 3D reference system by determining the 3D poses of at least two plenoptic devices. The processor uses structure-from-motion techniques to calculate a first 3D pose for the first plenoptic device and a second 3D pose for the second plenoptic device. Structure-from-motion is a computational photography method that reconstructs 3D structures from 2D images by analyzing overlapping features across multiple views. The resulting 3D poses define the position and orientation of each plenoptic device within the shared reference system, enabling precise spatial registration of captured data. This approach improves the accuracy of multi-camera 3D mapping and reduces errors caused by misalignment between devices. The system is particularly useful in dynamic environments where real-time 3D reconstruction is required.
13. The device according to claim 11 , wherein the processor is further configured to determine common refocusing parameters in said common 3D reference system from said at least one common point of interest, and to convert said common refocusing parameters in a rendering refocusing plane of a 3D reference system associated with said first plenoptic device.
This invention relates to plenoptic imaging systems, which capture light field data to enable post-capture refocusing. The problem addressed is the difficulty of aligning and refocusing images from multiple plenoptic devices in a consistent 3D reference system, particularly when capturing overlapping scenes from different viewpoints. The system includes at least two plenoptic devices capturing light field data of a scene. A processor identifies at least one common point of interest in the overlapping regions of the captured data. The processor then determines common refocusing parameters in a shared 3D reference system based on these common points. These parameters are used to refocus the images in a rendering plane of a 3D reference system associated with one of the plenoptic devices. This allows for consistent refocusing across multiple plenoptic devices, improving depth perception and 3D reconstruction accuracy in multi-view imaging applications. The system may be used in medical imaging, surveillance, or 3D mapping where precise alignment of multiple light field views is required.
14. The device according to claim 13 , wherein said refocusing is recomputed when a distance between said rendering refocusing plane and a plane perpendicular to an optical axis of said first plenoptic device, exceeds a predetermined value.
A plenoptic imaging system captures light field data to enable post-capture refocusing. However, existing systems may not dynamically adjust refocusing based on changing conditions, leading to suboptimal image quality. This invention addresses this by dynamically recomputing refocusing when the distance between the rendering refocusing plane and a plane perpendicular to the optical axis of the plenoptic device exceeds a predetermined threshold. The system includes a first plenoptic device with an array of microlenses and a sensor array to capture light field data. A processing unit processes this data to generate refocused images. The refocusing plane is initially set based on user input or automatic detection. If the distance between this plane and a reference plane perpendicular to the optical axis exceeds a predefined value, the system recomputes the refocusing to maintain image sharpness and clarity. This dynamic adjustment ensures consistent image quality even when the scene or viewing conditions change. The invention improves upon prior art by automatically recalculating refocusing parameters to adapt to varying distances, enhancing the flexibility and performance of plenoptic imaging systems.
15. The device according to claim 13 , wherein the processor is further configured to: determine of said at least one common point of interest by obtaining information representative of at least one point of interest associated with at least one plenoptic video data of interest capturing said scene and provided by a third plenoptic device; use respectively at least one transformation between said common 3D reference system and at least one 3D reference system, associated with said third plenoptic device, for converting said information into said common 3D reference system; and determine said common refocusing plane parameters from said information converted into said common 3D reference system.
This invention relates to plenoptic imaging systems, which capture light field data to enable post-capture refocusing and depth estimation. The problem addressed is the difficulty of aligning and refocusing plenoptic video data from multiple devices onto a shared 3D reference system, particularly when integrating data from third-party plenoptic devices. The system includes a processor that identifies at least one common point of interest in a scene by obtaining information about points of interest from plenoptic video data captured by a third plenoptic device. The processor then applies transformations between a common 3D reference system and the 3D reference system of the third device to convert the point-of-interest data into the common system. Using this converted information, the processor determines refocusing plane parameters for the common system, enabling consistent refocusing across multiple plenoptic devices. This allows seamless integration of data from different sources while maintaining accurate depth and focus alignment. The solution enhances applications like 3D reconstruction, augmented reality, and multi-camera plenoptic video processing by ensuring spatial coherence between disparate data streams.
16. The device according to claim 15 , wherein said information is provided automatically by said third plenoptic device.
A plenoptic imaging system captures light field data to reconstruct three-dimensional scenes. Traditional systems require manual input or separate devices to provide additional information for accurate reconstruction. This invention addresses the need for an automated solution by integrating a third plenoptic device that autonomously supplies necessary information to enhance image reconstruction. The third plenoptic device operates in conjunction with a primary plenoptic device and a secondary plenoptic device, each capturing different aspects of the light field. The third device automatically provides data such as depth information, angular resolution, or calibration parameters, eliminating the need for manual intervention. This improves the accuracy and efficiency of the 3D reconstruction process. The system ensures seamless integration by synchronizing the third device with the primary and secondary devices, ensuring consistent and reliable data acquisition. The automated provision of information reduces user effort and enhances the overall performance of the plenoptic imaging system.
17. The device according to claim 15 , wherein said information is: representative of a single point of interest associated with a single plenoptic video data of interest; and provided, through a user interface, associated with a device rendering one of said plenoptic video data capturing said scene.
This invention relates to plenoptic video systems, which capture and process light field data to enable interactive viewing and refocusing of scenes. The problem addressed is the need to efficiently identify and access specific points of interest within large plenoptic video datasets, which can be cumbersome due to the high dimensionality of the data. The device includes a system for processing plenoptic video data, where the data represents a scene captured from multiple viewpoints. The system extracts and displays information about a single point of interest within the plenoptic video data. This information is linked to a specific plenoptic video that captures the scene, allowing users to quickly locate and interact with the point of interest. The information is provided through a user interface associated with a device that renders the plenoptic video, enabling seamless navigation and exploration of the captured scene. The system enhances user experience by reducing the complexity of searching and accessing specific points within plenoptic video data, making it easier to analyze and interact with the captured light field information. This is particularly useful in applications such as virtual reality, medical imaging, and surveillance, where precise and efficient access to scene details is critical.
18. The device according to claim 15 , wherein said information is: representative of at least two points of interest associated with at least one plenoptic video data of interest; and provided, through at least one user interface, associated with at least one device rendering one of said at least one plenoptic video data of interest.
This invention relates to plenoptic video systems, which capture and process light field data to enable interactive viewing experiences, such as refocusing or changing viewpoints. The problem addressed is the need to efficiently present and interact with multiple points of interest within plenoptic video data, allowing users to explore key elements dynamically. The device includes a processing system that extracts and represents at least two points of interest from plenoptic video data. These points of interest are visually or contextually significant features within the video, such as objects, events, or regions of high detail. The system then provides this information through a user interface, enabling users to navigate or highlight these points during playback. The interface may include visual markers, annotations, or interactive controls that guide the user's attention or allow direct interaction with the plenoptic content. The device also includes a rendering system that displays the plenoptic video data on one or more output devices, such as displays or virtual reality headsets. The rendering system adjusts the presentation based on the points of interest, such as dynamically refocusing the video or adjusting the viewpoint to emphasize these features. This ensures that users can seamlessly explore the plenoptic content while maintaining awareness of key elements. The invention improves user engagement with plenoptic video by making it easier to identify and interact with important features, enhancing applications in fields like virtual reality, medical imaging, and interactive media.
19. The device according to claim 11 , wherein when at least two successive refocusings associated respectively with at least two rendering refocusing planes are performed at two distinct instants, said method comprises generating a sequence of frames whose respective focusing planes are located between said at least two rendering refocusing planes.
This invention relates to a device for generating a sequence of frames with intermediate focusing planes between two distinct refocusing planes. The technology addresses the challenge of creating smooth transitions in focus between different depth planes in imaging systems, particularly for applications like light field displays or computational photography. The device performs at least two successive refocusings, each associated with a distinct rendering refocusing plane, at different times. Between these refocusings, the device generates a sequence of frames where the focusing planes are positioned between the two rendering refocusing planes. This intermediate frame generation ensures a continuous and visually smooth transition in focus, enhancing the perception of depth and realism in the rendered content. The method leverages computational techniques to interpolate or extrapolate the focus positions, allowing for dynamic adjustments in focus without abrupt changes. This approach is particularly useful in systems requiring real-time focus adjustments, such as virtual reality, augmented reality, or advanced imaging devices. The invention improves user experience by providing seamless focus transitions, which is critical for applications demanding high visual fidelity and depth perception.
20. A non-transitory computer-readable storage medium storing program code instructions adapted to perform a method according to claim 1 when executed on a computer.
The invention relates to a computer-implemented method for optimizing data processing in a distributed computing environment. The problem addressed is the inefficiency in resource allocation and task scheduling across multiple computing nodes, leading to suboptimal performance and increased latency. The solution involves dynamically adjusting computational workloads based on real-time system metrics, such as node availability, network bandwidth, and processing capacity. The method includes monitoring these metrics, predicting future resource demands, and redistributing tasks to balance the load across the network. It also incorporates fault tolerance mechanisms to handle node failures or network disruptions without interrupting ongoing processes. The system further includes a feedback loop to refine predictions and adjustments over time, improving overall efficiency. The invention is implemented via program code stored on a non-transitory computer-readable medium, which, when executed, performs the described method. This approach ensures optimal resource utilization, reduces processing delays, and enhances system reliability in distributed computing environments.
Unknown
March 3, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.