Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A computer implemented method for warping virtual content, comprising: a compositor receiving first virtual content based on a first pose of a display system; the compositor receiving second virtual content based on a second pose of the display system; and the compositor processing the first and second virtual content in a single pass in a single pipeline, wherein processing the first and second virtual content includes the compositor selecting a first pixel of the first virtual content to correspond to an output pixel location, the compositor warping the first pixel of the first virtual content based on a third pose of the display system to generate a first warped pixel, the compositor selecting a second pixel of the second virtual content to correspond to the output pixel location, the compositor warping the second pixel of the second virtual content based on the third pose of the display system to generate a second warped pixel, and the compositor comparing a first depth of the first warped pixel and a second depth of the second warped pixel to identify a display pixel at the output pixel location.
This invention relates to virtual reality (VR) or augmented reality (AR) systems, specifically addressing the challenge of efficiently rendering and warping multiple layers of virtual content in real-time to match the dynamic pose of a display system, such as a head-mounted display (HMD). The method involves a compositor that receives first and second virtual content, each generated based on different poses of the display system. The compositor processes these layers in a single computational pass through a unified pipeline. For each output pixel location, the compositor selects corresponding pixels from the first and second virtual content, warps them based on a current (third) pose of the display system, and compares their depths to determine which pixel should be displayed. This approach ensures accurate and efficient rendering of multiple virtual content layers, accounting for depth-based occlusion and pose changes without requiring separate processing passes for each layer. The technique optimizes performance by reducing latency and computational overhead in VR/AR systems.
2. The method of claim 1 , wherein the first virtual content comprises first image information at the first pixel.
3. The method of claim 2 , wherein the first image information comprises a first brightness.
A system and method for image processing involves analyzing image data to enhance visual quality. The technology addresses the challenge of improving image clarity and detail, particularly in low-light or high-contrast environments. The method captures a first image with a first brightness level, which may be adjusted to optimize visibility of specific features. The system then processes this image to extract relevant information, such as edges, textures, or other distinguishing characteristics. This processed data is used to generate a second image with improved visual quality, ensuring better contrast, sharpness, or brightness. The method may also involve comparing the first and second images to refine the processing steps, ensuring accurate and reliable results. The system can be applied in various fields, including medical imaging, surveillance, and consumer electronics, where high-quality image output is critical. The invention enhances image clarity by dynamically adjusting brightness and processing parameters to adapt to different lighting conditions and image content.
4. The method of claim 2 , wherein the first image information comprises a first color.
5. The method of claim 2 , wherein the second virtual content comprises second image information at the second pixel.
A system and method for enhancing digital image processing involves generating and displaying virtual content overlaid on a captured image. The technology addresses the challenge of accurately aligning and rendering virtual content in real-world scenes, particularly in augmented reality (AR) applications, where misalignment or visual artifacts can degrade user experience. The method includes capturing an image of a real-world scene using an imaging device, such as a camera, and identifying a first pixel in the captured image corresponding to a specific location in the scene. Virtual content, such as a graphical overlay or digital object, is generated and positioned at the first pixel. The virtual content includes first image information, which may consist of color, texture, or other visual data, to ensure seamless integration with the real-world scene. Additionally, the method involves generating a second virtual content at a second pixel in the captured image. This second virtual content includes second image information at the second pixel, allowing for dynamic adjustments and enhancements to the displayed content. The second image information may differ from the first, enabling variations in visual effects, depth perception, or interactive elements. The system ensures that the virtual content is accurately aligned with the real-world scene, improving realism and user engagement in AR applications. The method may also include adjusting the virtual content based on changes in the real-world scene, such as movement or lighting variations, to maintain visual coherence.
6. The method of claim 5 , wherein the second image information comprises a second brightness.
7. The method of claim 5 , wherein the second image information comprises a second color.
A system and method for image processing involves capturing and analyzing visual data to enhance or modify image characteristics. The technology addresses challenges in accurately representing or altering image features, particularly in applications requiring precise color or texture adjustments. The method includes capturing a first image with initial color information and processing it to extract specific visual attributes. A second image is then generated or modified to include a second color, which may differ from the first image's color. This second color is applied to enhance or replace portions of the first image, improving visual quality or enabling specific visual effects. The process may involve comparing color data between the images, adjusting color values, or blending visual elements to achieve a desired output. The system can be used in applications such as digital photography, medical imaging, or computer vision, where accurate color representation or modification is critical. The method ensures that the second image's color is applied effectively, maintaining visual coherence or achieving a targeted aesthetic.
8. The method of claim 1 , wherein comparing the first depth of the first warped pixel and the second depth of the second warped pixel comprises: determining the first depth of the first warped pixel based on the third pose of the display system; determining the second depth of the second warped pixel based on the third pose of the display system; and comparing the first and second depths based on the third pose of the display system.
9. The method of claim 1 , wherein warping the first pixel of the first virtual content based on the third pose of the display system comprises applying a first transformation to the first pixel of the first virtual content to generate the first warped pixel.
10. The method of claim 1 , wherein warping the second pixel of the second virtual content based on the third pose of the display system comprises applying a second transformation to the second pixel of the second virtual content to generate the second warped pixel.
This invention relates to virtual content rendering in display systems, particularly addressing the challenge of accurately aligning virtual content with a user's perspective as the display system moves. The method involves dynamically adjusting virtual content to compensate for changes in the display system's pose, ensuring proper alignment with real-world objects. The process includes warping pixels of virtual content based on the display system's pose to maintain visual coherence. Specifically, a second transformation is applied to a second pixel of the second virtual content to generate a second warped pixel, correcting for the display system's third pose. This transformation ensures that the virtual content remains properly aligned with the user's viewpoint, even as the display system moves. The method may also involve similar transformations for other pixels and poses, ensuring consistent alignment across the entire virtual content. The invention improves the realism and stability of augmented reality (AR) or virtual reality (VR) displays by dynamically adjusting virtual content in real-time to match the display system's orientation and position. This reduces visual artifacts and enhances the user experience by maintaining accurate spatial registration between virtual and real-world elements.
11. The method of claim 1 , further comprising the compositor receiving the first virtual content from a first application.
12. The method of claim 11 , further comprising the compositor receiving the second virtual content from a second application different from the first application.
13. The method of claim 1 , wherein the compositor comprises a raster operations unit.
14. The method of claim 1 , further comprising: the compositor writing the display pixel to a memory; the compositor sending the display pixel to a display unit; and the display unit displaying an image to a user based on the display pixel.
15. The method of claim 14 , wherein the display unit is a projector.
16. The method of claim 1 , wherein the first virtual content is a pixel of a first virtual object.
17. The method of claim 1 , wherein the second virtual content is a pixel of a second virtual object.
A system and method for displaying virtual content in an augmented reality environment addresses the challenge of accurately aligning and rendering virtual objects with real-world scenes. The invention involves capturing a real-world scene using a camera, generating a depth map of the scene, and determining a pose of the camera relative to the real-world scene. A first virtual object is rendered based on the depth map and camera pose, and a second virtual object is rendered as a pixel within the first virtual object. The second virtual object is positioned and scaled relative to the first virtual object to ensure proper alignment with the real-world scene. The rendered virtual objects are then displayed on a display device, such as a head-mounted display, to create a seamless augmented reality experience. The method ensures that virtual content appears correctly positioned and scaled in the real-world environment, enhancing the realism and usability of augmented reality applications. The invention is particularly useful in applications requiring precise alignment of virtual objects with real-world scenes, such as gaming, navigation, and industrial training.
18. A system for warping virtual content, comprising: a warping unit to receive first and second virtual content generated based on first and second poses of a display device, and generate respective first and second warped virtual content therefrom, the warping unit comprising: a pose estimator to predict a third pose of the display device; and a transform unit; and a compositing unit comprising a blending unit, wherein the system processes the first and second virtual content in a single pass in a single pipeline including the warping unit selecting a first pixel of the first virtual content to correspond to an output pixel location, the warping unit warping the first pixel of the first virtual content based on the third pose of the display device to generate a first warped pixel, the warping unit selecting a second pixel of the second virtual content to correspond to the output pixel location, the warping unit warping the second pixel of the second virtual content based on the third pose of the display device to generate a second warped pixel, the compositing unit comparing a first depth of the first warped pixel and a second depth of the second warped pixel to identify a display pixel at the output pixel location.
19. The system of claim 18 , the warping unit further comprising a database, wherein the system processing the first and second virtual content in the single pass in the single pipeline includes the compositing unit writing the display pixel to the database, the compositing unit sending the display pixel to the display device, and the display device displaying an image to a user based on the display pixel.
This invention relates to a system for processing and displaying virtual content, addressing the challenge of efficiently rendering and compositing multiple virtual content streams in real-time. The system includes a warping unit that processes first and second virtual content streams in a single computational pass through a unified pipeline, eliminating the need for separate processing stages. The warping unit further includes a database that stores display pixels generated during the compositing process. A compositing unit writes these display pixels to the database and simultaneously sends them to a display device. The display device then renders an image for a user based on the processed display pixels. This approach ensures synchronized and efficient rendering of multiple virtual content streams, reducing latency and computational overhead. The system is particularly useful in applications requiring real-time display of multiple virtual content sources, such as augmented reality, virtual reality, or multi-stream video processing. The database allows for temporary storage and retrieval of display pixels, enabling further processing or adjustments if needed. The unified pipeline and single-pass processing enhance performance by minimizing data transfer and synchronization delays between components.
20. A computer program product embodied in a non-transitory computer readable medium, the computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for warping virtual content, the method comprising: a compositor receiving first virtual content based on a first pose of a display system; the compositor receiving second virtual content based on a second pose of the display system; and the compositor processing the first and second virtual content in a single pass in a single pipeline, wherein processing the first and second virtual content includes the compositor selecting a first pixel of the first virtual content to correspond to an output pixel location, the compositor warping the first pixel of the first virtual content based on a third pose of the display system to generate a first warped pixel, the compositor selecting a second pixel of the second virtual content to correspond to the output pixel location, the compositor warping the second pixel of the second virtual content based on the third pose of the display system to generate a second warped pixel, and the compositor comparing a first depth of the first warped pixel and a second depth of the second warped pixel to identify a display pixel at the output pixel location.
Unknown
March 30, 2021
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.