Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A system for depth sensing, comprising a processor to: receive image data from a communication camera and an augmented reality (AR) camera; detect one or more modulated lights in the image data from the communication camera using a composite waveform modulation; generate a visual representation of a local image region for each of the detected modulated lights; match the visual representation for each of the detected modulated lights with a region in the image data received from the AR camera; and estimate a distance between a dual camera receiver and the one or more modulated lights based on a disparity between a position of the visual representation and a position of the matched region in the image data.
A depth sensing system uses a processor to analyze image data from two cameras—a communication camera and an augmented reality (AR) camera—to estimate distances to modulated light sources. The system detects modulated lights in the communication camera's image data using composite waveform modulation techniques. For each detected light, the processor generates a visual representation of the surrounding image region. These representations are then matched with corresponding regions in the AR camera's image data. The system estimates the distance to the modulated lights by calculating the disparity between the positions of the visual representations in the communication camera's data and their matched regions in the AR camera's data. This approach leverages stereo vision principles to determine depth information from the positional differences between the two camera views. The system is designed to enhance depth perception in environments where modulated light sources are present, improving applications such as augmented reality, robotics, and spatial mapping. The use of composite waveform modulation ensures accurate detection of modulated lights, while the matching process enables precise depth estimation.
2. The system of claim 1 , wherein the processor is to modify an AR visualization based on the estimated distance.
A system for augmented reality (AR) visualization adjusts the display of virtual objects in real-time based on the estimated distance between a user and those objects. The system includes a processor that receives sensor data, such as from cameras or depth sensors, to determine the spatial relationship between the user and the virtual objects. The processor then modifies the AR visualization by scaling, repositioning, or altering the appearance of the virtual objects to enhance realism and usability. For example, as the user moves closer to a virtual object, the system may increase its size or adjust its perspective to simulate depth perception. Conversely, if the user moves farther away, the system may reduce the object's size or adjust its transparency to maintain visual coherence. This dynamic adjustment ensures that the AR experience remains intuitive and immersive, addressing the challenge of maintaining accurate spatial awareness in AR environments. The system may also incorporate additional context, such as environmental factors or user interactions, to further refine the visualization. By dynamically adapting the AR display based on distance, the system improves user engagement and interaction accuracy in AR applications.
3. The system of claim 2 , wherein the AR visualization comprises an overlay corresponding to an object comprising a modulated light, wherein the overlay is to be displayed based on the estimated distance.
Augmented reality (AR) systems enhance real-world environments by overlaying digital information. A challenge in AR is accurately displaying virtual objects in relation to real-world distances, ensuring proper alignment and depth perception. This invention addresses this by providing an AR system that estimates the distance to a real-world object and adjusts the display of an AR overlay accordingly. The overlay corresponds to an object emitting modulated light, which helps in tracking and aligning the virtual content with the real-world scene. The system dynamically adjusts the overlay's position, size, or other visual properties based on the estimated distance, improving the user's perception of the virtual object's placement in the real world. This ensures that the AR visualization appears correctly scaled and positioned relative to the real-world object, enhancing the overall AR experience. The modulated light from the object aids in precise distance estimation, enabling accurate overlay placement. This approach is particularly useful in applications requiring high precision, such as industrial AR, navigation, or interactive displays.
4. The system of claim 3 , comprising an augmented reality (AR) headset to display the AR visualization.
This invention relates to augmented reality (AR) systems for enhancing user interaction with physical environments. The system addresses the challenge of providing users with real-time, contextually relevant visual information overlaid on their physical surroundings, improving situational awareness and decision-making. The system includes an AR headset that displays an AR visualization, which overlays digital content onto the user's view of the real world. The AR visualization is generated based on data from one or more sensors, such as cameras, depth sensors, or environmental sensors, which capture information about the physical environment. The system processes this sensor data to identify relevant objects, surfaces, or conditions in the environment and determines appropriate digital content to display in relation to those elements. The AR headset may include a display, such as a transparent or semi-transparent screen, that allows the user to see both the real-world environment and the overlaid digital content simultaneously. The system may also incorporate user input mechanisms, such as gestures, voice commands, or gaze tracking, to enable interaction with the AR visualization. Additionally, the system may include a processing unit that executes algorithms to analyze sensor data, generate AR content, and render it in real time. The AR visualization may include annotations, labels, navigation cues, or other contextual information to assist the user in tasks such as navigation, object identification, or maintenance. The system may also support multi-user collaboration, allowing multiple users to share and interact with the same AR visualization in a shared environment. The invention aims to provide an intuitive and immersive way to enhance user perception and intera
5. The system of claim 1 , wherein the visual representation comprises a pixel patch generated using a local binary pattern.
A system for visual data processing generates a pixel patch as a visual representation of image data using a local binary pattern. The system captures image data from a scene using an imaging device, such as a camera, and processes the data to extract features. The pixel patch is created by comparing pixel intensities within a local neighborhood of the image, converting the intensity differences into binary values, and encoding these values into a pattern. This binary pattern represents local texture or structural information in the image. The system may further analyze the pixel patch to detect objects, classify scenes, or enhance image quality. The local binary pattern method is computationally efficient and robust to variations in lighting and noise, making it suitable for real-time applications in surveillance, medical imaging, or autonomous navigation. The system may integrate additional processing steps, such as noise reduction or feature extraction, to improve accuracy. The pixel patch can be used as input for machine learning models or other analytical tools to derive insights from the visual data.
6. The system of claim 1 , wherein the modulated lights comprise light emitting diode (LED) tags displaced onto an object.
This invention relates to a system for tracking or identifying objects using modulated light signals, specifically light emitting diode (LED) tags attached to or embedded in objects. The system addresses the challenge of accurately and efficiently monitoring objects in dynamic environments, such as logistics, manufacturing, or inventory management, where traditional tracking methods like barcodes or RFID may be unreliable or impractical. The LED tags emit modulated light signals that encode unique identifiers or status information about the objects they are attached to. These signals are detected and decoded by a sensing device, such as a camera or photodetector, which processes the light patterns to determine the object's identity, location, or other relevant data. The modulation of the light allows for high-speed data transmission and reduces interference from ambient light sources. The system may include multiple LED tags distributed across different objects, each emitting distinct modulated signals to enable simultaneous tracking of multiple items. The tags can be powered by batteries, energy harvesting, or wired connections, depending on the application. The sensing device may use image processing, signal filtering, or machine learning techniques to accurately decode the modulated light signals, even in low-light or cluttered environments. This approach improves upon existing tracking technologies by providing real-time, high-precision object identification without requiring direct line-of-sight or physical contact, making it suitable for automated systems and industrial applications.
7. The system of claim 6 , wherein the modulated lights are to communicate information corresponding to the object.
A system for communicating information about an object using modulated light signals. The system includes a light source configured to emit modulated light, where the modulation encodes data related to the object. This data may include identification, status, or other relevant attributes of the object. The system also includes a sensor or receiver capable of detecting the modulated light and decoding the embedded information. The light source may be integrated with or attached to the object, allowing the object to transmit its information wirelessly via light signals. The modulation technique can vary, including techniques like pulse-width modulation, amplitude modulation, or frequency modulation, depending on the application requirements. The system may also include processing components to manage the modulation and demodulation processes, ensuring accurate and reliable data transmission. This approach enables contactless and efficient information exchange, particularly useful in environments where radio frequency or other wireless methods may be impractical or restricted. The system can be applied in various fields, such as inventory management, asset tracking, or smart environments, where objects need to communicate their status or identity without physical contact or complex infrastructure.
8. The system of claim 1 , wherein the communication camera and the AR camera are disposed in a stereo setup position with a predetermined distance between the communication camera and the AR camera, wherein the processor is to further estimate the distance based on the predetermined distance between the communication camera and the AR camera.
This invention relates to a system combining augmented reality (AR) and communication cameras in a stereo setup to enhance distance estimation in AR applications. The system addresses the challenge of accurately determining distances in AR environments, particularly for applications requiring precise spatial awareness, such as virtual object placement or interactive AR experiences. The system includes at least one communication camera and at least one AR camera positioned in a stereo configuration with a fixed, predetermined distance between them. The stereo setup allows the system to capture depth information by analyzing disparities between the images from the two cameras. A processor within the system processes these images to estimate distances to objects or surfaces in the AR environment. The predetermined distance between the cameras serves as a reference for triangulation, improving the accuracy of depth perception. The system may also include additional components, such as a display for rendering AR content and sensors for tracking user movements. The processor integrates data from the cameras and other sensors to generate a coherent AR experience, where virtual objects are accurately positioned relative to real-world elements. This setup ensures that AR interactions remain stable and realistic, even in dynamic environments. The invention is particularly useful in applications requiring high-precision spatial mapping, such as industrial AR, gaming, or remote collaboration tools.
9. The system of claim 1 , wherein the processor is to detect the one or more modulated lights using an under-sampled orthogonal frequency shift on-off keying modulation.
A system for detecting modulated light signals in optical communication or sensing applications addresses challenges in accurately capturing and decoding light-based data, particularly in environments with noise or interference. The system includes a processor configured to detect one or more modulated light signals using an under-sampled orthogonal frequency shift on-off keying (OFS-OOK) modulation technique. OFS-OOK combines orthogonal frequency shift keying (OFSK) with on-off keying (OOK) to encode data in both frequency and amplitude, improving robustness against interference. The under-sampling approach reduces computational complexity by processing signals at a rate lower than the Nyquist rate, while still preserving critical signal characteristics. The processor may also include components for signal preprocessing, such as filtering or amplification, to enhance detection accuracy. This system is particularly useful in applications like visible light communication (VLC), optical wireless networks, or industrial sensing, where reliable and efficient light-based data transmission is required. The under-sampled OFS-OOK modulation allows for high data rates with reduced power consumption and hardware complexity, making it suitable for resource-constrained environments.
10. A method for depth sensing, comprising: receiving, via a processor, image data from a communication camera and an augmented reality (AR) camera; detecting, via the processor, one or more modulated lights in the image data from the communication camera; generating, via the processor, a visual representation of a local image region for each of the detected modulated lights, wherein generating the visual representation comprises generating a pixel patch of the one or more modulated lights using a local binary pattern; matching, via the processor, the visual representation for each of the detected modulated lights with a region in the image data received from the AR camera; estimating, via the processor, a distance between a dual camera receiver and the one or more modulated lights; and modifying, via the processor, an augmented reality (AR) visualization based on the estimated distance.
This invention relates to depth sensing for augmented reality (AR) applications, addressing the challenge of accurately determining distances between a dual-camera system and modulated light sources in an environment. The method involves a processor receiving image data from both a communication camera and an AR camera. The processor detects modulated lights in the communication camera's image data and generates a visual representation of local image regions for each detected light. This representation is created by producing a pixel patch of the modulated lights using a local binary pattern, which captures spatial relationships between pixels. The processor then matches these visual representations with corresponding regions in the AR camera's image data. Using these matches, the processor estimates the distance between the dual-camera receiver and the modulated lights. Finally, the processor modifies the AR visualization based on this estimated distance, enabling accurate depth perception and interaction in AR environments. The technique leverages modulated light detection and pattern matching to enhance spatial awareness in AR applications.
11. The method of claim 10 , wherein estimating the distance comprises calculating a disparity in pixel locations of the modulated lights and estimating the distance based on the disparity, a size of pixels, and a focal length of the AR camera and the communication camera.
This invention relates to augmented reality (AR) systems that use modulated light signals for distance estimation. The problem addressed is accurately determining the distance between an AR device and a target object in real-time, which is essential for precise AR overlays and interactions. The solution involves using modulated light signals emitted by a communication camera and captured by an AR camera to estimate distance based on pixel disparity. The method involves emitting modulated light signals from a communication camera, capturing the modulated light signals with an AR camera, and estimating the distance to the target object by calculating the disparity in pixel locations of the modulated lights. The distance is then determined based on the disparity, the size of the pixels, and the focal length of both the AR camera and the communication camera. This approach leverages the known optical properties of the cameras to derive accurate distance measurements without requiring additional hardware, improving the efficiency and accuracy of AR applications. The technique is particularly useful in scenarios where precise spatial awareness is critical, such as in AR navigation, object tracking, and interactive AR environments.
12. The method of claim 10 , wherein modifying the AR visualization comprises increasing a size of an overlay in the AR visualization based on the estimated distance.
Augmented reality (AR) systems enhance real-world environments with digital overlays, but accurately aligning these overlays with physical objects remains challenging, especially when user movement or environmental factors affect perception. This invention addresses the problem by dynamically adjusting AR visualizations based on estimated distances to improve alignment and user experience. The method involves tracking a user's position and orientation relative to a physical object in the real world. Sensors, such as cameras or depth sensors, capture environmental data, which is processed to estimate the distance between the user and the object. The AR system then modifies the visualization by scaling an overlay—such as a digital label, annotation, or 3D model—proportionally to the estimated distance. For example, as the user moves closer to the object, the overlay size increases to maintain visual consistency and alignment with the physical object. Conversely, as the user moves farther away, the overlay size decreases to prevent distortion or misalignment. This dynamic scaling ensures that AR content remains accurately positioned relative to the real-world environment, enhancing usability in applications like navigation, maintenance, or interactive training. The method may also incorporate additional adjustments, such as adjusting opacity or perspective, to further refine the visualization based on distance and viewing angle.
13. The method of claim 10 , wherein modifying the AR visualization comprises including an overlay in the visualization based on the estimated distance.
Augmented reality (AR) systems enhance real-world views with digital overlays, but accurately placing these overlays in 3D space remains challenging. Existing AR systems often struggle to align virtual objects with real-world environments due to imprecise distance estimation, leading to misaligned or unrealistic visualizations. This invention addresses the problem by dynamically modifying AR visualizations based on estimated distances to real-world objects or surfaces. The method involves capturing real-world data, such as images or depth information, to determine the distance between a user and a target object or surface. The AR system then adjusts the visualization by incorporating an overlay that accounts for this estimated distance. For example, if the system detects a wall at a specific distance, it may render a virtual object or annotation at the correct depth to appear seamlessly integrated with the real-world scene. The overlay can include visual cues, such as depth indicators or contextual information, to enhance user perception of spatial relationships. This approach improves the realism and usability of AR applications in fields like navigation, maintenance, or training, where accurate spatial alignment is critical. By dynamically adjusting the visualization based on real-time distance measurements, the system ensures that AR content appears correctly positioned relative to the physical environment.
14. The method of claim 10 , wherein estimating the distance comprises calculating a three-dimensional world position of the AR camera or the communication camera using triangulation.
This invention relates to augmented reality (AR) systems that enhance real-world environments with digital content. A key challenge is accurately determining the distance between a user's AR camera and a communication camera to properly overlay digital content in the real world. The invention addresses this by estimating the distance using triangulation to calculate a three-dimensional world position of either the AR camera or the communication camera. Triangulation involves using known reference points and angles to determine precise spatial coordinates, enabling accurate placement of AR content relative to the real-world environment. The method leverages the geometric relationships between the cameras and reference points to compute the distance, ensuring that digital overlays appear correctly aligned with physical objects. This approach improves the accuracy and realism of AR experiences by reducing misalignment errors that can occur with less precise distance estimation techniques. The invention is particularly useful in applications where precise spatial awareness is critical, such as interactive AR displays, navigation systems, or collaborative AR environments. By calculating the three-dimensional position through triangulation, the system ensures that digital content is seamlessly integrated into the user's view of the real world.
15. The method of claim 10 , wherein estimating the distance comprises calculating a three-dimensional world position of the AR camera or the communication camera based on two points and one or more constraints.
This invention relates to augmented reality (AR) systems that estimate distances using camera positioning. The problem addressed is accurately determining the spatial relationship between an AR camera and a communication camera in a 3D environment, which is essential for overlaying virtual objects realistically. The method involves calculating a three-dimensional world position of either the AR camera or the communication camera based on two points and one or more constraints. The two points may be derived from features detected in the environment, such as markers or natural landmarks, while the constraints could include known distances, angles, or other geometric relationships. By leveraging these inputs, the system refines the camera's position to improve AR object placement accuracy. This approach enhances the precision of AR applications, such as virtual object interaction or spatial mapping, by reducing errors in depth perception and alignment. The method is particularly useful in dynamic environments where camera movement or environmental changes could otherwise degrade performance. The solution integrates geometric calculations with real-time sensor data to provide reliable positioning for AR experiences.
16. The method of claim 10 , wherein generating the visual representation comprises generating the pixel patch of the one or more modulated lights for each of the AR camera and the communication camera.
This invention relates to augmented reality (AR) systems that use modulated light for communication between devices. The problem addressed is the need for efficient and reliable data transmission in AR environments, where visual information must be processed by both an AR camera and a communication camera. The solution involves generating a visual representation of modulated light signals that can be captured and decoded by both cameras simultaneously. The visual representation includes a pixel patch for each camera, ensuring that the communication data is accurately received by both devices. The modulated light signals are encoded with information, such as device identifiers or synchronization data, and the pixel patches are generated to optimize visibility and decoding accuracy for each camera. This approach enables seamless communication between AR devices while maintaining the integrity of the visual content displayed to the user. The method ensures that the modulated light signals are properly aligned and synchronized for both cameras, improving the reliability of data transmission in AR applications.
17. The method of claim 10 , wherein generating the visual representation comprises generating pixel patches using a neural network trained to match patches between the AR camera and the communication camera based on generated comparison values.
This invention relates to augmented reality (AR) systems that integrate visual data from multiple cameras, particularly an AR camera and a communication camera, to enhance user experience. The problem addressed is the difficulty in accurately aligning and blending visual data from different cameras in real-time AR applications, which can lead to misalignment, artifacts, or poor visual quality. The method involves generating a visual representation by processing data from both cameras. Specifically, it uses a neural network to generate pixel patches that align corresponding regions between the AR camera and the communication camera. The neural network is trained to compare and match these patches based on generated comparison values, ensuring accurate alignment. This approach improves visual coherence and reduces discrepancies between the two camera feeds, enhancing the overall AR experience. The neural network may be trained using supervised or unsupervised learning techniques, depending on the application. The method can be applied in various AR scenarios, such as virtual object placement, real-time video conferencing, or environmental mapping. By dynamically adjusting the alignment of pixel patches, the system ensures seamless integration of visual data from multiple sources, improving accuracy and user satisfaction.
18. The method of claim 10 , wherein detecting the one or more modulated lights and generating the visual representation is performed in parallel.
A method for processing modulated light signals in a visual display system addresses the challenge of efficiently detecting and interpreting modulated light while generating a visual representation. The system involves capturing modulated light signals, which may be emitted from a light source or reflected from an object, and analyzing these signals to extract information. The extracted information is then used to generate a visual representation, such as an image or graphical output, that corresponds to the detected light patterns. A key aspect of this method is the parallel processing of signal detection and visual representation generation, ensuring real-time or near-real-time performance. This parallel operation enhances system responsiveness and reduces latency, making it suitable for applications requiring immediate feedback, such as augmented reality, object tracking, or environmental sensing. The method may also include preprocessing steps to filter noise or enhance signal clarity before analysis, as well as post-processing to refine the visual output. By integrating these steps, the system provides an efficient and accurate way to interpret modulated light signals and display relevant information.
19. At least one non-transitory computer readable medium for depth sensing having instructions stored therein that, in response to being executed on a computing device, cause the computing device to: receive image data from a communication camera and an augmented reality (AR) camera; detect one or more modulated lights in the image data from the communication camera; generate a visual representation of a local image region using a local binary pattern for each of the detected modulated lights; match the visual representation for each of the detected modulated lights with a region in the image data received from the AR camera; estimate a distance between a dual camera receiver and the one or more modulated lights; and modify an augmented reality (AR) visualization based on the estimated distance.
This invention relates to depth sensing for augmented reality (AR) applications using dual-camera systems. The problem addressed is accurately determining the distance between a device and objects in an environment to enhance AR visualizations. The solution involves a non-transitory computer-readable medium storing instructions that, when executed, enable a computing device to process image data from both a communication camera and an AR camera. The system detects modulated lights in the communication camera's image data and generates a visual representation of each light using a local binary pattern. These representations are then matched with corresponding regions in the AR camera's image data. By analyzing these matches, the system estimates the distance between the dual-camera receiver and the modulated lights. This distance information is used to dynamically adjust the AR visualization, improving its realism and interaction with the physical environment. The approach leverages structured light patterns and binary pattern matching to achieve precise depth sensing, which is critical for applications like AR overlays, object tracking, and spatial mapping. The system enhances AR experiences by providing accurate depth perception, enabling more immersive and context-aware visualizations.
20. The at least one non-transitory computer readable medium of claim 19 , comprising instructions to calculate a disparity in pixel locations of the modulated lights and estimate the distance based on the disparity, a size of pixels, and a focal length of the AR camera and the communication camera.
This invention relates to augmented reality (AR) systems that use modulated light sources and cameras to estimate distances in a scene. The problem addressed is accurately determining distances to objects in an AR environment without requiring specialized hardware or complex calibration processes. The system uses at least one non-transitory computer-readable medium containing instructions to process images captured by an AR camera and a communication camera. The instructions calculate the disparity between pixel locations of modulated light sources in images from both cameras. The distance to the light sources is then estimated based on this disparity, the size of the camera pixels, and the focal lengths of both cameras. This approach leverages existing AR hardware to provide depth information without additional sensors, improving accuracy and reducing system complexity. The method is particularly useful in AR applications where precise spatial mapping is required, such as object placement, interaction, or navigation. The system may also include instructions to adjust the modulation of the light sources to enhance detection accuracy under varying environmental conditions. By combining multiple cameras and analyzing modulated light patterns, the invention enables robust distance estimation in dynamic AR environments.
21. The at least one non-transitory computer readable medium of claim 19 , comprising instructions to display content in an overlay of the AR visualization based on the estimated distance.
This invention relates to augmented reality (AR) systems that enhance user interaction by dynamically displaying content in an AR visualization based on estimated distances. The technology addresses the challenge of providing contextually relevant information in AR environments, where static overlays may not effectively convey spatial relationships or distance-based data. The system includes a computing device with a display and at least one sensor, such as a camera or depth sensor, to capture environmental data. The device processes this data to estimate distances between objects or between a user and objects in the real-world environment. Based on these distance estimates, the system generates an AR visualization that overlays digital content onto the real-world view. The content displayed in the overlay is dynamically adjusted according to the estimated distances, ensuring that information remains relevant and spatially coherent. For example, if a user approaches a physical object, the AR system may display additional details or annotations about that object in the overlay. Conversely, as the user moves away, the system may simplify or reduce the displayed content to avoid clutter. The dynamic adjustment ensures that the AR visualization remains intuitive and useful, adapting to the user's changing perspective and position. The invention improves AR applications by making them more responsive to real-world spatial relationships, enhancing usability in navigation, education, maintenance, and other fields where distance-based context is critical.
22. The at least one non-transitory computer readable medium of claim 19 , comprising instructions to increase a size of an overlay in the AR visualization based on the estimated distance.
This invention relates to augmented reality (AR) systems that dynamically adjust the size of an overlay in an AR visualization based on the estimated distance to a target object. The technology addresses the challenge of maintaining visual clarity and usability in AR environments where objects may appear at varying distances, making static overlays difficult to perceive or interact with effectively. The system estimates the distance to a target object in the real-world environment and adjusts the size of the AR overlay proportionally to ensure it remains legible and appropriately scaled for the user's perspective. This dynamic sizing helps prevent overlays from appearing too small or too large, improving user experience and interaction accuracy. The invention may also include additional features such as adjusting the overlay's position, opacity, or other visual properties based on the estimated distance to further enhance usability. The system is designed to work with AR devices like head-mounted displays or mobile devices equipped with cameras and sensors for distance estimation. By automatically scaling the overlay, the invention ensures that AR content remains visually coherent and contextually relevant, regardless of the user's distance from the target object.
Unknown
March 10, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.