Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method of determining a correspondence between a secondary device and a head-mounted device (HMD) physically unassociated with the secondary device, the method comprising: at the HMD with a processor, non-transitory memory, and an event camera including a plurality of pixel sensors disposed on the event camera at known locations relative to an HMD reference frame: obtaining light intensity data indicative of an intensity of light incident on the plurality of pixel sensors from a stream of pixel events output by the event camera, at least a portion of the light is emitted by a plurality of optical sources disposed on the secondary device at known locations relative to a secondary device reference frame, each respective pixel event is generated in response to a particular pixel sensor detecting a change in light intensity that exceeds a comparator threshold; identifying a set of optical sources among the plurality of optical sources that are visible to the event camera by recognizing defined illumination parameters associated with the set of optical sources using the light intensity data; generating location data in the HMD reference frame for the set of optical sources, the location data in the HMD reference frame generated using the light intensity data; and determining the correspondence between the secondary device and the HMD by mapping the location data in the HMD reference frame to respective known locations of optical sources of the set of optical sources relative to the secondary device reference frame.
This invention relates to determining the spatial correspondence between a secondary device and a head-mounted device (HMD) that are not physically connected. The problem addressed is establishing accurate positional relationships between such devices without direct physical association, which is critical for applications like augmented reality, robotics, or collaborative systems where precise spatial awareness is required. The method uses an event camera on the HMD, which includes multiple pixel sensors arranged at known positions relative to the HMD's reference frame. The event camera captures light intensity changes, generating pixel events when detected light intensity changes exceed a set threshold. The secondary device has multiple optical sources at known positions relative to its own reference frame. The HMD's processor analyzes the light intensity data from the event camera to identify which optical sources are visible. By recognizing predefined illumination parameters, the system determines the visible optical sources and generates their locations in the HMD's reference frame. The correspondence between the devices is then established by mapping these locations to the known positions of the optical sources relative to the secondary device's reference frame. This approach enables precise spatial alignment without physical connection, improving accuracy in dynamic environments.
2. The method of claim 1 further comprising: updating a display of the HMD based on the determined correspondence between the secondary device and the HMD.
This invention relates to head-mounted display (HMD) systems and methods for improving interaction between an HMD and a secondary device, such as a smartphone or tablet. The problem addressed is the lack of seamless integration between HMDs and secondary devices, which can lead to usability issues, such as misalignment or delays in content synchronization. The method involves determining a spatial correspondence between the HMD and the secondary device, which may include identifying their relative positions, orientations, or distances. This correspondence is used to adjust the display or functionality of the HMD, ensuring that content from the secondary device is properly aligned or synchronized with the HMD's view. For example, if the secondary device is used as a controller or input device, the HMD display may update to reflect the device's position or gestures in real time. The method may also involve tracking the secondary device's movements or interactions and dynamically updating the HMD display based on these inputs. This ensures that the user experience remains consistent and responsive, even as the secondary device moves relative to the HMD. The system may use sensors, such as cameras, accelerometers, or inertial measurement units (IMUs), to establish and maintain the correspondence between the devices. The goal is to provide a more intuitive and immersive interaction between the HMD and secondary devices, enhancing usability in applications such as gaming, virtual reality, or augmented reality.
3. The method of claim 1 , wherein the defined illumination parameters include a first wavelength associated with light emitted by a first subset of optical sources and a second wavelength associated with light emitted by a second subset of optical sources that is different from the first wavelength, the first and second subsets of optical sources among the plurality of optical sources.
This invention relates to a method for controlling illumination parameters in a system with multiple optical sources, such as LEDs or lasers, to achieve precise and adaptable lighting conditions. The problem addressed is the need for dynamic adjustment of illumination properties, such as wavelength, to optimize performance in applications like displays, imaging, or sensing. The method involves defining illumination parameters that include at least two distinct wavelengths. A first wavelength is associated with light emitted by a first subset of optical sources, while a second, different wavelength is associated with light emitted by a second subset of optical sources. These subsets are part of a larger group of optical sources, allowing for selective activation or modulation of specific wavelengths. The method enables independent control of different wavelength groups, facilitating applications requiring multi-spectral or tunable lighting. For example, in display systems, this allows for color mixing or dynamic spectral adjustments, while in sensing applications, it enables selective excitation of different materials or targets. The approach improves flexibility and precision in illumination control compared to systems with fixed or uniform wavelength outputs.
4. The method of claim 1 further comprising: filtering the light intensity data according to a frequency range to partition light intensity data corresponding to a visible wavelength range from light intensity data corresponding to a non-visible wavelength range.
This invention relates to a method for processing light intensity data to distinguish between visible and non-visible wavelength ranges. The method involves capturing light intensity data from a light source or environment, where the data includes contributions from both visible and non-visible wavelengths. The key innovation is the filtering step, which partitions the light intensity data into two distinct components: one corresponding to visible wavelengths (typically 380-750 nm) and another corresponding to non-visible wavelengths (e.g., infrared or ultraviolet). This separation allows for independent analysis or manipulation of the visible and non-visible light components, which can be useful in applications such as imaging, spectroscopy, or environmental monitoring. The filtering may be implemented using digital signal processing techniques, such as Fourier transforms or bandpass filters, to isolate the desired frequency ranges. By isolating visible and non-visible light, the method enables more accurate characterization of light sources or scenes, improving applications like colorimetry, remote sensing, or optical communication. The invention addresses the challenge of distinguishing between different wavelength ranges in light intensity data, which is critical for applications requiring precise spectral analysis.
5. The method of claim 4 , wherein the light intensity data corresponding to the non-visible wavelength range includes the light emitted by the plurality of optical sources.
This invention relates to optical sensing systems that detect light intensity across visible and non-visible wavelength ranges. The problem addressed is accurately capturing light emissions from multiple optical sources, particularly in non-visible spectra, to enable precise measurements or imaging applications. The system includes a plurality of optical sources emitting light in a non-visible wavelength range, such as infrared or ultraviolet. A detector captures light intensity data from these sources, distinguishing between ambient light and the emitted signals. The method involves processing this data to isolate and analyze the light emitted by the optical sources, ensuring accurate detection even in environments with varying background illumination. The system may also incorporate visible light detection, where the same or additional detectors capture intensity data in the visible spectrum. This dual-wavelength approach enhances the system's ability to correlate visible and non-visible light signals, improving accuracy in applications like environmental monitoring, medical imaging, or industrial inspection. The key innovation lies in the method's ability to differentiate and process light intensity data from multiple non-visible optical sources, ensuring reliable signal extraction in complex lighting conditions. This enables applications requiring high-precision detection, such as spectroscopy, remote sensing, or machine vision.
6. The method of claim 4 further comprising: determining a correspondence between an object in a scene disposed within a field of view of the event camera and the HMD using the light intensity data corresponding to the visible wavelength range.
This invention relates to event-based vision systems for head-mounted displays (HMDs) and addresses the challenge of accurately tracking objects in a scene using light intensity data from an event camera. The method involves capturing light intensity data in the visible wavelength range from an event camera, which detects changes in light intensity asynchronously rather than using traditional frame-based imaging. The system processes this data to identify and track objects within the field of view of both the event camera and the HMD. By analyzing the light intensity variations, the method establishes a correspondence between objects in the scene and the HMD, enabling precise spatial mapping and interaction. The technique leverages the high temporal resolution of event cameras to improve object detection and tracking in dynamic environments, particularly where traditional cameras may struggle with motion blur or low-light conditions. The method may also include additional steps such as filtering noise from the light intensity data, enhancing the accuracy of object correspondence. This approach enhances the performance of augmented reality (AR) and virtual reality (VR) applications by providing real-time, high-fidelity object tracking and interaction capabilities.
7. The method of claim 1 , wherein the defined illumination parameters include a portion of an illumination pattern of the plurality of optical sources that corresponds to the set of optical sources.
A system and method for controlling illumination in an optical imaging or sensing application involves dynamically adjusting illumination parameters to optimize performance. The technology addresses challenges in environments where lighting conditions vary or where precise control of illumination is needed to enhance image quality, sensor accuracy, or energy efficiency. The method includes selecting a subset of optical sources from a plurality of optical sources based on predefined criteria, such as target illumination intensity, spatial distribution, or power consumption constraints. The illumination parameters are defined to include a specific portion of an illumination pattern generated by the selected subset of optical sources. This portion of the pattern is tailored to the subset, ensuring that the illumination matches the desired characteristics for the application. The system may further include feedback mechanisms to adjust the illumination in real-time based on sensor data or environmental changes. The method improves efficiency by minimizing unnecessary power usage while maintaining optimal illumination for the task. Applications include machine vision, medical imaging, automotive lighting, and industrial automation.
8. The method of claim 1 , wherein the set of optical sources includes at least three optical sources of the plurality of optical sources.
This invention relates to optical communication systems, specifically methods for managing multiple optical sources to improve signal transmission. The problem addressed is the need for reliable and efficient optical signal distribution in systems where multiple optical sources are used, such as in fiber-optic networks or optical interconnects. The invention provides a method that ensures proper coordination and control of at least three optical sources from a larger set of optical sources to enhance signal integrity and reduce interference. The method involves selecting and activating a subset of optical sources to transmit data while maintaining synchronization and minimizing crosstalk. By using at least three optical sources, the system achieves redundancy and improved signal robustness. The optical sources may be lasers or light-emitting diodes, and the method includes adjusting their output power, wavelength, or modulation to optimize performance. The system may also incorporate feedback mechanisms to dynamically adjust the optical sources based on real-time conditions, such as signal quality or environmental factors. This approach ensures stable and high-speed data transmission in optical networks, making it suitable for applications requiring high bandwidth and low latency.
9. The method of claim 1 , wherein the secondary device controls a computing device that generates graphics data for display on a screen of the HMD, the computing device physically unassociated with the secondary device and the HMD.
This invention relates to a system for controlling a head-mounted display (HMD) using a secondary device, where the secondary device manages graphics data generation for the HMD through an external computing device. The external computing device, which is physically separate from both the secondary device and the HMD, generates graphics data for display on the HMD's screen. The secondary device communicates with the external computing device to control the graphics data generation process, enabling the HMD to display visual content based on user interactions or other inputs processed by the secondary device. This setup allows for flexible and scalable HMD control, where the secondary device acts as an intermediary between the user and the external computing device, which handles the computational workload of rendering graphics. The system may be used in applications such as virtual reality, augmented reality, or other immersive display environments where real-time graphics processing is required. The secondary device may also perform additional functions, such as input processing or system coordination, to enhance the user experience. The external computing device may be a standalone computer, server, or other processing unit capable of generating high-quality graphics data for the HMD.
10. The method of claim 1 , wherein the secondary device provides user input to a computing device that generates graphics data for display on a screen of the HMD, the computing device physically unassociated with the secondary device and the HMD.
This invention relates to a system for interacting with a head-mounted display (HMD) using a secondary device that is physically unassociated with the HMD or the computing device generating graphics for display. The secondary device captures user input, such as gestures or movements, and transmits this input to a computing device that processes the data and generates corresponding graphics. The graphics are then displayed on the HMD screen. The secondary device may include sensors, such as cameras or motion trackers, to detect user actions. The computing device, which is separate from both the secondary device and the HMD, receives the input data, interprets it, and renders visual content based on the user's actions. This setup allows for flexible interaction with the HMD without requiring direct physical connections between the devices. The system enables immersive experiences where the secondary device acts as an intermediary input source, enhancing user interaction with the HMD's displayed content. The invention addresses the need for wireless, untethered input methods in HMD environments, improving usability and reducing physical constraints.
11. The method of claim 1 , wherein the HMD includes a display to present augmented reality content, virtual reality content, or a combination thereof.
A head-mounted display (HMD) system enhances user experience by presenting augmented reality (AR) or virtual reality (VR) content. The HMD includes a display system capable of overlaying digital information onto the real world (AR) or immersing the user in a fully virtual environment (VR). The display may use optical components such as lenses, waveguides, or microdisplays to project content directly into the user's field of view. The system may incorporate sensors, such as cameras or inertial measurement units, to track the user's position and orientation, ensuring accurate alignment of virtual objects with the physical environment in AR mode. In VR mode, the display provides a fully enclosed visual experience, often with high-resolution screens and wide field-of-view optics to minimize motion sickness and improve immersion. The HMD may also include audio components, such as speakers or bone conduction headphones, to enhance the sensory experience. The system may further integrate with external devices, such as smartphones or gaming consoles, to stream or process content. This technology addresses the need for immersive, interactive digital experiences in applications like gaming, training, and remote collaboration.
12. A system comprising: a head-mounted device (HMD) having a plurality of optical sources to emit light, the plurality of optical sources disposed at known locations relative to an HMD reference frame; a secondary device physically unassociated with the HMD comprising an event camera having a plurality of pixel sensors arranged to receive the light emitted by the plurality of optical sources, the event camera to output a stream of pixel events generated by the plurality of pixel sensors, each of the plurality of pixel sensors disposed on the event camera at a known location relative to a secondary device reference frame and adapted to generate a respective pixel event in response to a breach of a respective comparator threshold indicative of an intensity of light incident on a respective pixel sensor; and a control node coupled to the secondary device to determine a correspondence between the HMD and the secondary device by mapping location data generated in a secondary device reference frame for a set of optical sources among the plurality of optical sources to respective known locations of the set of optical sources relative to the HMD reference frame, wherein the set of optical sources are uniquely identified by recognizing defined illumination parameters associated with the set of optical sources in light intensity data obtained from the stream of pixel events.
This invention relates to a system for determining the spatial correspondence between a head-mounted device (HMD) and a secondary device using event-based vision. The HMD includes multiple optical sources positioned at known locations relative to an HMD reference frame. A secondary device, physically separate from the HMD, contains an event camera with pixel sensors arranged to detect light emitted by the HMD's optical sources. The event camera outputs a stream of pixel events, where each event is generated when a pixel sensor detects a change in light intensity exceeding a predefined threshold. The pixel sensors are positioned at known locations relative to a secondary device reference frame. A control node connected to the secondary device processes the event data to establish a correspondence between the HMD and the secondary device. This is achieved by mapping the detected locations of a subset of the HMD's optical sources (as observed by the event camera) to their known positions in the HMD's reference frame. The subset of optical sources is uniquely identified by analyzing their illumination patterns in the event data, ensuring accurate tracking. This system enables precise spatial alignment between the HMD and the secondary device without direct physical connection, useful for augmented reality, motion tracking, or collaborative applications.
13. The system of claim 12 , wherein the secondary device includes an inertial measurement unit (IMU) to provide inertial data corresponding to movement of the secondary device in a global reference frame.
A system for tracking the position and orientation of a secondary device relative to a primary device, such as a handheld controller or wearable device, includes an inertial measurement unit (IMU) in the secondary device to provide inertial data. The IMU measures movement of the secondary device in a global reference frame, allowing for accurate tracking of its position and orientation. This data is used to determine the relative movement between the primary and secondary devices, enabling applications such as motion tracking, gesture recognition, or spatial positioning. The system may also include sensors or markers on the secondary device to enhance tracking accuracy. The IMU provides real-time inertial data, which can be combined with other sensor inputs to improve robustness in dynamic environments. This approach is useful in applications requiring precise motion tracking, such as virtual reality, augmented reality, or industrial automation, where maintaining accurate spatial awareness is critical. The system ensures reliable tracking even in scenarios where direct line-of-sight or external reference points may be limited.
14. The system of claim 13 , wherein the control node is adapted to normalize the correspondence between the HMD and the secondary device to the global reference frame using the inertial data.
The system relates to augmented reality (AR) or virtual reality (VR) environments where a head-mounted display (HMD) and a secondary device, such as a handheld controller or another HMD, interact within a shared space. A common challenge in such systems is maintaining accurate spatial alignment between the HMD and the secondary device, especially when both devices move independently. Misalignment can lead to inconsistencies in the AR/VR experience, such as incorrect object placement or interaction errors. The system includes a control node that processes inertial data from the HMD and the secondary device to establish and maintain a consistent global reference frame. The control node normalizes the positional and rotational data of both devices relative to this global frame, ensuring that their movements and interactions are synchronized. This normalization compensates for drift or misalignment caused by inertial sensor inaccuracies or environmental factors. The system may also incorporate additional sensors, such as cameras or external tracking systems, to enhance accuracy. By dynamically adjusting the correspondence between the HMD and the secondary device, the system ensures a seamless and immersive user experience in AR/VR applications.
15. The system of claim 12 , wherein the secondary device controls a computing device that generates graphics data for display on a screen of the HMD, the computing device physically unassociated with the secondary device and the HMD.
This invention relates to a system for managing graphics data in a head-mounted display (HMD) environment. The system addresses the challenge of efficiently controlling graphics rendering and display in HMDs, particularly when the computing device generating the graphics is physically separate from the HMD and any secondary control devices. The system includes a secondary device that communicates with the HMD and a remote computing device to manage the generation and display of graphics data. The secondary device controls the computing device, which produces the graphics data for display on the HMD screen. The computing device is physically independent of both the secondary device and the HMD, allowing for flexible system configurations. The secondary device may also handle user inputs, coordinate data processing, or manage communication between the HMD and the computing device. This setup enables efficient graphics rendering and display control in HMD systems where the computing hardware is not directly integrated with the display or control devices. The invention improves usability and performance by decoupling the graphics generation process from the physical HMD and control hardware.
16. The system of claim 12 , wherein the secondary device provides user input to a computing device that generates graphics data for display on a screen of the HMD, the computing device physically unassociated with the secondary device and the HMD.
This invention relates to a system for augmented reality (AR) or virtual reality (VR) applications, specifically addressing the challenge of integrating user input from a secondary device into a head-mounted display (HMD) without requiring direct physical or wired connections between the devices. The system includes a secondary device that captures user input, such as gestures, movements, or commands, and transmits this data to a computing device. The computing device, which is physically separate from both the secondary device and the HMD, processes the input and generates corresponding graphics data. This graphics data is then sent to the HMD for display on its screen, enabling real-time interaction in AR or VR environments. The secondary device may be a handheld controller, a wearable sensor, or another input device, and the computing device may be a standalone computer, server, or cloud-based system. The system ensures seamless integration of user input into the immersive experience without the need for direct physical connections, improving flexibility and usability in AR/VR applications.
17. The system of claim 12 , wherein the HMD includes a display to present augmented reality content, virtual reality content, or a combination thereof.
A system for head-mounted displays (HMDs) enhances user interaction with augmented reality (AR) or virtual reality (VR) content. The HMD includes a display capable of presenting AR content, VR content, or a combination of both. This allows users to experience immersive digital environments overlaid on or replacing the real world. The system may also include sensors to track the user's head movements and adjust the displayed content accordingly, ensuring seamless integration between virtual and physical spaces. Additionally, the HMD may incorporate input devices, such as controllers or hand-tracking systems, to enable user interaction with the displayed content. The system may further include processing components to render and manage the AR or VR content in real time, ensuring smooth and responsive user experiences. By combining these features, the system provides an advanced platform for immersive digital experiences, addressing the need for more interactive and engaging AR/VR applications.
18. The system of claim 12 , wherein the plurality of optical sources comprise: an optical source to emit light in a near-infrared wavelength range, an optical source to emit light in an ultra-violet wavelength range, or a combination thereof.
The invention relates to a system for optical sensing or imaging, addressing the need for versatile light sources capable of emitting light in different wavelength ranges to enhance detection accuracy and functionality. The system includes multiple optical sources configured to emit light in specific wavelength ranges, including near-infrared (NIR) and ultraviolet (UV) wavelengths. The NIR light source enables penetration through certain materials or tissues, useful for applications like medical imaging or material analysis. The UV light source facilitates detection of fluorescent markers or materials that absorb UV light, enhancing contrast in imaging or identification tasks. The system may combine these sources to provide multi-spectral or hyperspectral imaging, improving the ability to analyze or visualize samples across different wavelengths. This configuration allows the system to adapt to various applications, such as biological imaging, industrial inspection, or environmental monitoring, by selecting the appropriate wavelength range for the task. The optical sources are integrated into a cohesive system, ensuring synchronized or independent operation depending on the application requirements.
19. A system comprising: a head-mounted device (HMD) having a first plurality of optical sources to emit light and a first event camera to output a first stream of pixel events generated by a first plurality of pixel sensors, the first plurality of optical sources and the first plurality of pixel sensors disposed at known locations relative to an HMD reference frame; a secondary device physically unassociated with the HMD comprising a second plurality of optical sources to emit light and a second event camera to output a second stream of pixel events generated by a second plurality of pixel sensors, the second plurality of optical sources and the second plurality of pixel sensors disposed at known locations relative to a secondary device reference frame; and a first control node communicatively coupled to the HMD to determine correspondences between the HMD and the secondary device by mapping location data generated in the HMD reference frame for a set of second optical sources among the second plurality of optical sources to respective known locations of the set of second optical sources relative to the secondary device reference frame, wherein the set of second optical sources are identified by recognizing defined illumination parameters associated with the set of second optical sources in light intensity data obtained from the first stream of pixel events.
This invention relates to a system for determining spatial correspondences between a head-mounted device (HMD) and a secondary device using event-based vision. The system addresses the challenge of accurately tracking the relative positions of devices in dynamic environments, particularly where traditional camera-based methods may struggle with latency or computational overhead. The system includes an HMD equipped with multiple optical sources and an event camera. The event camera generates a stream of pixel events from its sensors, which detect changes in light intensity asynchronously. The optical sources and pixel sensors are positioned at known locations relative to the HMD's reference frame. A secondary device, physically separate from the HMD, also has multiple optical sources and an event camera with similar configurations, but its components are mapped to its own reference frame. A control node processes data from the HMD to identify a subset of the secondary device's optical sources by analyzing the HMD's event stream for defined illumination patterns. The node then maps the detected locations of these sources in the HMD's reference frame to their known positions in the secondary device's reference frame, establishing spatial correspondences between the two devices. This enables precise tracking of their relative positions without requiring direct physical connection or complex synchronization. The system leverages event-based sensing for low-latency, high-efficiency spatial mapping in applications such as augmented reality, robotics, or collaborative environments.
20. The system of claim 19 , wherein the HMD includes an inertial measurement unit (IMU) to provide inertial data corresponding to movement of the HMD in a global reference frame.
A head-mounted display (HMD) system includes an inertial measurement unit (IMU) to track the device's movement in a global reference frame. The IMU provides inertial data, such as acceleration and angular velocity, which is used to determine the HMD's position and orientation relative to a fixed global coordinate system. This data can be combined with other sensors, such as cameras or external tracking systems, to improve accuracy and reduce drift over time. The system may also include a processor to process the inertial data and generate a more stable and precise representation of the HMD's movement. This technology is useful in applications like virtual reality (VR), augmented reality (AR), and mixed reality (MR), where accurate tracking of the user's head movements is essential for immersive experiences. The IMU-based tracking helps maintain spatial awareness, reduce motion sickness, and enhance interaction with virtual or augmented environments. The system may also include calibration mechanisms to account for sensor errors and environmental factors, ensuring reliable performance in dynamic conditions.
Unknown
November 24, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.