A head-mounted display device includes an eye tracking sensor configured to obtain eye information by tracking both eyes of a user, a depth sensor configured to obtain depth information about one or more objects, and a processor configured to obtain information about a gaze point based on the eye information, and determine a measurement parameter of the depth sensor based on the information about the gaze point.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A head-mounted display device comprising: an eye tracking circuitry; a depth sensor; and one or more processors configured to: control the eye tracking circuitry to obtain a direction of a left eye of a user and a direction of a right eye of the user, obtain location information of a gaze point based on the obtained direction of the left eye of the user and the obtained direction of the right eye of the user, obtain first depth information through the depth sensor, based on the obtained first depth information, determine second depth information corresponding to the obtained location information of the gaze point, based on the determined second depth information, determine a depth measurement parameter of the depth sensor, wherein the depth measurement parameter including at least one of a parameter with respect to an output of an emission light or a parameter with respect to sensing of a reflection light; and based on the determined depth measurement parameter, control the depth sensor to obtain third depth information by using a portion of a sensing capability of the depth sensor.
This invention relates to head-mounted display (HMD) devices and addresses the problem of accurately and efficiently capturing depth information relevant to a user's gaze point. The device includes eye tracking circuitry and a depth sensor. Processors are configured to operate the eye tracking circuitry to determine the direction of both the user's left and right eyes. This eye direction data is then used to calculate the location of the user's gaze point. The device also obtains initial depth information using the depth sensor. Based on this initial depth information, the system determines second depth information specifically for the user's gaze point. Crucially, it then determines a depth measurement parameter for the depth sensor. This parameter can relate to the emission of light or the sensing of reflected light. Finally, using this determined depth measurement parameter, the processors control the depth sensor to acquire third depth information. This is achieved by utilizing a specific portion of the depth sensor's sensing capabilities, implying a more targeted and potentially optimized depth measurement at the gaze point.
2. The head-mounted display device of claim 1 , wherein the one or more processors is further configured to obtain two-dimensional (2D) location information of the gaze point based on the direction of the left eye of the user and the direction of the right eye of the user.
A head-mounted display device includes a gaze tracking system that determines a user's gaze point in a virtual or augmented reality environment. The device uses eye-tracking sensors to detect the direction of both the left and right eyes of the user. The system processes this data to calculate two-dimensional (2D) location information of the gaze point, which represents where the user is looking within the display's field of view. This gaze tracking functionality enables applications such as gaze-based interaction, attention tracking, or adaptive content rendering. The device may also include additional features like head pose tracking, display calibration, or user interface adjustments based on the detected gaze point. The system ensures accurate and responsive gaze tracking by analyzing the convergence of the left and right eye directions, improving interaction precision in virtual environments. The technology addresses the need for reliable gaze tracking in head-mounted displays to enhance user experience and enable new interaction methods.
3. The head-mounted display device of claim 1 , wherein the depth sensor is further configured to obtain the second depth information about a region of interest (ROI) set with respect to the gaze point based on the first depth information.
A head-mounted display device includes a depth sensor and a gaze tracking system. The device determines a gaze point of a user by tracking eye movements. The depth sensor captures first depth information about the environment, which is used to identify objects or regions of interest (ROIs) relative to the gaze point. The depth sensor then obtains second depth information specifically for the ROI, providing more detailed spatial data about the area the user is looking at. This allows the device to enhance virtual or augmented reality experiences by dynamically adjusting content based on the user's focus. The system may also use the depth information to improve object recognition, interaction accuracy, or depth perception in the displayed content. The gaze tracking and depth sensing work together to create a more immersive and responsive display experience.
4. The head-mounted display device of claim 3 , wherein the one or more processors is further configured to determine the depth measurement parameter of the depth sensor based on the second depth information about the ROI, and wherein the depth sensor is further configured to obtain the third depth information about the ROI according to the depth measurement parameter.
A head-mounted display device includes a depth sensor and one or more processors. The device captures first depth information about a region of interest (ROI) in an environment using the depth sensor. The processors analyze the first depth information to identify a target object within the ROI. The depth sensor then captures second depth information about the ROI at a higher resolution or accuracy than the first depth information. The processors determine a depth measurement parameter for the depth sensor based on the second depth information. The depth sensor uses this parameter to capture third depth information about the ROI, which may be used for further processing, such as object tracking, augmented reality rendering, or environmental mapping. The device dynamically adjusts the depth sensor's measurement settings to optimize performance for the identified target object, improving accuracy and efficiency in applications like virtual reality, augmented reality, or spatial computing.
5. The head-mounted display device of claim 3 , wherein the depth sensor is further configured to obtain the first depth information about the ROI by using at least one of a time of flight (TOF) method, a structured light (SL) method, or a stereo image (SI) method.
This invention relates to head-mounted display (HMD) devices equipped with depth sensing capabilities to enhance user interaction and virtual/augmented reality experiences. The device includes a depth sensor that captures depth information about a region of interest (ROI) in the user's environment. The depth sensor employs at least one of three methods to obtain this information: time-of-flight (TOF), structured light (SL), or stereo imaging (SI). TOF measures the time it takes for light to travel to an object and return, SL projects a known pattern and analyzes distortions to determine depth, and SI uses two or more cameras to triangulate depth based on parallax. The depth data is used to improve spatial awareness, object tracking, or interaction within the HMD's field of view. This technology addresses the need for accurate and reliable depth perception in wearable displays, enabling better integration of virtual elements with the real world and enhancing applications like gaming, navigation, and industrial training. The depth sensor's flexibility in using multiple methods ensures adaptability to different environmental conditions and performance requirements.
6. The head-mounted display device of claim 5 , wherein, when the depth sensor comprises a TOF depth sensor, the one or more processors is further configured to determine the depth measurement parameter based on the 2D location information of the gaze point such that some light sources corresponding to the gaze point among light sources included in the depth sensor are driven, and wherein the depth sensor is further configured to obtain the third depth information about the ROI by driving the some light sources.
A head-mounted display device includes a depth sensor and one or more processors. The device is designed to enhance depth measurement accuracy in augmented reality (AR) or virtual reality (VR) applications by selectively driving specific light sources in a time-of-flight (TOF) depth sensor based on the user's gaze point. The depth sensor captures 2D location information of the gaze point, and the processors determine a depth measurement parameter to activate only the light sources corresponding to the gaze point. This targeted activation reduces interference from other light sources, improving the accuracy of depth information for the region of interest (ROI) around the gaze point. The depth sensor then obtains refined depth data for the ROI by driving only the selected light sources, ensuring precise depth perception in AR/VR environments. This approach optimizes power efficiency and computational resources by focusing depth sensing on the relevant area, enhancing user experience in dynamic virtual environments.
7. The head-mounted display device of claim 3 , further comprising a display displaying a real space comprising the ROI, and wherein the one or more processors is further configured to control the display to display at least one virtual object on the ROI based on the third depth information about the ROI.
This invention relates to head-mounted display (HMD) devices designed to enhance user interaction with real-world environments by overlaying virtual objects onto specific regions of interest (ROIs) in the real space. The device includes a depth sensor that captures depth information about the real space, identifying one or more ROIs based on predefined criteria such as object type, size, or user-defined parameters. The system processes this depth data to generate a three-dimensional model of the real space, allowing the HMD to accurately position virtual objects relative to the ROIs. Additionally, the device includes a display that renders the real space along with the identified ROIs, and the processing unit controls the display to overlay virtual objects onto these ROIs based on refined depth information. This ensures that the virtual objects appear correctly aligned and interactive within the real-world context. The invention aims to improve augmented reality (AR) experiences by dynamically integrating virtual elements with physical environments, enhancing applications in gaming, navigation, and industrial training.
8. An method of operating head-mounted display device, the method comprising: obtaining a direction of a left eye of a user and a direction of a right eye of the user; obtaining location information of a gaze point based on the obtained direction of the left eye of the user and the obtained direction of the right eye of the user; obtaining first depth information through a depth sensor; based on the obtained first depth information, determining second depth information corresponding to the obtained location information of the gaze point; based on the determined second depth information, determining a depth measurement parameter of the depth sensor, wherein the depth measurement parameter including at least one of a parameter with respect to an output of an emission light or a parameter with respect to sensing of a reflection light; and based on the determined depth measurement parameter, obtaining third depth information by using a portion of a sensing capability of the depth sensor.
This invention relates to gaze-tracking and depth-sensing systems for head-mounted displays (HMDs). The problem addressed is improving depth measurement accuracy in HMDs by dynamically adjusting depth sensor parameters based on the user's gaze direction. Traditional depth sensors often use fixed settings, which may not optimize accuracy for the specific region being viewed by the user. The method involves tracking the user's eye directions to determine a gaze point location. A depth sensor captures initial depth data, and based on this, the system calculates refined depth information specific to the gaze point. Using this refined depth data, the system adjusts depth sensor parameters, such as emission light intensity or reflection light sensitivity, to enhance measurement precision for the gazed region. The adjusted parameters allow the depth sensor to operate within a portion of its full sensing capability, optimizing performance for the specific depth range of interest. This dynamic adjustment ensures more accurate depth perception in the area the user is actively viewing, improving virtual object placement and interaction in augmented or virtual reality environments. The system avoids unnecessary processing of irrelevant depth data, improving efficiency and reducing computational load.
9. The method of claim 8 , wherein the obtaining location information of the gaze point comprises obtaining two-dimensional (2D) location information of the gaze point based on the left eye direction of the left eye of the user and the right eye direction of the right eye of the user.
This invention relates to gaze tracking technology, specifically methods for determining the precise location of a user's gaze point in a two-dimensional (2D) space. The problem addressed is the need for accurate and reliable gaze tracking, which is essential for applications such as human-computer interaction, virtual reality, augmented reality, and accessibility tools. Traditional gaze tracking systems often rely on single-eye tracking or less precise methods, leading to inaccuracies in determining where a user is looking. The invention improves upon prior art by obtaining 2D location information of the gaze point using both the left and right eye directions of the user. By analyzing the direction of each eye independently and combining the data, the system achieves higher accuracy in gaze point estimation. This dual-eye approach compensates for potential errors that may arise from tracking a single eye, such as misalignment or occlusion. The method involves capturing eye movement data from both eyes, processing the directional information, and calculating the intersection or convergence point of the two gaze vectors to determine the precise 2D location of the gaze point. This technique enhances the reliability of gaze tracking in various real-world scenarios, including dynamic environments where users may move their heads or eyes rapidly. The invention is particularly useful in applications requiring precise gaze interaction, such as eye-controlled interfaces, assistive technologies, and immersive experiences.
10. The method of claim 8 , wherein the determining the second depth information comprises determining the second depth information about a region of interest (ROI) set with respect to the gaze point based on the first depth information.
This invention relates to depth sensing and gaze tracking in imaging systems, addressing the challenge of accurately determining depth information in regions of interest (ROIs) based on gaze points. The method involves capturing an image of a scene using an imaging device and obtaining first depth information for the scene. A gaze point is detected within the image, representing where a user is looking. The method then determines second depth information specifically for a region of interest (ROI) defined relative to the gaze point, using the first depth information. The ROI is dynamically set based on the gaze point, ensuring that depth measurements are focused on the area of interest. This approach improves efficiency and accuracy by concentrating computational resources on the relevant region rather than the entire scene. The method may also involve adjusting the ROI size or shape based on factors such as gaze stability or scene complexity. The invention is particularly useful in applications like augmented reality, human-computer interaction, and assistive technologies where precise depth perception in targeted areas is critical.
11. The method of claim 10 , further comprising: determining the depth measurement parameter of the depth sensor based on the second depth information about the ROI; and wherein the obtaining third depth information comprises obtaining the third depth information about the ROI according to the depth measurement parameter.
This invention relates to depth sensing systems, particularly for improving depth measurement accuracy in regions of interest (ROI) within a scene. The problem addressed is the challenge of obtaining precise depth information in specific areas of a captured image, where conventional depth sensors may produce inaccurate or unreliable measurements due to factors like occlusion, low reflectivity, or sensor limitations. The method involves capturing initial depth information of a scene using a depth sensor, then identifying a region of interest (ROI) within the scene where more accurate depth measurements are needed. A second depth measurement is obtained for the ROI, which may involve adjusting sensor parameters or using additional processing techniques to enhance accuracy. Based on this second depth information, a depth measurement parameter for the depth sensor is determined. This parameter is then applied to obtain third depth information about the ROI, ensuring improved accuracy in the final depth data. The process may include iterative refinement, where the depth measurement parameter is continuously adjusted based on feedback from subsequent depth measurements. This ensures that the depth sensor dynamically adapts to varying conditions within the ROI, such as changes in lighting or object movement. The method is particularly useful in applications like augmented reality, robotics, and autonomous navigation, where precise depth perception is critical.
12. The method of claim 11 , wherein the determining of the depth measurement parameter of the depth sensor comprises: when obtaining the first depth information using the TOF method, determining the depth measurement parameter based on the 2D location information of the gaze point such that some light sources corresponding to the gaze point among light sources included in the depth sensor are driven, and the obtaining of the third depth information about the ROI comprises obtaining the depth information about the ROI by driving the some light sources.
This invention relates to depth sensing systems, particularly those using Time-of-Flight (TOF) methods to measure depth in a region of interest (ROI). The problem addressed is the inefficiency of conventional TOF depth sensors, which often illuminate an entire scene with light sources, consuming excessive power and potentially causing interference or glare. The solution involves selectively driving only a subset of light sources in the depth sensor based on the 2D location of a gaze point, reducing power consumption and improving measurement accuracy. When obtaining depth information using the TOF method, the system first determines a depth measurement parameter by identifying which light sources correspond to the gaze point. Only those relevant light sources are activated to acquire depth information about the ROI, ensuring precise measurements while minimizing unnecessary illumination. This selective activation of light sources optimizes energy efficiency and reduces potential artifacts in the depth data. The method is particularly useful in applications like augmented reality, robotics, and human-computer interaction, where accurate depth sensing with minimal power usage is critical.
13. The method of claim 10 , wherein the obtaining of the first depth information about the ROI comprises obtaining the first depth information about the ROI by using at least one of a time of flight (TOF) method, a structured light (SL) method, or a stereo image (SI) method.
This invention relates to depth sensing techniques for capturing depth information about a region of interest (ROI) in a scene. The problem addressed is the need for accurate and reliable depth measurement methods to enhance applications such as augmented reality, robotics, and computer vision. The invention provides a method for obtaining depth information about a ROI using at least one of three depth sensing techniques: time of flight (TOF), structured light (SL), or stereo imaging (SI). The TOF method measures the time it takes for light to travel from a sensor to an object and back, converting this time into depth data. The SL method projects a known pattern onto the scene and analyzes the deformation of the pattern to determine depth. The SI method uses two or more cameras to capture images of the scene from different angles, then calculates depth by triangulating the disparities between the images. The method involves selecting one or more of these techniques to capture depth information about the ROI, ensuring flexibility in adapting to different environmental conditions and application requirements. The use of multiple techniques can improve accuracy and robustness, particularly in challenging lighting or texture conditions. This approach enables precise depth mapping, which is critical for applications requiring spatial awareness, such as object recognition, 3D reconstruction, and autonomous navigation.
14. A non-transitory computer-readable recording medium having recorded thereon a program for executing the method of claim 8 .
A system and method for optimizing data processing in a distributed computing environment addresses inefficiencies in task allocation and resource utilization. The invention involves a distributed computing system where tasks are dynamically assigned to processing nodes based on real-time performance metrics, such as processing speed, memory availability, and network latency. The system monitors these metrics across multiple nodes and adjusts task distribution to balance the workload, reducing bottlenecks and improving overall system efficiency. A central coordinator collects performance data from each node, analyzes it to identify underutilized or overloaded nodes, and reallocates tasks accordingly. The system also includes a predictive model that forecasts future resource demands based on historical data, allowing proactive adjustments before performance degradation occurs. Additionally, the system supports fault tolerance by detecting node failures and redistributing tasks to operational nodes without interrupting processing. The method ensures optimal resource usage, minimizes idle time, and enhances scalability by dynamically adapting to changing workloads and system conditions. This approach is particularly useful in large-scale computing environments where static task allocation leads to inefficiencies.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
February 25, 2020
February 15, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.