Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. An apparatus, comprising: a sensor configured to detect a direction of a gaze of a user and output a first gaze signal corresponding to the detected direction of the gaze of the user; a digital signal processing unit configured to: receive the first gaze signal from the sensor; receive a second gaze signal; receive a third gaze signal; determine a priority for each of the first gaze signal, the second gaze signal, and the third gaze signal; select at least one of the first gaze signal, the second gaze signal, and the third gaze signal based on the priority; identify, in an electronic data storage, one or more available actions that correspond to the selected at least one of the first gaze signal, the second gaze signal, and the third gaze signal; select an action from the one or more available actions; and execute a set of computer-readable instructions that correspond to the action.
The invention relates to a gaze-tracking apparatus designed to interpret and prioritize multiple gaze signals to execute specific actions. The apparatus includes a sensor that detects the direction of a user's gaze and outputs a corresponding first gaze signal. The system also receives a second and third gaze signal, which may originate from additional sensors or external sources. A digital signal processing unit processes these signals by assigning a priority to each, selecting the highest-priority signal, and identifying available actions stored in an electronic data storage that match the selected signal. The system then selects and executes an action by running the corresponding computer-readable instructions. This approach allows the apparatus to dynamically respond to user gaze input, even when multiple conflicting or overlapping gaze signals are present, ensuring accurate and context-aware action execution. The invention is particularly useful in applications requiring precise gaze-based interaction, such as augmented reality, assistive technologies, or human-computer interfaces.
2. The apparatus of claim 1 , further comprising a microphone positioned relative to a position of the sensor to provide information indicating directionality of one or more sounds that correspond to the second gaze signal.
This invention relates to an apparatus for analyzing gaze and sound directionality to enhance user interaction with a device. The apparatus includes a sensor configured to detect a second gaze signal, which represents a user's gaze direction. The sensor may be an eye-tracking device or camera that captures gaze data. The apparatus also includes a microphone positioned relative to the sensor to provide directional sound information. The microphone captures audio signals corresponding to the second gaze signal, allowing the system to correlate gaze direction with sound sources. This helps determine the directionality of sounds that align with the user's gaze, improving context-aware applications such as voice commands, augmented reality, or assistive technologies. The microphone may be an array or directional microphone to enhance spatial audio resolution. The system may use the combined gaze and sound data to filter relevant audio inputs, reduce background noise, or prioritize sound sources based on the user's focus. This integration enables more accurate and responsive interactions in environments where both visual and auditory cues are important.
3. The apparatus of claim 2 , wherein the second gaze signal is selected based on a determination that the detected direction of the gaze of the user that corresponds to the first gaze signal is in a different direction as compared to the one or more sounds that correspond to the second gaze signal, wherein the action is selected based on the detected direction of the one or more sounds.
This invention relates to gaze-tracking systems that integrate audio cues to improve user interaction with devices. The problem addressed is the mismatch between a user's visual focus and audio sources, which can lead to confusion or inefficiency in human-computer interaction. The apparatus includes a gaze-tracking system that detects a user's gaze direction and generates a first gaze signal representing this direction. It also includes an audio detection system that identifies the direction of one or more sounds in the environment and generates a second gaze signal. The second gaze signal is selected when the user's detected gaze direction (from the first gaze signal) differs from the direction of the sounds. The system then performs an action based on the direction of the detected sounds, such as redirecting the user's attention or adjusting device outputs. For example, if a user is looking away from a sound source, the system may prioritize audio cues over visual focus, ensuring the user does not miss important auditory information. The apparatus may also include a display or speaker system to provide feedback or adjustments based on the detected gaze and sound directions. This improves user experience by aligning visual and auditory inputs, particularly in environments where multiple stimuli compete for attention.
4. The apparatus of claim 1 , wherein the first gaze signal is associated with a first priority and the second gaze signal is associated with a second priority, wherein when selecting one of the first gaze signal, the second gaze signal, and the third gaze signal based on the priority, the digital signal processing unit is configured to: determine that the third gaze signal corresponds to an unwanted signal that corresponds to a third priority that has a lower priority than the first priority for the first gaze signal and the second priority for the second gaze signal; and deemphasize the third gaze signal in view of the third priority having a lower priority than the first priority and the second priority.
This invention relates to gaze tracking systems that process multiple gaze signals to determine a user's focus. The problem addressed is the presence of unwanted or low-priority gaze signals that can interfere with accurate tracking, such as signals from unintended eye movements or environmental noise. The apparatus includes a digital signal processing unit that receives and processes at least three gaze signals. Each signal is assigned a priority level, where higher-priority signals are given greater weight in determining the user's gaze direction. The system is designed to identify and deemphasize low-priority signals, such as those from unintended eye movements or noise, to improve tracking accuracy. Specifically, if a third gaze signal is determined to be unwanted and has a lower priority than the other signals, the processing unit reduces its influence on the final gaze determination. This prioritization ensures that only relevant gaze data is used, enhancing the reliability of the tracking system. The invention is particularly useful in applications requiring precise gaze detection, such as augmented reality, medical diagnostics, or user interface control.
5. The apparatus of claim 4 , wherein deemphasizing the third gaze signal includes removing audio from the third gaze signal.
This invention relates to gaze-tracking systems that process multiple gaze signals from different users or devices to enhance user interaction with a display. The problem addressed is the need to selectively prioritize certain gaze signals while suppressing others to improve focus and reduce interference in multi-user or multi-device environments. The apparatus includes a gaze-tracking system that receives at least three gaze signals from different sources, such as multiple users or devices. The system processes these signals to determine which gaze signals should be emphasized or deemphasized based on predefined criteria, such as user priority, attention levels, or device relevance. The apparatus includes a signal processor that adjusts the emphasis of the gaze signals by modifying their characteristics. Specifically, the apparatus deemphasizes a third gaze signal by removing audio components from it, ensuring that only the most relevant gaze data is retained for further processing. This selective suppression helps reduce distractions and improves the accuracy of gaze-based interactions in shared or collaborative environments. The system may also include additional components for analyzing gaze patterns, filtering noise, and dynamically adjusting signal emphasis based on real-time conditions. The overall goal is to enhance user experience by ensuring that the most important gaze inputs are prioritized while minimizing interference from less relevant signals.
6. The apparatus of claim 1 , wherein the digital signal processing unit is further configured to determine that a person is speaking in a direction of the user based on one or more of the first gaze signal, the second gaze signal, and the third gaze signal, wherein the action is selected based on a determination that the person is speaking in the direction of the user.
This invention relates to a digital signal processing apparatus designed to enhance user interaction with audio devices by determining the direction of a speaker relative to a user. The apparatus includes multiple sensors, such as gaze-tracking or directional microphones, to capture signals indicating the direction of a person speaking. These signals are processed to determine whether the speaker is facing the user, which influences the selection of an appropriate action, such as adjusting audio output, focusing microphone sensitivity, or triggering a response. The system may use one or more gaze signals or other directional indicators to assess the speaker's orientation. By analyzing these signals, the apparatus can dynamically adapt to the user's environment, improving audio clarity and interaction efficiency. The invention addresses challenges in distinguishing relevant speech from background noise or off-axis speakers, ensuring that the device responds accurately to the intended speaker. This technology is particularly useful in smart assistants, hearing aids, or conferencing systems where directional audio processing is critical.
7. The apparatus of claim 1 , further comprising an imaging sensor configured to detect an image, wherein the action is selected based on the image.
This invention relates to an apparatus for performing actions based on image analysis. The apparatus includes a processing system that executes a machine learning model to determine an action to perform. The machine learning model is trained to generate a plurality of candidate actions and select one of the candidate actions based on input data. The apparatus further includes an imaging sensor that captures an image, and the action is selected based on the detected image. The processing system may also receive additional input data, such as sensor data or user input, to further refine the action selection. The machine learning model may be trained using a training dataset that includes images and corresponding actions, allowing the apparatus to learn associations between visual features and appropriate responses. The apparatus may be used in applications such as robotics, autonomous systems, or smart devices where real-time decision-making based on visual input is required. The invention addresses the challenge of integrating image-based decision-making into automated systems, improving their adaptability and responsiveness to dynamic environments.
8. The apparatus of claim 1 , wherein the third gaze signal includes an identification signal from a sound source, the identification signal including metadata that identifies either the sound source itself, or a person associated with the sound source.
This invention relates to an apparatus for processing gaze signals to enhance human-computer interaction, particularly in environments where audio and visual cues are used to identify and track sound sources or individuals. The problem addressed is the need to accurately associate gaze direction with specific sound sources or individuals in real-time, improving the precision of attention tracking and interaction systems. The apparatus includes a gaze tracking system that detects a user's gaze direction and generates a gaze signal. It also processes a third gaze signal derived from an audio input, which includes an identification signal from a sound source. This identification signal contains metadata that uniquely identifies either the sound source itself or a person associated with the sound source. The metadata may include identifiers such as device IDs, user profiles, or biometric data, enabling the system to correlate the gaze direction with the correct sound source or individual. The apparatus further integrates this third gaze signal with other gaze signals (e.g., from eye-tracking or head-motion sensors) to refine the accuracy of gaze-based interactions. This allows the system to distinguish between multiple potential targets of attention, improving applications in virtual reality, augmented reality, or assistive technologies where precise gaze tracking is critical. The metadata in the identification signal ensures that the system can reliably associate gaze direction with the intended sound source or person, even in noisy or multi-source environments.
9. A method, comprising: receiving a first gaze signal from a sensor; receiving a second gaze signal; receiving a third gaze signal; determining a priority for each of the first gaze signal, the second gaze signal, and the third gaze signal; selecting at least one of the first gaze signal, the second gaze signal, and the third gaze signal based on the priority; identifying, in an electronic data storage, one or more available actions that correspond to the selected at least one of the first gaze signal, the second gaze signal, and the third gaze signal; selecting an action from the one or more available actions; and executing a set of computer-readable instructions that correspond to the action.
This invention relates to gaze-based interaction systems, specifically methods for processing multiple gaze signals to determine and execute user-intended actions. The problem addressed is the ambiguity and noise in gaze tracking when multiple gaze signals are received, making it difficult to accurately determine user intent for triggering actions in electronic systems. The method involves receiving three distinct gaze signals from one or more sensors, each representing potential user gaze inputs. A priority is assigned to each gaze signal based on predefined criteria, such as signal strength, duration, or user calibration data. The highest-priority gaze signal is selected, and an electronic data storage is queried to identify available actions corresponding to that signal. These actions may include system commands, interface navigation, or other executable functions. One action is then selected from the available options, and the corresponding computer-readable instructions are executed to perform the desired function. This approach improves the reliability of gaze-based interaction by resolving conflicts between multiple gaze inputs and ensuring that only the most relevant action is triggered. The system dynamically adapts to user behavior by prioritizing signals and mapping them to executable actions stored in a database.
10. The method of claim 9 , further comprising receiving information indicating directionality of one or more sounds that correspond to the second gaze signal.
This invention relates to audio-visual systems that integrate gaze tracking with directional sound processing. The problem addressed is the lack of synchronization between visual focus and audio directionality in multimedia environments, leading to a disjointed user experience. The system captures a user's gaze direction to determine where they are looking on a display. It then processes audio signals to identify sounds that correspond to the gazed-at region, analyzing their directional properties. By correlating gaze data with sound directionality, the system enhances audio rendering by adjusting playback to align with the user's visual focus, improving immersion and reducing cognitive load. The method involves detecting a second gaze signal indicating a new point of interest, then receiving information about the directionality of sounds associated with that gaze point. This allows dynamic audio adjustments, such as spatial audio positioning or volume balancing, based on real-time gaze tracking. The system may also filter or prioritize sounds based on their relevance to the gazed-at content, ensuring that audio cues remain contextually aligned with the user's visual attention. This approach is particularly useful in virtual reality, augmented reality, and interactive multimedia applications where seamless integration of visual and auditory stimuli is critical.
11. The method of claim 10 , wherein the second gaze signal is selected based on a determination that a detected direction of a gaze of a user that corresponds to the first gaze signal is in a different direction as compared to the one or more sounds that correspond to the second gaze signal, wherein the action is selected based on the detected direction of the one or more sounds.
This invention relates to gaze-tracking systems that integrate audio cues to enhance user interaction. The problem addressed is the need for more intuitive and context-aware user interfaces, particularly in scenarios where visual attention and auditory cues must be synchronized for accurate system responses. The method involves detecting a user's gaze direction using a first gaze signal and determining whether the gaze is aligned with one or more sounds detected in the environment. If the gaze direction differs from the sound source, a second gaze signal is selected, which corresponds to the direction of the detected sound. An action is then triggered based on the sound's direction, ensuring the system responds to auditory cues even when the user's visual focus is elsewhere. This approach improves interaction accuracy in noisy or multi-modal environments by dynamically adjusting responses based on both visual and auditory inputs. The system may include gaze-tracking hardware, such as cameras or sensors, and audio detection components like microphones. The method ensures seamless integration between visual and auditory data, allowing for more natural and adaptive user interactions. This is particularly useful in applications like virtual reality, augmented reality, or assistive technologies where environmental awareness is critical. The invention enhances user experience by reducing reliance on visual input alone and incorporating contextual audio information for more precise system responses.
12. The method of claim 9 , wherein the first gaze signal is associated with a first priority and the second gaze signal is associated with a second priority, wherein selecting one of the first gaze signal, the second gaze signal, and the third gaze signal based on the priority includes: determining that the third gaze signal corresponds to an unwanted signal that corresponds to a third priority that has a lower priority than the first priority for the first gaze signal and the second priority for the second gaze signal; and deemphasizing the third gaze signal in view of the third priority having a lower priority than the first priority and the second priority.
This invention relates to gaze tracking systems that process multiple gaze signals to determine a user's intended focus. The problem addressed is the challenge of accurately selecting the most relevant gaze signal when multiple signals are detected, some of which may be unintended or irrelevant (e.g., accidental glances or noise). The system captures at least three gaze signals from a user, each representing a different potential focus point. Each signal is assigned a priority level, where higher-priority signals indicate more likely intended focus. The system evaluates these signals and identifies any low-priority signals (e.g., those below a threshold or deemed unwanted). These low-priority signals are deemphasized or excluded from further processing to reduce interference. The remaining higher-priority signals are then used to determine the user's actual focus, improving accuracy in applications like eye-tracking interfaces, attention monitoring, or assistive technologies. The method ensures that transient or irrelevant gaze data does not distort the system's output.
13. The method of claim 12 , wherein deemphasizing the third gaze signal includes removing audio from the third gaze signal.
This invention relates to gaze tracking systems that process multiple gaze signals to enhance user interaction with a device. The problem addressed is the interference caused by extraneous gaze signals, such as those from unintended users or environmental factors, which can degrade the accuracy and reliability of gaze-based input. The solution involves selectively deemphasizing or removing certain gaze signals to improve the system's performance. The method involves detecting a primary gaze signal from an intended user and at least one secondary gaze signal from an unintended source. The secondary gaze signal is identified based on criteria such as signal strength, direction, or timing. To mitigate its impact, the system deemphasizes the secondary gaze signal by removing audio components associated with it. This ensures that only the primary gaze signal influences the device's response, reducing errors and improving user experience. The system may also apply additional filtering techniques to further refine the gaze tracking accuracy. The approach is particularly useful in multi-user environments or scenarios where background noise or unintended gaze inputs could interfere with the intended interaction.
14. The method of claim 9 , further comprising determining that a person is speaking in a direction of a user based on one or more of the first gaze signal, the second gaze signal and the third gaze signal, wherein the action is selected based on a determination that the person is speaking in the direction of the user.
This invention relates to gaze-based interaction systems that determine whether a person is speaking toward a user and select actions based on this determination. The system uses multiple gaze signals from different sensors or devices to track the direction of a person's gaze and infer whether they are addressing the user. The method involves analyzing the first gaze signal, which represents the user's gaze direction, the second gaze signal, which represents the person's gaze direction, and the third gaze signal, which may represent an additional gaze input or environmental context. By evaluating these signals, the system determines if the person is speaking directly to the user. If confirmed, the system selects an appropriate action, such as initiating a response, adjusting device settings, or triggering an automated process. The invention improves human-computer interaction by enabling more natural and context-aware responses based on gaze direction, reducing the need for explicit user input. This is particularly useful in applications like virtual assistants, augmented reality, and collaborative environments where understanding conversational intent is critical. The system enhances user experience by making interactions more intuitive and responsive to social cues.
15. The method of claim 9 , further comprising detecting an image, wherein the action is selected based on the image.
A system and method for selecting actions based on detected images. The technology operates in the domain of computer vision and automated decision-making, addressing the problem of dynamically determining appropriate actions in response to visual inputs. The method involves capturing an image from a sensor, such as a camera, and analyzing the image to identify relevant features or objects. Based on the detected content, an action is selected from a predefined set of possible actions. The selection process may involve comparing the image data to stored templates, applying machine learning models, or using rule-based logic to determine the most suitable response. The system can be integrated into various applications, including robotics, surveillance, and automated quality control, where real-time visual feedback is required to trigger specific actions. The method ensures that the chosen action is contextually relevant to the detected image, improving the accuracy and efficiency of automated systems. The invention enhances the adaptability of machines to dynamic environments by enabling them to respond intelligently to visual stimuli.
16. The method of claim 9 , wherein the third gaze signal includes an identification signal from a sound source, the identification signal including metadata that identifies either the sound source itself, or a person associated with the sound source.
This invention relates to gaze tracking systems that integrate audio identification to enhance user interaction with devices. The technology addresses the challenge of accurately determining a user's focus in environments where multiple audio sources may be present, improving the precision of gaze tracking by correlating visual and auditory inputs. The method involves capturing a third gaze signal that includes an identification signal from a sound source. This identification signal contains metadata that uniquely identifies either the sound source itself or a person associated with the sound source. The system uses this metadata to refine gaze tracking, ensuring that the user's attention is correctly attributed to the intended audio source. This is particularly useful in applications such as virtual reality, augmented reality, or assistive technologies where accurate user intent detection is critical. The method builds on a broader system that tracks gaze direction using multiple sensors, including a first gaze signal from a first sensor and a second gaze signal from a second sensor. The third gaze signal, which includes the audio identification, is combined with these visual inputs to improve accuracy. The metadata in the identification signal may include unique identifiers like device serial numbers, user profiles, or biometric data, enabling precise differentiation between multiple potential sources of sound. This integration of audio and visual data enhances the reliability of gaze tracking in dynamic environments.
17. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising: receive a first gaze signal from a sensor; receive a second gaze signal; receive a third gaze signal; determine a priority for each of the first gaze signal, the second gaze signal, and the third gaze signal; select at least one of the first gaze signal, the second gaze signal, and the third gaze signal based on the priority; identify, in an electronic data storage, one or more available actions that correspond to the selected at least one of the first gaze signal, the second gaze signal, and the third gaze signal; select an action from the one or more available actions; and execute the action.
This invention relates to gaze-based interaction systems, specifically methods for processing multiple gaze signals to determine and execute user actions. The problem addressed is the ambiguity and inefficiency in interpreting gaze data when multiple gaze signals are received simultaneously or in rapid succession, leading to incorrect or delayed action execution. The solution involves a system that receives and prioritizes multiple gaze signals from one or more sensors, selects the most relevant signal based on predefined criteria, and then identifies and executes corresponding actions from a stored set of available options. The system first captures three distinct gaze signals, each representing a potential user intent. It then assigns a priority to each signal, which may be based on factors such as signal strength, duration, or contextual relevance. The highest-priority signal is selected, and the system searches an electronic storage to find matching actions. From these, an action is chosen—either automatically or with additional user input—and executed. This approach improves the accuracy and responsiveness of gaze-based interfaces by resolving conflicts between competing gaze inputs. The invention is applicable in fields like augmented reality, assistive technologies, and human-computer interaction, where precise gaze interpretation is critical.
18. The non-transitory machine-readable medium of claim 17 , wherein the second gaze signal is selected based on a determination that a detected direction of a gaze of a user that corresponds to the first gaze signal is in a different direction as compared to one or more sounds that correspond to the second gaze signal, wherein the action is selected based on the detected direction of the one or more sounds.
This invention relates to gaze tracking and audio processing in computing systems, specifically addressing the challenge of determining user intent when gaze direction does not align with audio sources. The system captures a first gaze signal representing a user's gaze direction and one or more sounds from an audio source. If the detected gaze direction differs from the direction of the sounds, the system selects a second gaze signal—likely from a different time or context—where the gaze aligns with the audio source. Based on this alignment, the system performs an action, such as adjusting audio output or modifying user interface elements. The method ensures that audio-related actions are triggered only when the user's gaze confirms engagement with the sound source, improving accuracy in interactive systems. The invention may be implemented in virtual reality, augmented reality, or other environments where gaze and audio inputs are used for user interaction. The solution enhances user experience by reducing false activations when gaze and sound directions are mismatched.
19. The non-transitory machine-readable medium of claim 17 , wherein the first gaze signal is associated with a first priority and the second gaze signal is associated with a second priority, wherein the operation to select one of the first gaze signal, the second gaze signal, and the third gaze signal based on the priority includes: determine that the third gaze signal corresponds to an unwanted signal that corresponds to a third priority that has a lower priority than the first priority for the first gaze signal and the second priority for the second gaze signal; and deemphasize the third gaze signal in view of the third priority having a lower priority than the first priority and the second priority.
This invention relates to gaze tracking systems that process multiple gaze signals to determine a user's intended focus. The problem addressed is the presence of unwanted or low-priority gaze signals that can interfere with accurate gaze tracking, such as signals from unintended eye movements or environmental noise. The solution involves assigning priority levels to gaze signals and selectively deemphasizing or ignoring lower-priority signals to improve tracking accuracy. The system processes at least three gaze signals, each associated with a priority level. The first and second gaze signals are assigned higher priorities, while the third signal is identified as an unwanted signal with a lower priority. The system determines that the third signal should be deemphasized due to its lower priority compared to the first and second signals. This deemphasis reduces the influence of the unwanted signal, allowing the system to focus on the higher-priority gaze data for more reliable tracking. The method ensures that only relevant gaze signals contribute to the final tracking output, enhancing the system's robustness in noisy or ambiguous scenarios. This approach is particularly useful in applications requiring precise gaze detection, such as virtual reality, augmented reality, or assistive technologies.
20. The non-transitory machine-readable medium of claim 17 , the operations further include determine that a person is speaking in a direction of a user based on one or more of the first gaze signal, the second gaze signal and the third gaze signal, wherein the action is selected based on a determination that the person is speaking in the direction of the user.
This invention relates to gaze-based interaction systems that determine whether a person is speaking toward a user and select actions based on this determination. The system uses multiple gaze signals from different sensors to analyze the direction of a person's gaze and speech. The invention addresses the problem of accurately detecting when a person is addressing a user in a multi-person environment, which is crucial for applications like virtual assistants, augmented reality, or human-computer interaction. The system processes gaze signals from at least three sensors to track the person's gaze direction. These signals are analyzed to determine whether the person is speaking toward the user. If the person is speaking in the user's direction, the system selects an appropriate action, such as responding to the person or adjusting system behavior. The invention improves upon prior systems by using multiple gaze signals to enhance accuracy in determining the direction of speech, reducing false positives where the system might incorrectly interpret a person's gaze or speech as directed toward the user. The system may also integrate additional data, such as audio cues or facial orientation, to further refine the determination. This multi-modal approach ensures more reliable interaction in dynamic environments where visual and auditory signals may be ambiguous. The invention is particularly useful in scenarios where multiple people are present, and the system must distinguish between different speakers and their intended recipients.
Unknown
January 21, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.