Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A non-transitory machine-readable medium encoded with machine-executable instructions, wherein the machine-executable instructions, when executed by a processor, cause: capturing a user interaction via one or more sensors within a vehicle to generate one or more sensor signals, wherein the one or more sensors comprise an imaging device that generates one or more sensor signals comprising an image or video sequence; extracting features from frames of the image or video sequence; detecting gestures and movements in the user interaction through a spatiotemporal correlation between extracted features from consecutive frames of the image or video sequence through one or more of image subtraction or triggers activated by exceeding thresholds; determining one or more actions corresponding to one or more detected gestures or movements; synthesizing via a synthesizer a synthesized audio signal corresponding to the one or more actions in accordance with a set of rules; combining the synthesized audio signal with a primary audio signal to generate a combined audio signal; and reproducing the combined audio signal via one or more speakers sensors within the vehicle.
This invention relates to a system for processing user interactions within a vehicle to generate audio feedback. The system captures user interactions using sensors, including an imaging device that records images or video sequences. Features are extracted from frames of the recorded video, and gestures or movements are detected by analyzing spatiotemporal correlations between consecutive frames, using techniques such as image subtraction or threshold-based triggers. Once gestures are identified, corresponding actions are determined, and a synthesized audio signal is generated based on these actions according to predefined rules. The synthesized audio is then combined with a primary audio signal to produce a combined output, which is reproduced through vehicle speakers. The system enables real-time interaction recognition and audio response, enhancing user experience in vehicle environments. The invention leverages sensor data, feature extraction, gesture detection, and audio synthesis to create an interactive audio feedback mechanism.
2. The non-transitory machine-readable medium of claim 1 , wherein the processor is further programmed for causing rendering of a tactile feedback corresponding to the one or more actions corresponding to the one or more actions within the vehicle.
This invention relates to a non-transitory machine-readable medium containing instructions for a processor to execute functions related to vehicle control and user interaction. The system is designed to enhance driver awareness and safety by providing tactile feedback in response to vehicle actions. The processor is programmed to detect and interpret vehicle actions, such as braking, acceleration, or steering adjustments, and generate corresponding tactile feedback signals. These signals are transmitted to a tactile feedback device, which may be integrated into the vehicle's steering wheel, seat, or other driver interface components. The tactile feedback is synchronized with the detected actions to provide real-time haptic responses, alerting the driver to changes in vehicle dynamics or potential hazards. The system may also include additional features, such as adjusting the intensity or pattern of the feedback based on the severity or urgency of the detected action. This tactile feedback mechanism aims to improve driver situational awareness and reaction time, particularly in scenarios where visual or auditory alerts may be less effective or distracting. The invention is particularly useful in autonomous or semi-autonomous vehicles, where tactile feedback can help maintain driver engagement and readiness to take control when necessary.
3. The non-transitory machine-readable medium of claim 1 , wherein the processor is further programmed for causing rendering of a virtual feedback corresponding to the one or more actions through a vision system within the vehicle.
This invention relates to vehicle control systems that use machine-readable instructions to enhance driver interaction. The system addresses the challenge of providing intuitive feedback to drivers when performing actions such as adjusting vehicle settings or navigating interfaces. The invention involves a processor executing instructions stored on a non-transitory machine-readable medium to process driver inputs and generate virtual feedback. This feedback is displayed through a vehicle's vision system, such as a heads-up display or augmented reality interface, to visually confirm actions like adjusting climate controls, seat positions, or infotainment settings. The system ensures real-time, context-aware feedback, improving driver awareness and reducing cognitive load. The vision system may overlay graphical indicators, animations, or text to indicate successful execution of commands or system responses. This approach enhances user experience by providing immediate, non-intrusive feedback without requiring physical buttons or excessive manual interaction. The invention is particularly useful in modern vehicles with advanced driver-assistance systems (ADAS) and touchless control interfaces.
4. The non-transitory machine-readable medium of claim 1 , wherein the one or more sensors comprise a plurality of ultrasonic sensors within the vehicle that generate one or more sensor signals comprising ultrasonic data, features being extracted from the ultrasonic data and the consecutive frames of the image or video sequence, wherein the one or more detected gestures or movements are detected based on the extracted features from the ultrasonic data and the consecutive frames of the image or video sequence.
A system for detecting gestures or movements within a vehicle using a combination of ultrasonic sensors and image or video data. The system addresses the challenge of accurately identifying user gestures or movements in a vehicle environment, where traditional optical sensors may be limited by lighting conditions or occlusions. The system includes multiple ultrasonic sensors mounted within the vehicle, which generate ultrasonic data as sensor signals. These signals are processed to extract features, which are then analyzed alongside consecutive frames from an image or video sequence. The extracted features from both the ultrasonic data and the image/video frames are used to detect and interpret gestures or movements. This multi-modal approach improves detection accuracy by leveraging the strengths of both ultrasonic sensing and visual data, ensuring reliable gesture recognition even in challenging conditions. The system may be used for various applications, such as controlling vehicle functions or enhancing driver assistance systems.
5. The non-transitory machine-readable medium of claim 1 , wherein the one or more sensors comprise radio frequency identification (RFID) sensors within the vehicle that generate one or more sensor signals comprising RFID-based movement data based on detected movement of RFID gloves, features being extracted from the RFID-based movement data and the consecutive frames of the image or video sequence, wherein the one or more detected gestures or movements are detected based on the extracted features from the RFID-based movement data and the consecutive frames of the image or video sequence.
This invention relates to a system for detecting gestures or movements within a vehicle using a combination of radio frequency identification (RFID) sensors and image or video data. The system addresses the challenge of accurately tracking hand movements in a vehicle environment, where traditional optical sensors may be limited by lighting conditions or occlusions. The system includes one or more RFID sensors integrated into the vehicle, which detect movement of RFID-equipped gloves worn by occupants. These sensors generate RFID-based movement data, which is processed to extract relevant features. Additionally, the system captures consecutive frames of an image or video sequence, which are also analyzed to extract features. The detected gestures or movements are determined by combining the extracted features from both the RFID-based movement data and the image or video frames. This hybrid approach improves accuracy by leveraging the strengths of both RFID and visual sensing, ensuring reliable gesture recognition even in challenging conditions. The system may be used for various in-vehicle applications, such as gesture-based control of infotainment systems or driver assistance features.
6. The non-transitory machine-readable medium of claim 1 , wherein detecting one or more gestures or movements in the user interaction through a spatiotemporal correlation between extracted features from consecutive frames of the image or video sequence comprises determining a rate of change of the extracted features through the image subtraction.
This invention relates to gesture recognition systems that analyze image or video sequences to detect user interactions. The core problem addressed is accurately identifying gestures or movements from sequential visual data, which is challenging due to variations in lighting, background noise, and motion complexity. The solution involves a spatiotemporal correlation technique that compares extracted features from consecutive frames to determine gestures or movements. Specifically, the system calculates the rate of change of these features using image subtraction, a method that highlights differences between frames to isolate motion. This approach enhances gesture detection by focusing on dynamic changes rather than static elements, improving accuracy in real-time applications. The extracted features may include spatial patterns, motion vectors, or other relevant visual characteristics. By analyzing the rate of change through subtraction, the system can distinguish intentional gestures from incidental movements, making it suitable for applications like virtual reality, human-computer interfaces, and automated surveillance. The method ensures robustness by leveraging temporal consistency in feature evolution, reducing false positives in gesture recognition.
7. The non-transitory machine-readable medium of claim 1 , wherein detecting one or more gestures or movements in the user interaction through a spatiotemporal correlation between extracted features from consecutive frames of the image or video sequence comprises determining a rate of change of the extracted features through the triggers activated by exceeding thresholds.
This invention relates to gesture recognition systems that analyze image or video sequences to detect user interactions. The problem addressed is accurately identifying gestures or movements from sequential visual data, which can be challenging due to noise, varying lighting conditions, and complex backgrounds. The system processes consecutive frames of an image or video sequence to extract features, such as spatial and temporal characteristics of the user's movements. These features are then analyzed using spatiotemporal correlation techniques to detect meaningful gestures. A key aspect is determining the rate of change of these extracted features, which is calculated by monitoring triggers activated when predefined thresholds are exceeded. These triggers help distinguish intentional gestures from random or irrelevant movements. The method involves comparing feature changes across frames to identify patterns that correspond to specific gestures. By tracking the rate of change, the system can filter out noise and improve recognition accuracy. The thresholds are set based on empirical data or predefined criteria to ensure reliable detection. This approach enhances the robustness of gesture recognition in real-world applications, such as human-computer interaction, augmented reality, and assistive technologies. The system may also include additional processing steps, such as feature extraction and correlation analysis, to refine gesture detection further.
8. The non-transitory machine-readable medium of claim 1 , wherein the one or more sensors comprise pressure sensors or touch sensors that detect one or more of touch or movement, wherein the one or more detected gestures or movements are detected based on the touch or movement detected by the pressure sensors or touch sensors and the extracted features from the consecutive frames of the image or video sequence.
A system for gesture or movement detection in a computing environment uses one or more sensors, including pressure sensors or touch sensors, to detect touch or movement. The sensors capture input data, which is processed to extract features from consecutive frames of an image or video sequence. These extracted features, combined with the sensor data, are used to identify specific gestures or movements. The system may analyze pressure or touch patterns to determine the nature of the detected input, such as a swipe, tap, or other gesture. The extracted features from the image or video sequence provide additional context, improving accuracy in gesture recognition. This approach enhances interaction with devices by combining sensor-based input with visual data analysis, enabling more precise and responsive gesture detection. The system is particularly useful in applications requiring touch or movement-based control, such as virtual reality, augmented reality, or touch-sensitive interfaces.
9. A method of interacting with an entertainment system with a vehicle, comprising: capturing a user interaction via one or more sensors within the vehicle to generate one or more sensor signals, wherein the one or more sensors comprise an imaging device that generates one or more sensor signals comprising an image or video sequence; extracting features from frames of the image or video sequence; detecting gestures and movements in the user interaction through a spatiotemporal correlation between extracted features from consecutive frames of the image or video sequence through one or more of image subtraction or triggers activated by exceeding thresholds; determining one or more actions corresponding to one or more detected gestures or movements; synthesizing via a synthesizer a synthesized audio signal corresponding to the one or more actions in accordance with a set of rules; combining the synthesized audio signal with a primary audio signal to generate a combined audio signal; and reproducing the combined audio signal via one or more speakers within the vehicle.
This invention relates to an interactive entertainment system for vehicles that responds to user gestures and movements. The system captures user interactions using sensors, including an imaging device that generates images or video sequences. Features are extracted from frames of the captured video, and gestures or movements are detected by analyzing spatiotemporal correlations between consecutive frames. This detection may involve image subtraction or threshold-based triggers. Once gestures are identified, corresponding actions are determined, and a synthesized audio signal is generated based on these actions according to predefined rules. The synthesized audio is then combined with a primary audio signal to produce a combined output, which is reproduced through the vehicle's speakers. The system enables hands-free interaction with entertainment features, such as adjusting volume, changing tracks, or controlling other functions, by interpreting user gestures in real time. The approach leverages computer vision and audio processing to enhance user experience without requiring physical contact with controls.
10. The method of claim 9 , further comprising: rendering a tactile feedback corresponding to the one or more actions within the vehicle.
A method for enhancing user interaction within a vehicle involves providing tactile feedback corresponding to one or more actions performed by a user. The method includes detecting user inputs, such as gestures or touch interactions, and processing these inputs to determine the corresponding actions. These actions may include adjusting vehicle settings, navigating menus, or controlling vehicle functions. The system then generates and delivers tactile feedback, such as vibrations or haptic responses, to confirm the execution of the actions. This feedback is synchronized with the user's interactions to provide real-time confirmation, improving user experience by reducing reliance on visual or auditory cues. The method may also involve analyzing the context of the user's actions, such as the current driving conditions or user preferences, to customize the tactile feedback. The system ensures that the feedback is perceptible and non-distracting, enhancing safety and usability. The method may be integrated into vehicle control interfaces, such as touchscreens, steering wheel controls, or other input devices, to provide consistent and intuitive feedback across different interaction points. The tactile feedback can vary in intensity, duration, or pattern based on the type of action or the urgency of the feedback, ensuring clear communication without overwhelming the user. This approach improves interaction efficiency and reduces cognitive load, particularly in dynamic driving environments.
11. The method of claim 9 , further comprising: rendering a virtual feedback corresponding to the one or more actions in through a vision system within the vehicle.
A system and method for providing virtual feedback within a vehicle involves detecting one or more actions performed by a user, such as gestures or interactions with vehicle controls, and generating corresponding virtual feedback. The feedback is rendered through a vision system, such as a heads-up display (HUD) or augmented reality (AR) interface, to provide real-time visual responses to the user's actions. This enhances user interaction by confirming actions, guiding operations, or providing additional information without requiring physical feedback mechanisms. The method may include capturing user actions via sensors, processing the input to determine the intended action, and generating appropriate visual feedback, such as icons, animations, or text, to be displayed within the vehicle's field of view. The system ensures seamless integration with existing vehicle interfaces, improving user experience and reducing reliance on traditional physical feedback methods. The technology is particularly useful in modern vehicles where digital interfaces and augmented reality are increasingly used to enhance driver and passenger interactions.
12. The method of claim 9 , wherein the one or more sensors comprise a plurality of ultrasonic sensors within the vehicle that generate one or more sensor signals comprising ultrasonic data, features being extracted from the ultrasonic data and the consecutive frames of the image or video sequence, wherein the one or more detected gestures or movements are detected based on the extracted features from the ultrasonic data and the consecutive frames of the image or video sequence.
This invention relates to gesture and movement detection within a vehicle using a combination of ultrasonic sensors and image or video data. The system addresses the challenge of accurately detecting user gestures or movements in a vehicle environment, where traditional optical sensors may be limited by lighting conditions or occlusions. The method involves using multiple ultrasonic sensors mounted within the vehicle to generate ultrasonic data, which is processed to extract relevant features. Simultaneously, consecutive frames from an image or video sequence are analyzed to detect gestures or movements. The extracted features from both the ultrasonic data and the image/video frames are combined to improve detection accuracy. The ultrasonic sensors provide additional spatial and motion data that complement the visual information, enhancing reliability in various lighting and environmental conditions. This approach enables robust gesture recognition for applications such as vehicle control, user interaction, or safety monitoring. The system may be integrated into existing vehicle sensor networks to provide a more comprehensive sensing solution.
13. The method of claim 9 , wherein the one or more sensors comprise radio frequency identification (RFID) sensors within the vehicle that generate one or more sensor signals comprising RFID-based movement data based on detected movement of RFID gloves, features being extracted from the RFID-based movement data and the consecutive frames of the image or video sequence, wherein the one or more detected gestures or movements are detected based on the extracted features from the RFID-based movement data and the consecutive frames of the image or video sequence.
This invention relates to gesture recognition systems for vehicles, specifically using RFID-based movement data combined with image or video analysis to detect gestures or movements. The problem addressed is improving the accuracy and reliability of gesture recognition in vehicles, where traditional vision-based systems may struggle with lighting conditions, occlusions, or other environmental factors. The system includes one or more sensors, such as RFID sensors, installed within the vehicle. These sensors detect movement of RFID-equipped gloves worn by the user, generating RFID-based movement data. The system captures consecutive frames from an image or video sequence, which may be from a camera or other imaging device. Features are extracted from both the RFID-based movement data and the consecutive frames. The detected gestures or movements are determined by analyzing these extracted features from both data sources. By combining RFID sensor data with visual data, the system enhances gesture recognition accuracy, reducing false positives and improving responsiveness in vehicle environments. This approach is particularly useful for in-vehicle human-machine interfaces, such as controlling infotainment systems or adjusting vehicle settings through gestures.
14. The method of claim 9 , wherein detecting one or more gestures or movements in the user interaction through a spatiotemporal correlation between extracted features from consecutive frames of the image or video sequence comprises determining a rate of change of the extracted features through the image subtraction.
This invention relates to gesture recognition systems that analyze user interactions through image or video sequences. The problem addressed is accurately detecting gestures or movements in real-time by analyzing spatiotemporal correlations between features extracted from consecutive frames. Traditional methods often struggle with noise, occlusions, or varying lighting conditions, leading to inaccurate gesture detection. The method involves capturing an image or video sequence of a user's interaction and extracting features from consecutive frames. These features are then compared using image subtraction to determine their rate of change over time. By analyzing this rate of change, the system identifies gestures or movements based on spatiotemporal correlations. This approach improves accuracy by focusing on dynamic changes rather than static features alone. The method can be applied in various applications, such as human-computer interaction, augmented reality, or robotics, where precise gesture recognition is essential. The use of image subtraction helps reduce computational complexity while maintaining robustness against environmental variations.
15. The method of claim 9 , wherein detecting one or more gestures or movements in the user interaction through a spatiotemporal correlation between extracted features from consecutive frames of the image or video sequence comprises determining a rate of change of the extracted features through the triggers activated by exceeding thresholds.
This invention relates to gesture recognition in image or video sequences, addressing the challenge of accurately detecting and interpreting user gestures or movements in real-time applications. The method involves analyzing consecutive frames of an image or video sequence to extract features that represent user interactions. These features are then correlated spatiotemporally to identify gestures or movements by evaluating their rate of change. The system uses predefined thresholds to trigger detection events when the rate of change exceeds these thresholds, ensuring reliable and responsive gesture recognition. The approach enhances accuracy by focusing on dynamic changes in extracted features rather than static comparisons, making it suitable for applications like virtual reality, human-computer interaction, and automated surveillance. The method may also incorporate additional processing steps, such as noise filtering or feature normalization, to improve robustness against environmental variations. By dynamically adjusting thresholds based on contextual factors, the system adapts to different interaction scenarios, ensuring consistent performance across varying conditions. The invention provides a scalable solution for real-time gesture recognition, reducing false positives and improving user experience in interactive systems.
16. The method of claim 9 , wherein the one or more sensors comprise pressure sensors or touch sensors that detect one or more of touch or movement, wherein the one or more detected gestures or movements are detected based on the touch or movement detected by the pressure sensors or touch sensors and the extracted features from the consecutive frames of the image or video sequence.
This invention relates to gesture recognition systems that use a combination of sensor data and image processing to detect and interpret user gestures or movements. The problem addressed is improving the accuracy and reliability of gesture recognition by integrating multiple input sources, such as pressure or touch sensors, with visual data from cameras or other imaging devices. The system includes one or more sensors, such as pressure or touch sensors, that detect physical interactions like touch or movement. These sensors provide additional input alongside a sequence of images or video frames captured by an imaging device. The system processes the sensor data to identify gestures or movements, such as tapping, swiping, or pressing, and correlates this information with features extracted from the image or video sequence. By combining sensor-based detection with visual analysis, the system enhances gesture recognition performance, particularly in scenarios where visual data alone may be ambiguous or insufficient. The method involves capturing a sequence of images or video frames, extracting features from these frames, and analyzing the sensor data to detect gestures or movements. The extracted features and sensor data are then used together to determine the specific gesture or movement being performed. This approach improves robustness in gesture recognition by leveraging both tactile and visual inputs, making the system more adaptable to different environments and user interactions.
17. The method of claim 9 , further comprising: determining a state of the vehicle state through one or more of accelerometers or a global positioning sensor (GPS) in the one or more sensors, wherein the one or more detected gestures or movements are detected based on the state of the vehicle and the extracted features from the consecutive frames of the image or video sequence.
This invention relates to a system for detecting gestures or movements of a user within a vehicle using sensor data and image processing. The system addresses the challenge of accurately interpreting user gestures in dynamic environments, such as vehicles, where motion and vibrations can interfere with gesture recognition. The method involves capturing image or video sequences of the user and analyzing consecutive frames to extract features that represent gestures or movements. These features are then compared to a predefined set of gesture patterns to identify the specific gesture being performed. To enhance accuracy, the system incorporates data from vehicle sensors, such as accelerometers or GPS, to determine the vehicle's state (e.g., speed, acceleration, or location). This vehicle state data is used to refine gesture detection by accounting for external factors like vehicle motion, which could otherwise be misinterpreted as user gestures. The system dynamically adjusts gesture recognition based on the vehicle's state, ensuring reliable performance even in moving conditions. This approach improves the robustness of gesture-based interactions in automotive environments.
18. The method of claim 9 , wherein the one or more sensors comprise microphones for detecting sounds within the vehicle that generate one or more sensor signals comprising audio data, wherein the one or more detected gestures or movements are detected based on the detected sounds in the audio data and the extracted features from the consecutive frames of the image or video sequence.
This invention relates to a system for detecting gestures or movements within a vehicle using a combination of audio and visual data. The system addresses the challenge of accurately identifying user gestures or movements in a vehicle environment, where traditional visual-only approaches may be limited by lighting conditions, occlusions, or other factors. The system includes one or more sensors, such as microphones, that capture sounds within the vehicle and generate sensor signals containing audio data. These sounds are analyzed to detect gestures or movements, such as hand movements, speech, or other audible cues. Additionally, the system processes image or video sequences from the vehicle's interior to extract features from consecutive frames. The detected sounds in the audio data are correlated with the extracted visual features to improve gesture or movement recognition accuracy. By combining audio and visual data, the system enhances the reliability of gesture detection in dynamic and challenging vehicle environments. This approach allows for more robust interpretation of user intent, enabling better interaction with in-vehicle systems, such as infotainment or safety features. The integration of multiple sensor modalities ensures that gestures or movements are detected even when one type of data (e.g., visual) is partially obscured or unreliable.
19. An in-vehicle entertainment system, comprising: one or more sensors that generate one or more sensor signals, the one or more sensors comprising an imaging device that generates one or more sensor signals comprising an image or video sequence; one or more speakers; a processor coupled to the one or more sensors and the one or more speakers, wherein the processor is programmed to: capture a user interaction via the one or more sensors within a vehicle to generate one or more sensor signals, wherein the one or more sensors comprise an imaging device that generates one or more sensor signals comprising an image or video sequence; extract features from frames of the image or video sequence; detect gestures and movements in the user interaction through a spatiotemporal correlation between extracted features from consecutive frames of the image or video sequence through one or more of image subtraction or triggers activated by exceeding thresholds; determine one or more actions corresponding to one or more detected gestures or movements; synthesize via a synthesizer a synthesized audio signal corresponding to the one or more actions in accordance with a set of rules; combine the synthesized audio signal with a primary audio signal to generate a combined audio signal; and reproduce the combined audio signal via the one or more speakers within the vehicle.
An in-vehicle entertainment system captures user interactions within a vehicle using one or more sensors, including an imaging device that generates images or video sequences. The system processes these sensor signals to detect gestures and movements by extracting features from consecutive frames and analyzing spatiotemporal correlations through techniques such as image subtraction or threshold-based triggers. Once gestures or movements are identified, the system determines corresponding actions and synthesizes an audio signal that matches these actions based on predefined rules. This synthesized audio signal is then combined with a primary audio signal to produce a combined output, which is reproduced through the vehicle's speakers. The system enables intuitive, gesture-based control of in-vehicle entertainment features, enhancing user interaction without requiring physical contact with controls. The use of spatiotemporal analysis ensures accurate gesture recognition, while audio synthesis provides real-time feedback for user actions. This approach improves accessibility and convenience in vehicle environments.
Unknown
December 3, 2019
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.