Systems and methods herein describe a multi-modal interaction system. The multi-modal interaction system, receives a selection of an augmented reality (AR) experience within an application on a computer device, displays a set of AR objects associated with the AR experience on a graphical user interface (GUI) of the computer device, display textual cues associated with the set of augmented reality objects on the GUI, receives a hand gesture and a voice command, modifies a subset of augmented reality objects of the set of augmented reality objects based on the hand gesture and the voice command, and displays the modified subset of augmented reality objects on the GUI.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
3. The method of claim 2, wherein the selection is a user input received at the computer device.
A system and method for selecting and processing data entries in a computer device involves identifying a set of data entries from a database and presenting them to a user. The user can then select one or more of these entries through an input mechanism, such as a touchscreen, keyboard, or mouse. The selected entries are then processed according to predefined criteria, which may include filtering, sorting, or further analysis. The system ensures that the user's selection is accurately captured and applied to the subsequent processing steps, improving the efficiency and accuracy of data handling. This method is particularly useful in applications where user interaction is required to refine or customize data processing, such as in data analysis, document management, or user interface customization. The system may also include additional features, such as visual feedback to confirm the selection or error handling to manage invalid inputs. The overall goal is to provide a seamless and intuitive way for users to interact with data entries, ensuring that their selections are properly integrated into the processing workflow.
4. The method of claim 3, wherein the augmented reality experience is displayed as a selectable user interface element within the application.
Augmented reality systems. This invention addresses the need for integrated and easily accessible augmented reality (AR) experiences within software applications. Specifically, it describes a method for presenting an augmented reality experience such that it appears as an interactive element, like a button or icon, within the graphical user interface (GUI) of an application. This selectable user interface element, when activated by the user, triggers the display of the augmented reality experience. This allows users to seamlessly initiate and engage with AR content directly from within their current application environment, without needing to navigate to a separate AR viewer or application. The AR experience is therefore presented as a feature or option offered by the main application itself.
5. The method of claim 1, wherein the textual cues are hints associated with the hand gesture and the voice command.
This invention relates to a system for interpreting combined hand gestures and voice commands to control a device. The problem addressed is the ambiguity and inefficiency of relying solely on either gestures or voice commands, which can lead to misinterpretation or require excessive input. The solution involves using textual cues as hints to clarify the relationship between hand gestures and voice commands, improving accuracy and reducing user effort. The system captures hand gestures and voice commands from a user, then analyzes these inputs in conjunction with textual cues. The textual cues are predefined hints that specify how the hand gesture and voice command should be interpreted together. For example, a gesture like a swipe combined with a voice command like "select" may be ambiguous, but a textual cue such as "swipe to select" clarifies the intended action. The system processes these inputs to determine the user's intent and executes the corresponding command on a device, such as a computer, smartphone, or smart home system. By incorporating textual cues, the system enhances the precision of multimodal input interpretation, making interactions more intuitive and reducing errors. This approach is particularly useful in environments where gestures and voice commands alone may be insufficient or prone to misinterpretation.
6. The method of claim 1, wherein the textual cues are temporarily displayed on the GUI for a predetermined duration of time.
This invention relates to graphical user interfaces (GUIs) that incorporate textual cues to guide user interactions. The problem addressed is the need to provide temporary, non-intrusive guidance within a GUI to assist users in navigating or completing tasks without permanently altering the interface. The solution involves displaying textual cues on the GUI for a predetermined duration, ensuring they appear only when needed and do not clutter the interface long-term. These cues may include instructions, hints, or contextual information that appear dynamically based on user actions or predefined triggers. The cues are designed to fade or disappear after the set duration, maintaining a clean and unobstructed interface. This approach enhances usability by offering timely assistance without overwhelming the user with persistent on-screen elements. The method ensures that the cues are contextually relevant and do not disrupt the user experience by remaining visible only for the necessary time. This temporary display mechanism is particularly useful in applications where user guidance is required intermittently, such as software tutorials, workflow assistance, or error recovery prompts. The invention improves user interaction efficiency by providing just-in-time information without requiring manual dismissal or permanent interface modifications.
13. The computing system of claim 12, wherein the selection is a user input received at the computer device, and wherein the augmented reality experience is displayed as a selectable user interface element within the application.
This invention relates to computing systems that provide augmented reality (AR) experiences within applications. The system addresses the challenge of integrating AR features in a way that is intuitive and user-friendly, ensuring seamless interaction between users and digital content overlaid on the real world. The computing system includes a computer device configured to display an application with an augmented reality experience. The AR experience is presented as a selectable user interface element, allowing users to easily access and interact with AR content. User input received at the computer device triggers the selection of the AR experience, enabling dynamic and interactive overlays within the application. The system may also include a server that communicates with the computer device to provide AR content, ensuring that the experience is responsive and contextually relevant. The AR experience can be customized based on user preferences or environmental factors, enhancing engagement and usability. This approach improves the accessibility of AR features, making them more integrated and less intrusive within applications.
14. The computing system of claim 11, wherein the textual cues are hints associated with the hand gesture and the voice command.
This invention relates to computing systems that integrate hand gestures and voice commands for user interaction. The system addresses the challenge of improving the accuracy and efficiency of multimodal input by associating textual cues with hand gestures and voice commands. These cues serve as hints to enhance the system's ability to interpret and respond to user inputs. The computing system includes a processor and memory storing instructions that, when executed, enable the system to process hand gestures and voice commands. The system captures and analyzes these inputs to determine user intent. Textual cues are linked to specific gestures and commands, providing additional context to disambiguate ambiguous inputs. For example, if a user performs a gesture while speaking a command, the associated textual cue helps the system distinguish between similar gestures or commands. The system may also include a display for visual feedback, ensuring users receive clear responses to their inputs. The integration of textual cues improves the system's performance by reducing errors and enhancing responsiveness. This approach is particularly useful in applications requiring precise control, such as virtual reality, gaming, or assistive technologies. The invention aims to create a more intuitive and reliable multimodal interaction experience.
15. The computing system of claim 11, wherein the textual cues are temporarily displayed on the GUI for a predetermined duration of time.
A computing system provides a graphical user interface (GUI) that enhances user interaction by displaying textual cues to guide or inform users. The system includes a display device for presenting the GUI, a processor, and a memory storing instructions executable by the processor to generate and manage the GUI. The textual cues are dynamically displayed on the GUI to assist users in navigating or performing tasks. These cues are temporarily shown for a predetermined duration, ensuring they do not persist indefinitely and avoid cluttering the interface. The system may also include input devices for user interaction, such as keyboards, mice, or touchscreens, and may further incorporate machine learning models to adapt the cues based on user behavior or context. The temporary display of cues helps maintain a clean and unobtrusive interface while providing timely guidance. This approach is particularly useful in applications where user assistance is needed without overwhelming the display with persistent information.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
May 10, 2022
April 16, 2024
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.