Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. An electronic device comprising: a housing; a touch screen display that is located inside the housing and is exposed through a first portion of the housing; a microphone that is located inside the housing and is exposed through a second portion of the housing; at least one speaker that is located inside the housing and is exposed through a third portion of the housing; a wireless communication circuit that is located inside the housing; a processor that is located inside the housing and is electrically connected to the touch screen display, the microphone, the speaker, and the wireless communication circuit; and a memory that is located inside the housing and is electrically connected to the processor, wherein the memory stores instructions that, when executed, cause the processor to: receive a first user input via at least one of the touch screen display and the microphone, the first user input including a request to perform a task using at least one of the electronic device and an external device, transmit first data associated with the first user input to an external server via the wireless communication circuit, receive a first response from the external server via the wireless communication circuit, the first response including information associated with the first user input and/or sequence information of states of the electronic device for performing at least a portion of the task, wherein the sequence information corresponds to at least one representative dialog stored in the memory, and wherein when an input corresponding to the at least one representative dialog is received from a user by the processor, at least a portion of the task is performed, after receiving the first response, in response to receiving a second user input including an utterance and/or a user operation via the touch screen display, set a command shortcut to be mapped to the at least one representative dialog, the command shortcut including a text which is acquired based on the utterance and/or the user operation, wherein after the command shortcut is set, when an input including the utterance and/or the user operation is received from the user by the processor, at least a portion of the task is performed, and transmit second data associated with the second user input to the external server via the communication circuit.
This invention relates to an electronic device with voice and touch-based interaction capabilities, designed to streamline task execution through automated dialog sequences and customizable command shortcuts. The device includes a housing containing a touch screen display, microphone, speaker, wireless communication circuit, processor, and memory. The processor executes instructions to receive user inputs via touch or voice, transmit associated data to an external server, and receive responses containing task-related information or sequence data for performing tasks. The device stores representative dialogs in memory, allowing tasks to be triggered when user inputs match these dialogs. Additionally, the device enables users to create command shortcuts based on voice utterances or touch operations, mapping them to specific dialogs for faster task execution. After setting a shortcut, subsequent matching inputs automatically perform the associated task. The device also transmits data from these interactions to the external server, facilitating further processing or analysis. This system enhances user efficiency by reducing repetitive inputs and enabling seamless task automation through learned dialog patterns and customizable shortcuts.
2. The electronic device of claim 1 , wherein the instructions further cause the processor to: receive a third user input via the at least one of the touch screen display or the microphone after the second data is transmitted, the third user input including the utterance and/or the user operation, transmit third data associated with the third user input to the external server via the wireless communication circuit, receive a second response from the external server via the wireless communication circuit, the second response including the sequence information of the states of the electronic device for performing at least the portion of the task, and perform at least the portion of the task according to the sequence information.
This invention relates to an electronic device with enhanced voice and touch-based interaction capabilities for performing tasks. The device includes a processor, a touch screen display, a microphone, and a wireless communication circuit. The device receives user inputs, such as voice commands or touch operations, and transmits associated data to an external server. The server processes this data and returns sequence information detailing the states of the device required to perform a task. The device then executes the task according to this sequence. The invention improves upon prior systems by allowing iterative interaction, where a user can provide additional inputs after an initial response, enabling more complex or refined task execution. The device dynamically adjusts its operations based on subsequent user inputs, ensuring accurate and context-aware task performance. This system is particularly useful in applications requiring multi-step processes, such as smart home control, virtual assistants, or automated workflows, where precise and adaptive responses are essential. The invention ensures seamless integration between user commands and device actions, enhancing usability and efficiency.
3. The electronic device of claim 1 , wherein the instructions further cause the processor to: store a log associated with the first user input received via the touch screen display, and transmit the log as a part of the first data.
This invention relates to electronic devices with touch screen displays, focusing on improving user input handling and data transmission. The device includes a touch screen display, a processor, and memory storing instructions that, when executed, enable the device to detect a first user input via the touch screen display and generate first data based on the input. The device then transmits this data to a remote server or another device. The invention further includes storing a log of the first user input and transmitting this log as part of the first data. This log may include details such as the type of input (e.g., tap, swipe, or gesture), timing, coordinates, or other metadata. The log transmission allows for remote analysis, debugging, or user behavior tracking. The device may also receive a response based on the transmitted data, which could include feedback, corrections, or additional instructions. The invention aims to enhance user interaction by ensuring accurate input capture and enabling remote processing or monitoring of touch interactions. This is particularly useful in applications requiring precise input tracking, such as accessibility features, security authentication, or user experience analytics.
4. The electronic device of claim 1 , wherein the utterance includes an utterance personalized by the user, and wherein the user operation includes a text input and/or an input selecting a graphic user interface by the user.
This invention relates to electronic devices that process user inputs, including voice and graphical interface interactions. The problem addressed is improving user interaction by personalizing responses based on user-specific utterances and inputs. The device includes a processor and memory storing instructions for processing user operations, such as text input or selections from a graphical user interface (GUI). The system captures and analyzes user utterances, which may be personalized by the user, to generate context-aware responses. The device also interprets user inputs, such as typed text or GUI selections, to further refine interactions. By combining voice and graphical inputs, the system enhances user experience through adaptive and personalized responses. The invention aims to bridge voice and visual interfaces, ensuring seamless and intuitive user engagement. The device may include additional features like natural language processing (NLP) to interpret personalized utterances and machine learning to adapt to user preferences over time. The system ensures that user inputs, whether spoken or graphical, are accurately processed to provide relevant and customized outputs. This approach improves accessibility and efficiency in human-computer interaction.
5. The electronic device of claim 1 , wherein: the at least one representative dialog includes a request to perform the task, and the instructions further cause the processor to: receive a voice via the microphone, and transmit fourth data including another representative dialog corresponding to the voice to the external server.
This invention relates to electronic devices with voice interaction capabilities, specifically addressing the challenge of enabling natural, context-aware voice-based task execution. The device includes a microphone, a processor, and a network interface for communicating with an external server. The system is designed to facilitate task performance through voice commands by generating representative dialogs that reflect user intent. When a user requests a task via voice input, the device captures the audio and transmits it to the server, which processes the request and returns a response dialog. The device then outputs this response, allowing the user to interact with the system conversationally. The invention further enables iterative dialog by capturing subsequent user responses and transmitting them to the server for continued processing, ensuring seamless task execution through natural language interaction. This approach improves user experience by reducing the need for manual input and providing a more intuitive interface for device control. The system is particularly useful in smart home devices, virtual assistants, and other voice-activated applications where efficient, context-aware communication is essential.
6. The electronic device of claim 5 , wherein the instructions further cause the processor to: transmit the voice to the external server via the wireless communication circuit, receive text data corresponding to the voice from the external server via the wireless communication circuit, and identify the other representative dialog matched with the command shortcut if the text data corresponds to the command shortcut.
This invention relates to electronic devices with voice command processing capabilities. The problem addressed is the need for efficient and accurate voice command recognition, particularly in scenarios where users may use command shortcuts or representative dialogs to trigger specific functions. The invention improves upon prior systems by integrating cloud-based voice-to-text conversion and local command matching to enhance responsiveness and accuracy. The electronic device includes a processor, a wireless communication circuit, and memory storing instructions. The device captures voice input from a user and transmits it to an external server via the wireless communication circuit. The server processes the voice and returns text data corresponding to the spoken words. The device then analyzes the text data to determine if it matches a predefined command shortcut. If a match is found, the device identifies a corresponding representative dialog, which is a predefined response or action associated with the shortcut. This allows the device to execute commands quickly without requiring the user to speak full phrases, improving efficiency and user experience. The system ensures that even if the user's voice input is partially recognized, the command can still be executed if the text data corresponds to a known shortcut. This approach leverages cloud-based processing for accurate transcription while maintaining local control for command execution.
7. The electronic device of claim 5 , wherein the instructions further cause the processor to: receive the sequence information of the states of the electronic device that corresponds to the second data and is used to perform at least the portion of the task via the wireless communication circuit after the second data is transmitted, and perform at least the portion of the task.
This invention relates to electronic devices configured to perform tasks using wireless communication. The problem addressed is the need for efficient task execution in electronic devices that rely on wireless data transmission, particularly when the device must perform a task based on received data and then report its state back to a source. The electronic device includes a processor, a wireless communication circuit, and memory storing instructions. The instructions cause the processor to transmit second data to another device via the wireless communication circuit. After transmission, the device receives sequence information corresponding to the second data, which describes the states of the electronic device as it performs at least a portion of the task. The device then uses this sequence information to execute the task or a portion of it. This ensures that the device can dynamically adjust its operations based on the received state information, improving task execution efficiency and reliability in wireless communication environments. The invention may be part of a larger system where the device interacts with other components to complete tasks, leveraging wireless data exchange to coordinate operations.
8. The electronic device of claim 1 , wherein: the at least one representative dialog includes a request to perform the task, and the instructions further cause the processor to: receiving a text input via the touch screen display, and transmit fifth data including another representative dialog corresponding to the text input to the external server.
This invention relates to electronic devices with natural language processing capabilities for task automation. The problem addressed is the lack of intuitive, conversational interfaces for users to request and manage automated tasks on their devices. The solution involves an electronic device with a touch screen display and a processor executing instructions to facilitate task automation through dialog-based interactions. The device generates at least one representative dialog representing a task, which includes a request to perform the task. The device receives text input from the user via the touch screen and transmits data corresponding to this input to an external server. The server processes the input and returns another representative dialog, enabling a back-and-forth conversation to refine or execute the requested task. This system allows users to interact with automated tasks in a natural, conversational manner, improving accessibility and ease of use. The device may also include sensors and communication interfaces to gather contextual data, which can be used to enhance the dialog generation and task execution. The external server may employ machine learning models to interpret user inputs and generate appropriate responses, ensuring accurate task automation. This approach streamlines task management by reducing the need for complex manual inputs, making automation more accessible to a broader range of users.
9. The electronic device of claim 8 , wherein the instructions further cause the processor to identify the other representative dialog matched with the command shortcut if the text input corresponds to the command shortcut.
This invention relates to electronic devices with natural language processing capabilities, specifically for improving user interaction through command shortcuts. The problem addressed is the inefficiency in traditional command input methods, which often require users to input lengthy or complex commands, leading to delays and reduced productivity. The electronic device includes a processor and memory storing instructions that, when executed, enable the device to process user input. The device receives text input from a user and determines whether it corresponds to a predefined command shortcut. If a match is found, the device identifies a representative dialog associated with the command shortcut. This dialog serves as a simplified or alternative representation of the command, allowing the user to execute functions more efficiently. The system may also include a database of command shortcuts and their corresponding representative dialogs, which can be dynamically updated or learned from user behavior. The device may further analyze the context of the text input to improve the accuracy of shortcut matching, ensuring that the correct command is executed even if the input is ambiguous or partially incomplete. This approach enhances user experience by reducing the need for verbose or technical command inputs, making the device more intuitive and responsive.
10. A method of executing an operation of an electronic device, the method comprising: receiving a first user input, wherein the first user input includes a request to perform a task using at least one of the electronic device or an external device; transmitting first data associated with the first user input to an external server; receiving a first response from the external server, wherein the first response includes information associated with the first user input and/or sequence information of states of the electronic device for performing at least a portion of the task, wherein the sequence information corresponds to at least one representative dialog, and wherein when an input corresponding to the at least one representative dialog is received from a user, at least a portion of the task is performed; after receiving the first response, in response to receiving a second user input including an utterance and/or a user operation via a touch screen display of the electronic device, setting a command shortcut to be mapped to the at least one representative dialog, the command shortcut including a text which is acquired based on the utterance and/or the user operation, wherein after the command shortcut is set, when an input including the utterance and/or the user operation is received from the user, at least a portion of the task is performed; and transmitting second data associated with the second user input to the external server.
This invention relates to a method for executing operations in an electronic device by leveraging user inputs, external servers, and command shortcuts. The method addresses the problem of efficiently performing tasks on electronic devices, particularly when those tasks involve interactions with external devices or servers. The system receives a first user input requesting a task, which may involve the electronic device or an external device. This input is transmitted to an external server, which responds with information related to the request or sequence information detailing the states of the device required to perform the task. The sequence information corresponds to at least one representative dialog, meaning that when a user provides an input matching this dialog, the device executes at least part of the task. After receiving this response, the system processes a second user input, which can be an utterance or a touchscreen operation. Based on this input, a command shortcut is created and mapped to the representative dialog. The shortcut includes text derived from the utterance or operation. Once set, if the user provides the same input again, the device performs the associated task portion. The system also transmits data from the second input to the external server. This method streamlines task execution by allowing users to create shortcuts for frequently used commands, reducing the need for repetitive inputs and improving efficiency.
11. The method of claim 10 , further comprising: receiving a third user input after the second data is transmitted, wherein the third user input includes the utterance and/or the user operation; transmitting third data associated with the third user input to the external server; receiving a second response from the external server, wherein the second response includes the sequence information of the states of the electronic device for performing at least the portion of the task; and performing at least the portion of the task according to the sequence information.
This invention relates to a method for controlling an electronic device based on user inputs, particularly for performing tasks by interpreting user utterances and operations. The method addresses the challenge of enabling an electronic device to execute tasks by dynamically processing user inputs and coordinating with an external server to determine the appropriate sequence of device states for task completion. The method involves receiving a first user input, which may include a spoken utterance or a physical operation, and transmitting first data associated with this input to an external server. The server processes the input and returns a first response containing sequence information for the device states required to perform at least a portion of the task. The device then executes this portion of the task according to the received sequence information. Subsequently, a second user input is received, which may also include an utterance or operation, and second data associated with this input is transmitted to the server. The server provides a second response with updated sequence information for the device states needed to complete the task. The device then performs the task or a portion of it based on this updated sequence. Additionally, the method includes receiving a third user input after the second data is transmitted, where the third input may again include an utterance or operation. Third data associated with this input is sent to the server, which returns a second response with further sequence information for the device states required to perform at least a portion of the task. The device then executes the task or a portion of it according to this latest sequence information. This iterative process allows the device to dynamically adjust task execution ba
12. The method of claim 10 , further comprising: storing a log associated with the first user input received via the touch screen display of the electronic device, and transmitting the log as a part of the first data.
This invention relates to electronic devices with touch screen displays and methods for handling user input. The problem addressed is the need to efficiently capture, log, and transmit user interactions with touch-sensitive interfaces, particularly for diagnostic, analytical, or security purposes. The method involves receiving a first user input via the touch screen display of an electronic device. The input is processed to generate first data, which may include touch coordinates, timing, or gesture patterns. A log is created to record details of the first user input, such as the type of interaction, duration, or associated metadata. This log is then transmitted as part of the first data, either to a local storage system or an external server for further analysis. The method may also include receiving a second user input, processing it to generate second data, and transmitting the second data separately or in combination with the first data. The log may be structured to include timestamps, input sequences, or contextual information (e.g., device state or application context). The transmission can occur over a network or via a direct connection, depending on the application. This approach ensures comprehensive tracking of touch interactions, enabling improved debugging, user behavior analysis, or system monitoring. The log can be used to reconstruct input sequences, identify anomalies, or optimize touch interface performance. The method is applicable to smartphones, tablets, or other touch-enabled devices where detailed input logging is required.
13. The method of claim 10 , wherein the utterance second user includes an utterance personalized by a user, and wherein the user operation includes a text input and/or an input selecting a graphic user interface by the user.
This invention relates to a method for processing user interactions in a voice-based or graphical user interface system. The method addresses the challenge of personalizing and improving user interactions by dynamically adapting to user inputs, including both voice utterances and graphical interface selections. The method involves receiving an initial user utterance, which may be a voice command or a text input, and processing it to determine an appropriate system response. The system then generates a second user utterance that is personalized based on the user's input, ensuring the interaction is tailored to the user's preferences or context. Additionally, the method allows for user operations that include text input or selections made through a graphical user interface, such as clicking buttons or selecting options. These inputs further refine the system's responses, making interactions more intuitive and efficient. By integrating voice and graphical inputs, the method enhances user experience by providing flexible and adaptive interaction modes. The system can interpret and respond to both spoken and visual inputs, ensuring seamless transitions between different input methods. This approach improves accessibility and usability, particularly in environments where users may switch between voice and graphical interfaces. The method ensures that user preferences and historical data are leveraged to personalize responses, making interactions more natural and efficient.
14. The method of claim 10 , further comprising: storing the at least one representative dialog including a request to perform the task; receiving a voice via a microphone; and transmitting fourth data including another representative dialog corresponding to the voice to the external server.
This invention relates to voice-based task automation systems, specifically improving natural language interaction with external servers to perform tasks. The problem addressed is the lack of efficient methods for capturing and reusing voice-based task requests in automated systems, particularly when integrating with external servers. The system involves a method for processing voice inputs to perform tasks by generating representative dialogs that encapsulate the task request. These dialogs are stored and later used to execute the task when similar voice inputs are received. The method includes receiving a voice input via a microphone, converting it into text, and generating a representative dialog that includes the task request. This dialog is transmitted to an external server, which processes the request and performs the task. The system also stores the representative dialog for future use, allowing it to recognize and respond to similar voice inputs more efficiently. Additionally, the system can receive subsequent voice inputs, convert them into text, and generate another representative dialog corresponding to the new voice input. This new dialog is transmitted to the external server, enabling continuous interaction and task execution. The stored dialogs improve the system's ability to handle repetitive or similar requests, reducing processing time and enhancing user experience. The invention focuses on seamless integration with external servers, ensuring that voice-based task requests are accurately translated into executable commands.
15. The method of claim 14 , further comprising: transmitting the voice to the external server; receiving text data corresponding to the voice from the external server; and identifying the other representative dialog matched with the command shortcut if the text data corresponds to the command shortcut.
This invention relates to voice-based command systems for interactive dialog management. The technology addresses the challenge of efficiently processing voice commands in a dialog system, particularly when integrating with external servers for speech-to-text conversion and command recognition. The method involves capturing a voice input from a user and transmitting it to an external server for conversion into text data. The system then analyzes the received text data to determine if it matches a predefined command shortcut. If a match is found, the system identifies a corresponding representative dialog associated with the command shortcut. This allows the system to execute or retrieve the appropriate response or action based on the recognized command, improving the efficiency and responsiveness of the dialog system. The method may also include generating a response dialog based on the identified representative dialog and outputting the response dialog to the user. This ensures that the system provides a coherent and contextually relevant reply to the user's voice command. The system may further include a database or storage mechanism for storing command shortcuts and their associated representative dialogs, enabling quick retrieval and processing of commands. By leveraging external servers for speech-to-text conversion, the system can offload processing tasks, reducing computational overhead and improving scalability. The method ensures seamless integration of voice commands into interactive dialog systems, enhancing user experience and system performance.
16. A non-transitory machine-readable storage medium recording a program to perform a method of executing an operation of an electronic device, the method comprising: receiving a first user input, wherein the first user input includes a request to perform a task using at least one of the electronic device or an external device; transmitting first data associated with the first user input to an external server; receiving a first response from the external server, wherein the first response includes information associated with the first user input and/or sequence information of states of the electronic device for performing at least a portion of the task, wherein the sequence information corresponds to at least one representative dialog, and wherein when an input corresponding to the at least one representative dialog is received from a user, at least a portion of the task is performed; after receiving the first response, in response to receiving a second user input including an utterance and/or a user operation via a touch screen display of the electronic device, setting a command shortcut to be mapped to the at least one representative dialog, the command shortcut including a text which is acquired based on the utterance and/or the user operation, wherein after the command shortcut is set, when an input including the utterance and/or the user operation is received from the user, at least a portion of the task is performed; and transmitting second data associated with the second user input to the external server.
This invention relates to a system for executing operations on an electronic device based on user inputs and interactions with an external server. The system addresses the problem of simplifying task execution by dynamically creating command shortcuts based on user interactions. When a user provides a first input requesting a task, the device sends associated data to an external server, which responds with information about the task and sequence information detailing the device states required to perform the task. This sequence information corresponds to representative dialogs that, when triggered by user input, execute at least part of the task. After receiving the server response, the device can set a command shortcut mapped to a representative dialog. This shortcut is created based on a second user input, which may include an utterance or touchscreen operation. Once set, the shortcut allows the user to perform the task by repeating the utterance or operation. The device also transmits data from the second input to the server, enabling further processing or learning. The system enhances user experience by reducing repetitive inputs for common tasks through adaptive shortcut creation.
Unknown
January 28, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.