Embodiments may relate to intuitive user-interface features for a head-mountable device (HMD), in the context of a hybrid human and computer-automated response system. An illustrative method may involve a head-mountable device (HMD) that comprises a touchpad: (a) sending a speech-segment message to a hybrid response system, wherein the speech-segment message is indicative of a speech segment that is detected in audio data captured at the HMD, and wherein the speech-segment is associated with a first user-account with the hybrid response system, (b) receiving a response message that includes a response to the speech-segment message and an indication of a next action corresponding to the response to the speech-segment message, (c) displaying a screen interface that includes an indication of the response, and (d) while displaying the response, detecting a singular touch gesture and responsively initiating the at least one next action.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method comprising: detecting a first touch gesture on a touch interface of a client computing device, and responsively sending a speech-segment message to a hybrid response system, wherein the speech-segment message comprises an initial query corresponding to a speech segment in audio data captured at the client computing device; receiving, at the client computing device, a response message comprising both: (a) an answer to the initial query, and (b) an indication of a next action that is cued as a response to an anticipated follow-up request related to the answer to the initial query; and in response to receiving the response message, the client computing device: (a) displaying, on a graphic display associated with the client computing device, a first screen indicating the answer to the initial query, and (b) while displaying the first screen interface, detecting touchpad data corresponding to a second touch gesture on the touch interface and responsively initiating and displaying a second screen interface corresponding to the next action.
2. The method of claim 1 , wherein the second touch gesture comprises a single tap on the touch interface.
3. The method of claim 1 , wherein initiating the next action comprises displaying a first screen interface that indicates the information related to the response.
4. The method of claim 1 , wherein the indication of the next action comprises an indication of a transaction that is associated with the response message.
5. The method of claim 4 , wherein the response message comprises an indication of a product or service, and wherein the transaction comprises a purchase of the product.
6. The method of claim 4 , wherein the speech-segment message is associated with a first user-account, and wherein initiating the next action comprises initiating a purchase of a product or service via a second user-account that is associated with the first user-account.
7. The method of claim 6 , wherein initiating the at least one next action comprises displaying the first screen interface that requests input that indicates whether or not to purchase a product or service via a second user-account associated with the first user-account.
8. The method of claim 1 , wherein the speech-segment message is associated with a first user-account, and wherein the response message comprises proposed text for a message that can be sent via at least one second user-account associated with the first user-account, and wherein the indication of the next action comprises an indication to display the first screen interface that indicates message-capable applications associated with the first user-account.
9. The method of claim 8 , wherein initiating the next action comprises displaying the first screen interface that indicates the one or more message-capable applications associated with the first user-account.
10. The method of claim 1 , wherein the speech-segment message is associated with a first user-account, wherein the response message comprises proposed text for the message that can be sent via at least one second user-account associated with the first user-account, and wherein the indication of the next action comprises an indication to send a message that includes the proposed text via the at least one second user-account.
11. The method of claim 10 , wherein initiating the next action comprises sending, via the at least one second user-account, the message that includes the proposed text.
12. A computing device comprising: at least one microphone configured to provide an audio signal based on speech detected at the computing device; a touch interface; a control system configured to: detect a first touch gesture on the touch interface and responsively send a speech-segment message to a hybrid response system, wherein the speech-segment message comprises an initial query corresponding to a speech segment in audio data captured using the at least one microphone; receiving a response message comprising both: (a) an answer to the initial query, and (b) an indication of a next action that is cued as a response to an anticipated follow-up request related to the answer to the initial query; and in response to receipt of the response message: (a) display, on a graphic display associated with the client computing device, a first screen interface indicating the answer to the initial query, and (b) while displaying the first screen interface, detect touch data corresponding to a second touch gesture on the touch interface and responsively initiate and display a second screen interface corresponding to the next action.
13. The system of claim 12 , wherein the touch gesture comprises a single tap on the touch interface.
14. The system of claim 12 , wherein the indication of the next action comprises information related to the response, and wherein initiating the at least one next action comprises displaying a screen interface that indicates the information related to the response.
15. The system of claim 12 , wherein the indication of the next action comprises an indication of a transaction that is associated with the response, and wherein the response comprises an indication of a product or service, and wherein the transaction comprises a purchase of the product.
16. A non-transitory computer readable medium having stored therein instructions executable by at least one processor to cause the computing device of a client computing device to perform functions comprising: detecting a first touch gesture on a touch interface of the client computing device, and responsively sending a speech-segment message to a hybrid response system, wherein the speech-segment message comprises an initial query corresponding to a speech segment in audio data captured at the client computing device; receiving, at the client computing device, a response message comprising both: (a) an answer to the initial query, and (b) an indication of a next action that is cued as a response to an anticipated follow-up request related to the answer to the initial query; and in response to receiving the response message, the client computing device: (a) displaying, on a graphic display associated with the client computing device, a first screen interface indicating the answer to the initial query, and (b) while displaying the first screen interface, detecting touchpad data corresponding to a second touch gesture on the touch interface and responsively initiating and displaying a second screen interface corresponding to the next action.
17. The non-transitory computer readable medium of claim 16 , wherein the second touch gesture comprises a single tap on the touch interface.
18. A method carried out by a hybrid response computing system comprising: receiving a speech-segment message from a client computing device, wherein the speech-segment message is indicative of a speech segment detected in audio data captured at the client computing device by at least one microphone, wherein the speech-segment is associated with a first user-account with the hybrid response computing system, and wherein the speech-segment message comprises an initial query; in response to receipt of the speech segment message, and without use of any subsequent input from the user-facing computing device, the hybrid response computing system: determining (a) an answer to the initial query, (b) an anticipated follow-up request related to the answer, and (c) a next action responsive to the anticipated follow-up request, wherein the next action is cued to be performed in response to detection of a singular touch gesture while the answer is displayed at the client computing device; sending, to the client computing device, a response message that: (a) comprises the answer to the initial query and (b) does not indicate to the user-facing computing device to display the next action; receiving, by the hybrid response computing device, an input message indicating receipt of touchpad data corresponding to performance of a singular touch gesture on the touchpad of the client computing device while a first card interface including an indication of the answer to the initial query is displayed on a graphic display associated with the client computing device; responsive to the input message, the hybrid response computing system initiating the cued next action corresponding to the anticipated follow-up request.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
January 27, 2017
October 24, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.