Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for providing a graphical user interface to determine intent of a user based at least in part on movement of an eye or eyes of the user, the method comprising: displaying a plurality of menu items on a display; identifying, with a detector, when at least one eye of the user is directed at a first menu item of the plurality of menu items; identifying, with the detector, a saccade of the at least one eye of the user from the first menu item towards an activation icon on the display; confirming, with the detector, that the saccade of the at least one eye of the user is completed to within a predetermined distance from a location of the activation icon; and initiating performance of one or more actions in relation to the first menu item, wherein a visual confirmation is presented on the display indicating that the at least one eye of the user has completed the saccade to within the predetermined distance from the location of the activation icon, wherein the visual confirmation comprises replacing the activation icon with an icon corresponding to the first menu item.
A method for controlling a graphical user interface using eye movements. The method displays a menu with multiple items on a screen. An eye tracker detects when the user looks at a specific menu item. If the eye tracker then detects the user's eye quickly moves (saccades) from the selected menu item toward a designated activation icon on the screen, and the saccade ends close enough to the icon, then the system performs an action related to that menu item. To visually confirm the selection, the activation icon changes to a new icon representing the selected menu item.
2. The method of claim 1 , wherein a visual confirmation is presented on the display indicating that the at least one eye of the user is directed at the first menu item.
In addition to the base method, the system provides visual feedback when the user is looking at a menu item, before the saccade to the activation icon even occurs. This visual confirmation helps the user know which menu item is currently selected by their gaze.
3. The method of claim 2 , wherein the visual confirmation comprises one of the group consisting of presenting a border at least partially around the first menu item, changing brightness of the first menu item, changing color of the first menu item, adding an indicator in a region of the first menu item, and adding imagery on the display that augments the first menu item.
The visual confirmation provided when the user is looking at a menu item (before saccade) can take several forms: displaying a border around the menu item, changing its brightness or color, adding a special indicator inside the menu item's area, or adding extra imagery on the screen to enhance the appearance of the selected item.
4. The method of claim 1 , wherein the plurality of menu items comprise a plurality of images, wherein the first menu item is one of the plurality of images, and wherein the action comprises replacing the plurality of images on the display with an image corresponding to the first menu item.
If the displayed menu items are images, selecting one with the gaze-directed saccade method replaces all the images on the screen with a single, larger image corresponding to the selected menu item.
5. The method of claim 1 , wherein the icon comprises one of the group consisting of a thumbnail of the first menu item, a representation of the first menu item, a likeness of the first menu item, an image of the first menu item, and a portion of the first menu item.
The icon used to replace the activation icon (confirming the selection) can be a small version (thumbnail) of the selected menu item, a symbolic representation of it, a close likeness, a direct image of it, or just a portion of the menu item.
6. The method of claim 1 , wherein the one or more actions comprise identifying the first menu item from a database of object templates.
The action performed when a menu item is selected using the gaze-directed saccade method can involve identifying the selected menu item from a database of object templates. This could be used for object recognition applications.
7. The method of claim 1 , wherein the one or more actions comprise purchasing an item corresponding to the first menu item.
The action performed when a menu item is selected using the gaze-directed saccade method can be purchasing an item that corresponds to the selected menu item, essentially enabling eye-controlled shopping.
8. The method of claim 1 , wherein the one or more actions comprise retrieving first menu item information related to the identified first menu item from one or more databases of information about identified objects.
The action performed when a menu item is selected using the gaze-directed saccade method can be retrieving information about the selected object from one or more databases. This information would be related to the identified object.
9. The method of claim 8 , wherein the one or more actions further comprise displaying the first menu item information on the display.
After retrieving information about the selected menu item (using gaze-directed saccade), the system displays this information on the screen for the user to see.
10. The method of claim 8 , wherein the one or more actions action further comprise performing one or more additional actions related to the first menu item information.
After retrieving information about the selected menu item (using gaze-directed saccade), the system performs additional actions based on this information.
11. The method of claim 10 , wherein the one or more additional actions comprise sending information regarding an identity of the user to a processor that is operatively coupled to the identified first menu item.
The system can send the user's identity to a processor connected to the selected menu item after using gaze-directed saccade to select the item and retrieving related information.
12. The method of claim 1 , wherein the performance of the one or more actions is initiated within less than 250 milliseconds after the saccade is completed within the predetermined distance from the location of the activation icon.
The action associated with the selected menu item is initiated very quickly, within 250 milliseconds after the user's eye movement (saccade) ends near the activation icon.
13. The method of claim 1 , wherein the performance of the one or more actions is initiated within less than 200 milliseconds after the saccade is completed within the predetermined distance from the location of the activation icon.
The action associated with the selected menu item is initiated extremely quickly, within 200 milliseconds after the user's eye movement (saccade) ends near the activation icon.
14. The method of claim 1 , wherein the one or more actions comprise launching an application associated with the first menu item.
The action associated with the selected menu item is to launch an application that is associated with the selected menu item.
15. The method of claim 1 , wherein the plurality of menu items comprise icons associated with respective applications.
The menu items that can be selected with the gaze-directed saccade method are icons, where each icon is linked to a specific application. Selecting the icon launches the corresponding application.
16. The method of claim 1 , wherein the activation icon is presented at a fixed location on the display independent from a field on the display including the plurality of menu items.
The activation icon, used as the target for the user's eye saccade, is located in a fixed place on the screen, separate from the area where the menu items are displayed.
17. The method of claim 1 , wherein each of the plurality of menu items is associated with a different action and wherein the one or more actions comprises initiating performance of an action associated with the first menu item that is different than actions performed by other menu items of the plurality of menu items.
Each menu item triggers a different action. Selecting one menu item with the gaze-directed saccade method performs a different action than selecting other menu items in the menu.
18. A system for providing a graphical user interface to determine intent of a user, the system comprising: a detector configured to monitor movement of at least one eye of the user; an electronic display operatively associated with the detector; a processing unit operatively coupled to the detector and electronic display; and a module stored in memory, the module to configure the processing unit to: display a plurality of menu items on the display; identify when at least one eye of the user is directed at a first menu item of the plurality of menu items on the display; identify a saccade of the at least one eye of the user from the first menu item towards an activation icon on the display; confirm that the saccade of the at least one eye is completed to within a predetermined distance from a location of the activation icon on the display; and initiate performance of one or more actions in relation to the first menu item including presenting a visual confirmation on the display indicating that the at least one eye of the user has completed the saccade to within the predetermined distance from the location of the activation icon, wherein the visual confirmation comprises replacing the activation icon with an icon corresponding to the first menu item.
A system that allows a user to interact with a graphical user interface through eye movements. It includes an eye tracker, a display screen, and a processor. Software instructs the processor to show a menu of items, detect when the user looks at one of those items, detect when the user's eye moves quickly (saccades) from that item towards an activation icon on the screen, and verify that the eye movement ends close to the icon. If so, it starts an action related to the selected menu item. The system confirms the selection visually by changing the activation icon to a new icon representing the selected menu item.
19. The system of claim 18 , wherein the module further is to configure the processing unit to provide visual confirmation on the display that the at least one eye of the user is directed at the first menu item.
The eye tracking system also provides visual confirmation that the user's gaze is directed at a particular menu item even before the saccade to the activation icon occurs. This feedback helps the user ensure the correct item is being selected.
20. The system of claim 18 , wherein the module is to configure the processing unit to initiate performance of the one or more actions within less than 250 milliseconds after the saccade is completed within the predetermined distance from the location of the activation icon.
The eye tracking system initiates the action associated with the selected menu item very quickly, within 250 milliseconds after the user's eye movement (saccade) ends near the activation icon.
21. A system for providing a graphical user interface to determine intent of a user, the system comprising: a detector configured to monitor movement of at least one eye of the user; a processing unit operatively coupled to the detector and a display; and a module stored in memory, the module to configure the processing unit to: display a plurality of menu items on the display; identify when at least one eye of the user is directed at a first menu item of the plurality of menu items at a first location on the display; identify a saccade of the at least one eye of the user from the first menu item towards an activation icon at a second location on the display; confirm that the saccade of the at least one eye is completed to within a predetermined distance from the second location of the activation icon; and initiate performance of one or more actions in relation to the first menu item, the one or more actions comprising replacing the activation icon at the second location with an icon corresponding to the first menu item.
A system that uses eye tracking to control a graphical interface. The system has an eye tracker, a processor, and a display. The processor displays menu items, detects when a user looks at one item at a first location on the display and then quickly moves their eyes (saccades) to an activation icon at a second location on the display, confirming the eye movement ends close enough to the activation icon. When this happens, the action to initiate includes changing the activation icon at the second location to an icon representing the selected menu item.
22. The system of claim 21 , wherein initiating performance of the one or more actions further comprises initiating an action associated with the first menu item.
In addition to changing the activation icon, the system initiates another action specifically associated with the menu item the user selected with their eyes (gaze-directed saccade).
23. The system of claim 21 , wherein the icon comprises one of the group consisting of a thumbnail of the first menu item, a representation of the first menu item, a likeness of the first menu item, an image of the first menu item, and a portion of the first menu item.
The icon used to replace the activation icon (confirming the selection) can be a small version (thumbnail) of the selected menu item, a symbolic representation of it, a close likeness, a direct image of it, or just a portion of the menu item.
24. A method for providing a graphical user interface to determine intent of a user, the method comprising: displaying a plurality of menu items on a display; identifying, with a detector, when at least one eye of the user is directed at a first menu item of the plurality of menu items at a first location; identifying, with the detector, a saccade of the at least one eye of the user from the first location towards an activation icon at a second location on the display; confirming, with the detector, that the saccade of the at least one eye of the user is completed from the first menu item to within a predetermined distance from the second location of the activation icon; and initiating performance of one or more actions in relation to the first menu item, the one or more actions comprising replacing the activation icon at the second location with an icon corresponding to the first menu item.
A method for controlling a graphical user interface using eye movements: The method shows a menu of items on a screen. An eye tracker detects when the user looks at a menu item at a first location. The tracker identifies a fast eye movement (saccade) from the first location to an activation icon at a second location on the screen. The system confirms the eye movement ends close enough to the activation icon and then performs one or more actions related to that menu item. The action to initiate includes changing the activation icon at the second location to an icon representing the selected menu item.
25. The method of claim 24 , wherein the menu items comprise a plurality of images, wherein the first menu item is a first image of the plurality of images; and wherein the one or more actions further comprise presenting an image of the first image on the display.
If the displayed menu items are images, selecting one with the gaze-directed saccade method and replacing the activation icon also presents an image of the selected image on the display.
26. The method of claim 24 , wherein initiating performance of the one or more actions further comprises initiating an action associated with the first menu item.
In addition to changing the activation icon, the system initiates another action specifically associated with the menu item the user selected with their eyes (gaze-directed saccade).
27. The method of claim 24 , wherein the icon comprises one of the group consisting of a thumbnail of the first menu item, a representation of the first menu item, a likeness of the first menu item, an image of the first menu item, and a portion of the first menu item.
The icon used to replace the activation icon (confirming the selection) can be a small version (thumbnail) of the selected menu item, a symbolic representation of it, a close likeness, a direct image of it, or just a portion of the menu item.
28. The method of claim 24 , wherein the one or more actions comprise launching an application associated with the first menu item.
The action associated with the selected menu item is to launch an application that is associated with the selected menu item.
Unknown
November 21, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.