A computing device for identifying a gesture performed by a tracked object in an image stream, said computing device comprising a memory and a controller, wherein said controller is configured to: determine a movement vector for the tracked object, compare the movement vector against a reference library of gestures, and thereby identify a gesture matching the movement vector, wherein said movement vector comprises at least one partial movement having a direction being identified by a relative angle and wherein said comparison is based on said relative angle.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A computing device for identifying a gesture performed by a tracked object in an image stream, said computing device comprising a memory and a controller, wherein said controller is configured to: determine a movement vector for the tracked object; compare the movement vector against a reference library of gestures, and thereby identify a gesture matching the movement vector, wherein said movement vector comprises at least one partial movement having a direction being identified by a relative angle (α) and wherein said comparison is based on said relative angle (α), wherein said controller is further configured to traverse a state diagram, wherein a transition corresponds to an element of said movement vector and at least one state corresponds to an action; whereby the controller executes a transition from a state to another according to elements of the movement vector until a gesture is identified by reaching the corresponding state, and wherein said controller is further configured to preemptively execute an action corresponding to an assumed gesture as said gesture is being detected, assume that the gesture being detected is an assumed gesture if the state of the state diagram currently being visited is on a transition path to said gesture to be detected, determine a distance of a movement (dist), determine if the distance of the movement is above a first distance threshold value (VI), and if so, compare an angle of the movement with a library angle (β).
A computing device identifies gestures from a tracked object in a video stream. It determines a movement vector (direction and magnitude) of the object and compares it to a gesture library. The comparison uses relative angles of partial movements within the movement vector. The device uses a state diagram where transitions represent movement elements and states represent actions. The device preemptively executes actions for an assumed gesture if the current state is on the path to detecting that gesture. If a movement's distance exceeds a threshold, its angle is compared against a library angle.
2. The computing device of claim 1 , wherein said state diagram comprises at least one stable state, wherein a stable state is a state corresponding to an action or an initial state and wherein said controller is further configured to return to a previous stable state when a no other transition exists in the state diagram for the elements in the movement vector.
The gesture recognition system described in claim 1 uses a state diagram containing stable states (representing actions or the initial state). If the movement vector doesn't match any available transitions from the current state, the system reverts to the previous stable state. This enables the system to recover from incomplete or noisy gesture inputs by returning to a known valid state and waiting for further, clearer input.
3. The computing device of claim 2 , wherein said controller is further configured to if no further partial movement is detected within a timeout threshold (T 1 ).
Building upon the gesture recognition system using state diagrams and stable states as in claim 2, if no further movement is detected within a defined timeout period (T1), the system returns to a previous stable state. This handles situations where a user pauses mid-gesture, preventing the system from indefinitely waiting for the gesture to complete and improving responsiveness.
4. The computing device of claim 1 , wherein said controller is further configured to present or effect a result of the actions corresponding to the assumed gesture as the state for the gesture to be detected is reached.
The gesture recognition system described in claim 1 preemptively presents or executes the result of an action corresponding to an assumed gesture as the state for the full gesture is reached. This provides immediate feedback to the user, improving the perceived responsiveness of the system and creating a more fluid and intuitive interaction.
5. The computing device of claim 1 , wherein said state transition diagram comprises at least one stable state on a path to another stable state for enabling a first gesture to be defined by a sub-sequence of movements for a second gesture.
In the gesture recognition system described in claim 1, the state transition diagram contains at least one stable state on the path to another stable state. This allows a first gesture to be defined as a subset of movements of a second, more complex gesture. This enables gesture hierarchies, where simpler gestures can be extended to create more complex commands.
6. The computing device of claim 1 , wherein said controller is further configured to identify a gesture for a plurality of objects by traversing a plurality of state diagrams each being associated with each of the plurality of objects, wherein said controller is further configured to identity a gesture for said plurality of objects is being tracked by traversing the associated state diagram, wherein a transition in an associated state diagram corresponds to an element of a movement vector for the associated object to be tracked.
The gesture recognition system described in claim 1 can identify gestures for multiple tracked objects simultaneously. It traverses a separate state diagram for each object, where each transition corresponds to a movement element of that object. The system identifies multi-object gestures by simultaneously tracking the state of each object's associated state diagram.
7. The computing device of claim 1 , wherein said controller is further configured to identify a gesture for a plurality of objects by traversing a state diagram wherein each transition corresponds to a movement of the plurality of objects.
Instead of using a separate state diagram for each tracked object as in claim 6, this gesture recognition system identifies multi-object gestures by traversing a single state diagram where each transition represents a movement involving multiple tracked objects simultaneously. This allows defining gestures that require coordinated movements across multiple objects.
8. The computing device of claim 1 , wherein said controller is further configured to determine that the angle matches the library angle if the angle being compared is within an error tolerance (φ) of said library angle.
The gesture recognition system described in claim 1 determines if an angle matches a library angle if the measured angle is within a specified error tolerance (φ) of the stored library angle. This accounts for inaccuracies in tracking and variations in user performance, making the gesture recognition more robust.
9. The computing device of claim 1 , wherein said movement vector comprises at least one further dimension taken from a group consisting of: depth, rotation, degrees of turning, distance of movement and speed of movement.
In addition to relative angle, the movement vector used by the gesture recognition system described in claim 1 can include other dimensions like depth, rotation, degrees of turning, movement distance, and movement speed. These additional dimensions allow for more complex and nuanced gesture recognition.
10. The computing device of claim 9 , wherein said controller is configured to differentiate between a short and a long movement.
The gesture recognition system as in claim 9, that includes further dimensions in its movement vector, is configured to differentiate between short and long movements. This allows for distinguishing similar gestures based on the distance traveled by the tracked object.
11. The computing device of claim 1 , wherein said controller is configured to differentiate between a fast and a slow movement.
The gesture recognition system as in claim 9, that includes further dimensions in its movement vector, is configured to differentiate between fast and slow movements. This allows for distinguishing similar gestures based on the speed of the tracked object.
12. The computing device of claim 1 , wherein said controller is configured to determine a distance based on a size of the object to be tracked, a distance to the object to be tracked, size of a display and the field of view of a camera.
The gesture recognition system described in claim 1 determines movement distance based on the tracked object's size, its distance from the camera, the display size, and the camera's field of view. This normalizes movement distance across varying user setups and viewing conditions, providing more consistent gesture recognition.
13. A method for use in a computing device for identifying a gesture performed by a tracked object in an image stream, said method comprising: determining, using a processor, a movement vector for the tracked object; comparing, using a processor, the movement vector against a reference library of gestures, and thereby identifying a gesture matching the movement vector, wherein said movement vector comprises at least one partial movement having a direction being identified by a relative angle (α) and wherein said comparison is based on said relative angle (α), and wherein said method further comprises traversing a state diagram, wherein a transition corresponds to an element of said movement vector and at least one state corresponds to an action, whereby a transition from a state to another is executed according to elements of the movement vector until a gesture is identified by reaching the corresponding state, preemptively executing an action corresponding to an assumed gesture as said gesture is being detected, wherein the gesture being detected is assumed to be an assumed gesture if the state of the state diagram currently being visited is on a transition path to said gesture to be detected, determining, a distance of a movement (dist), determining if the distance of the movement is above a first distance threshold value (VI), and if so, comparing an angle of the movement with a library angle (β).
A method for identifying gestures from a tracked object in a video stream using a processor. It involves determining a movement vector (direction and magnitude) of the object and comparing it to a gesture library based on relative angles of partial movements. A state diagram is traversed, where transitions represent movement elements and states represent actions. The method preemptively executes actions for an assumed gesture if the current state is on the path to detecting that gesture. If a movement's distance exceeds a threshold, its angle is compared against a library angle.
14. A non-transitory computer readable storage medium encoded with instructions that, when loaded and executed on a processor, causes the method according to claim 13 to be performed.
A non-transitory computer-readable storage medium stores instructions that, when executed by a processor, perform the method of gesture recognition as described in claim 13, which involves determining a movement vector, comparing it to a gesture library using relative angles, traversing a state diagram, preemptively executing actions, and comparing movement angles if a distance threshold is met.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
March 16, 2016
June 27, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.