Patentable/Patents/US-10977717
US-10977717

Hand actions monitoring device

PublishedApril 13, 2021
Assigneenot available in USPTO data we have
Inventorsnot available in USPTO data we have
Technical Abstract

A system, a wearable device and method for monitoring hand actions. The method comprises: obtaining a visual input from a sensor located on the wearable device worn by a subject. The sensor is configured to capture at least an interior portion of a hand of the subject. The method further comprises identifying, based on the visual input, an object and an action performed by the hand of the subject with respect to an object. The method further comprises performing a responsive action based on at least one of the action and the object. The wearable device is configured to be utilized for self-service shopping, manual fulfillment of a shopping order of a customer, mapping locations of items, detecting violation of safety rules, monitoring health-care systems, managing labs or manufacturing plant, or for other applications related to hand actions monitoring.

Patent Claims
21 claims

Legal claims defining the scope of protection, as filed with the USPTO.

1

1. A method comprising: obtaining a visual input from a sensor located on a wearable device, wherein the wearable device is worn by a subject and utilized during a collection session, wherein the collection session is defined as beginning upon an identification of a first activity, wherein the collection session is defined as ending upon an identification of a second activity, wherein the sensor is configured to capture at least an interior portion of a hand of the subject; identifying, based on the visual input, a picking-up action performed by the subject using the hand of the subject in which the subject picks up an item, wherein the picking-up action is a non-gesture action, that is not intended to convey an instruction to the wearable device, wherein said identifying comprises identifying the item; performing a responsive action based on at least one of the picking-up action and the item; and in response to detecting a tampering event potentially interfering with monitoring of the hand of the subject using the wearable device, performing an anti-tampering action, wherein the anti-tampering action comprises issuing an alert, wherein said detecting is performed only during the collection session, whereby the subject is unable to perform the tampering event during the collection session without resulting in the anti-tampering action and the subject is able to perform the tampering event after the collection session ends without resulting in the anti-tampering action.

2

2. The method of claim 1 further comprises: in response to detecting the first activity being performed, continuously monitoring the subject, wherein said continuously monitoring the subject comprises said obtaining the visual input, wherein said continuously monitoring is terminated upon detecting a check out the second activity being performed.

3

3. The method of claim 1 , wherein the interior portion of the hand comprises a distal portion of a palm, whereby the visual input captures at least a portion of the item when the item is being held by the hand.

4

4. The method of claim 1 , wherein the wearable device is configured to be worn on a wrist of the subject, whereby positioning the sensor to face a palm of the hand.

5

5. The method of claim 1 , wherein a view of the sensor is blocked, at least in part, by the hand.

6

6. The method of claim 1 , wherein at least 5 % of the visual input comprises a view of the interior portion of the hand.

7

7. The method of claim 1 , wherein the collection session is a self-service shopping session of the subject in a store, whereby enabling the subject to shop in the store without manually scanning purchased items before leaving the store; wherein the item is an item for sale in the store; wherein the responsive action comprises: updating a virtual cart of the subject to include the item for sale.

8

8. The method of claim 7 further comprises: obtaining a second visual input from the sensor; determining, based on the second visual input, a second non-gesture action relating to the item for sale; determining based on the second non-gesture action, that the subject has decided not to purchase the item for sale; and in response to said determining, based on the second visual input and the second non-gesture action, updating the virtual cart of the subject to exclude the item for sale.

9

9. The method of claim 7 , wherein said identifying comprises identifying the item for sale based on the visual input and based on a positioning reading of the wearable device; wherein the method further comprises: determining, based on the positioning reading of the wearable device, a subset of a catalog of items; and performing product recognition to identify the item for sale, wherein the product recognition is performed with respect to the subset of the catalog of items.

10

10. The method of claim 7 further comprises: obtaining a second visual input from the sensor, wherein the second visual input captures a content of a physical tote; determining, based on the virtual cart and on the second visual input, a discrepancy between content of the virtual cart and content of the physical tote; and in response to said determining, performing a second responsive action, wherein the second responsive action comprises at least one of: marking the virtual cart as invalidated and updating the virtual cart based on the second visual input.

11

11. The method of claim 1 , wherein the tampering event comprises at least one of: removal of the wearable device and blocking view of the sensor.

12

12. The method of claim 1 , wherein said detecting the tampering event comprises obtaining readings from a motion sensor of the wearable device to determine a behavioral pattern of the subject that is indicative of an attempt to prevent monitoring of the hand of the subject.

13

13. The method of claim 1 , wherein the wearable device is utilized for manual fulfillment of a shopping order of a customer, wherein the shopping order comprises a list of items; wherein the subject is a picker picking items to fulfill the shopping order of the customer; wherein the picking-up action comprises picking up the item and placing the item in a tote associated with the shopping order of the customer; and wherein the responsive action comprises: identifying a corresponding item to the picked up item in the list of items; and marking the corresponding item as fulfilled.

14

14. The method of claim 1 , wherein the picking-up action comprises placing the item; wherein the responsive action comprises: determining a geospatial location of the item after being placed, and updating a mapping of items in a facility to indicate the item is located at the geospatial location.

15

15. The method of claim 1 , wherein the wearable device comprises a first wearable component and a second wearable component, wherein the first wearable component is configured to be worn on the hand of the subject, wherein the second wearable component is configured to be worn on a second hand of the subject, wherein the sensor comprises a first sensing component located on the first wearable component and a second first sensing component located on the second wearable component, wherein the visual input comprises input from the first sensing component and from the second sensing component.

16

16. The method of claim 1 , wherein the responsive action comprises: comparing the picking-up action with a rule; and in response to a violation of the rule, issuing a safety alert.

17

17. A wearable device, wherein the wearable device is configured to be worn on a hand of a user and utilized during a collection session, wherein the collection session is defined as beginning upon an identification of a first activity, wherein the collection session is defined as ending upon an identification of a second activity, wherein the wearable device comprises: a visual sensor configured to continuously capture an interior portion of the hand, wherein the wearable device is configured to provide images captured by the visual sensor to be utilized to identify a a picking-up action performed by the hand of the user and to identify an object upon which the picking-up action is performed, wherein the picking-up action is a non-gesture action, that is not intended to convey an instruction to the wearable device, a communication unit configured to connect the wearable device to a controller external to the wearable device; and a tampering detection module, wherein said tampering detection module is configured to monitor and detect a tampering event during the collection session and cause performance of an anti-tampering action, wherein the anti-tampering action comprises issuance of an alert in response to a detection of the tampering event, wherein the tampering event is an event potentially interfering with monitoring of the hand of the user, wherein said tampering detection module is configured to avoid monitoring user activity outside the collection session, whereby the user is unable to perform the tampering event during the collection session without resulting in the anti-tampering action and the user is able to perform the tampering event after the collection session ends without resulting in the anti-tampering action.

18

18. The wearable device of claim 17 , wherein activation and de-activation of the wearable device is performed automatically by the controller.

19

19. The wearable device of claim 17 , wherein the wearable device is configured to be utilized for self-service shopping, wherein the wearable device is configured to be utilized to identify items grabbed by the hand and moved to or from a physical shopping tote of the user, wherein the items are identifiable based on input of the visual sensor.

20

20. The wearable device of claim 17 , wherein the wearable device is configured to be associated with a virtual cart upon initiating a self-shopping session, wherein the virtual cart indicates a list of items shopped by the user, wherein the virtual cart is automatically updated based on items moved to and from a shopping cart.

21

21. The wearable device of claim 17 , wherein the wearable device is configured to be utilized for manual fulfillment of a shopping order of a customer, wherein the shopping order comprises a list of items; wherein the user is a picker tasked with picking items to fulfill the shopping order of the customer; wherein the picking-up action comprises picking up the object and placing the object in a tote associated with the shopping order of the customer.

Classification Codes (CPC)

Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.

Patent Metadata

Filing Date

May 22, 2020

Publication Date

April 13, 2021

Want to explore more patents?

Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.

Citation & reuse

Analysis on this page is generated by Patentable — an AI-powered patent intelligence platform. AI-generated summaries, explanations, and analysis may be reused with attribution and a visible link back to the canonical URL below. Patent abstracts and claims are USPTO public domain.

Cite as: Patentable. “Hand actions monitoring device” (US-10977717). https://patentable.app/patents/US-10977717

© 2026 Patentable. All rights reserved.

Patentable is a research and drafting-assistant tool, not a law firm, and does not provide legal advice. Documents we generate are drafts for review by a licensed patent attorney.