An imaging system and method uses grouping and elimination to label images of unknown items. The items may be stacked together with known or unknown items. The items may be packages, such as packages of beverage containers. A machine learning model may be used to infer skus of the packages. The machine learning model is trained on known skus but is not trained on unknown skus. Multiple images of the same unknown sku are grouped using the machine learning model. Elimination based upon lists of expected skus is used to label each group of unknown skus.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method for identifying a plurality of items in a plurality of stacks using a computing system including at least one machine learning model, the method including: a) receiving a plurality of images including at least one image of each the plurality of items in each of the plurality of stacks; b) generating an output based upon each of the plurality of images using the at least one machine learning model, wherein each of the plurality of items is not recognized based upon its associated output; c) forming a plurality of groups of the outputs associated with the plurality of images, by grouping similar ones of the outputs together; d) comparing the plurality of groups to a plurality of lists of expected items, each of the plurality of lists of expected items associated with one of the plurality of stacks; and e) based upon step d), assigning one of the expected items to each of the plurality of groups.
2. The method of claim 1 further including: f) training at least one new machine learning model using at least some of the plurality of images and the assigned ones of the expected items.
3. The method of claim 1 further including a step of imaging the plurality of stacks of the plurality of items prior to step a).
4. The method of claim 1 wherein step e) includes using elimination based upon the plurality of images and based upon the plurality of lists of expected items.
5. The method of claim 4 wherein elimination is performed based upon quantities of expected items indicated in the plurality of lists.
6. The method of claim 1 wherein the plurality of stacks are stacked on a plurality of pallets.
7. The method of claim 6 wherein the plurality of items are a plurality of packages.
8. The method of claim 7 wherein the plurality of packages contain beverage containers and wherein the at least one machine learning model is trained on images of a plurality of known packages containing beverage containers.
9. The method of claim 1 wherein steps c) to e) are performed for each of a plurality of different subsets of the plurality of stacks.
10. A method for identifying a plurality of items in a plurality of stacks using a computing system including at least one machine learning model, the method including: a) receiving a plurality of images including at least one image of each the plurality of items in each of the plurality of stacks; b) generating an output based upon each of the plurality of images using the at least one machine learning model; c) forming a plurality of groups of the outputs associated with the plurality of images, by grouping similar ones of the outputs together; d) comparing the plurality of groups to a plurality of lists of expected items, each of the plurality of lists of expected items associated with one of the plurality of stacks; e) based upon step d), assigning one of the expected items to each of the plurality of groups, wherein steps c) to e) are performed for each of a plurality of different subsets of the plurality of stacks; and f) assigning a confidence value to each of the assignments of one of the expected items based upon a number of different subsets upon which that one of the expected items was assigned.
11. A method for identifying a plurality of items in a plurality of stacks using a computing system including at least one machine learning model, the method including: a) receiving a plurality of images including at least one image of each the plurality of items in each of the plurality of stacks; b) generating an output based upon each of the plurality of images using the at least one machine learning model; c) forming a plurality of groups of the outputs associated with the plurality of images, by grouping similar ones of the outputs together; d) comparing the plurality of groups to a plurality of lists of expected items, each of the plurality of lists of expected items associated with one of the plurality of stacks; e) based upon step d), assigning one of the expected items to each of the plurality of groups; f) identifying a recognized subset of the plurality of items based upon step b); and g) performing steps c) to e) for the plurality of items not identified in step f).
12. A computing system for evaluating a stack of a plurality of items comprising: at least one processor; and at least one non-transitory computer-readable medium storing: at least one machine learning model; and instructions that, when executed by the at least one processor, cause the computing system to perform operations comprising: a) receiving a plurality of images including at least one image of each of the plurality of items in each of the plurality of stacks; b) generating an output based upon each of the plurality of images using the at least one machine learning model wherein each of the plurality of items is not recognized based upon its associated output; c) forming a plurality of groups of the outputs associated with the plurality of images, by grouping similar ones of the outputs together; d) comparing the plurality of groups to a plurality of lists of expected items, each of the plurality of lists of expected items associated with one of the plurality of stacks; and e) based upon step d), assigning one of the expected items to each of the plurality of groups.
13. The system of claim 12 wherein the operations further include: g) training at least one new machine learning model using at least one of the plurality of images to which one of the expected items was assigned in step e).
14. The system of claim 12 wherein operation e) includes using elimination based upon the images of the plurality of stacks and based upon the plurality of lists of expected items.
15. The system of claim 12 wherein the plurality of stacks are stacked on a plurality of pallets.
16. The system of claim 15 wherein the plurality of items are a plurality of packages.
17. The system of claim 16 wherein the plurality of packages contain beverage containers and wherein the at least one machine learning model is trained on images of a plurality of known packages containing beverage containers.
18. The system of claim 16 further including at least one camera for taking the plurality of images of the stacks of the plurality of items.
19. A method for identifying a plurality of packages stacked on at least one pallet using a computing system, the method including: a) receiving a plurality of images including at least one image of each the plurality of packages on the at least one pallet; b) generating an output based upon each of the plurality of images; c) grouping similar ones of the outputs into each of a plurality of groups; d) comparing the plurality of groups to at least one list of SKUs expected to be on the at least one pallet, each of the at least one list of SKUs including an expected quantity of each of a plurality of SKUs; and e) based upon step d), assigning a different one of the SKUs to each of the plurality of groups.
20. The method of claim 19 wherein step f) includes assigning each of the SKUs to a different one of the plurality of groups.
21. The method of claim 19 wherein step d) is performed based upon an analysis by the computing system using at least one machine learning model trained with images of packages.
22. The method of claim 21 wherein the at least one machine learning model is trained with images of packages of beverage containers.
23. The method of claim 22 wherein step f) includes assigning each of the SKUs to a different one of the plurality of groups.
24. The method of claim 23 further including training a new machine learning model based upon the plurality of images.
25. The method of claim 19 wherein the at least one pallet is a plurality of pallets.
26. The method of claim 25 wherein step d) is performed based upon an analysis by the computing system using at least one machine learning model trained with images of packages of beverage containers and wherein the plurality of packages are packages of beverage containers.
27. A computing system for identifying SKUs associated with each of a plurality of packages of beverage containers on at least one pallet comprising: at least one processor; and at least one non-transitory computer-readable medium storing: at least one machine learning model; and instructions that, when executed by the at least one processor, cause the computing system to perform operations comprising: a) receiving a plurality of images of the plurality of packages including at least one image of each of the plurality of packages on each of the at least one pallet; b) generating an output based upon each of the plurality of images using the at least one machine learning model, wherein the at least one machine learning model has not been trained on the SKUs associated with the plurality of packages; c) forming a plurality of groups of the outputs associated with the plurality of images, by grouping similar ones of the outputs together; d) comparing the plurality of groups to at least one list of SKUs expected to be on each at least one pallet; and e) based upon step d), assigning one of the SKUs from the at least one list of SKUs to each of the plurality of groups.
28. The computing system of claim 27 wherein the at least one machine learning model is trained on images of packages of beverage containers.
29. The computing system of claim 28 wherein the operations include using elimination based upon the plurality of images and based upon the at least one list of SKUs.
30. The computing system of claim 29 further including at least one camera for taking the plurality of images.
31. A method for identifying SKUs of a plurality of packages using a computing system including at least one machine learning model, the method including: a) receiving a plurality of images including at least one image of each the plurality of packages; b) the computing system generating an output based upon each of the plurality of images using the at least one machine learning model, wherein each of the plurality of packages is not recognized based upon its associated output; c) the computing system forming a plurality of groups of the outputs associated with the plurality of images, by grouping similar ones of the outputs together; and d) the computing system assigning a SKU to each of the plurality of groups based upon the plurality of groups and based upon a plurality of lists of expected SKUs.
32. The method of claim 31 wherein step d) further includes comparing the plurality of groups to the plurality of lists of expected SKUs, wherein each of the plurality of lists of expected SKUs includes an expected quantity of each of the expected SKUs.
33. The method of claim 31 further including: e) before step a), presenting the plurality of lists of expected SKUs so that the plurality of packages can be retrieved.
34. The method of claim 31 further including: e) identifying SKUS of a recognized subset of the plurality of packages based upon step b); and f) performing steps c) to d) for the plurality of packages not identified in step e).
35. The method of claim 31 further including: e) training at least one new machine learning model using at least some of the plurality of images and the assigned ones of the expected SKUs.
36. The method of claim 31 wherein step d) includes using elimination based upon the plurality of images and based upon quantities of expected SKUs in the plurality of lists of expected SKUs.
37. The method of claim 31 wherein the plurality of packages contain beverage containers and wherein the at least one machine learning model is trained on images of a plurality of known packages containing beverage containers.
38. A computing system for identifying SKUs of each of a plurality of packages comprising: at least one processor; and at least one non-transitory computer-readable medium storing: at least one machine learning model; and instructions that, when executed by the at least one processor, cause the computing system to perform operations including: a) receiving a plurality of images of the plurality of packages including at least one image of each of the plurality of packages; b) generating an output based upon each of the plurality of images using the at least one machine learning model; c) forming a plurality of groups of the outputs associated with the plurality of images by grouping similar ones of the outputs together; d) assigning one of a plurality of SKUs to each of the plurality of groups based upon the plurality of groups and based upon a plurality of lists of quantities of expected SKUs.
39. The computing system of claim 38 wherein operation d) includes assigning one of the plurality of SKUs to each of the plurality of groups by comparing the plurality of groups to the plurality of lists of expected SKUs.
40. The computing system of claim 38 wherein the operations further include: e) presenting the plurality of lists of expected SKUs based upon a plurality of orders from stores so that the plurality of packages can be retrieved.
41. The computing system of claim 38 wherein the operations include using elimination based upon the plurality of groups and based upon the plurality of lists of expected SKUs.
42. The computing system of claim 38 wherein the at least one machine learning model is not trained on the plurality of SKUs, such that the SKUs of the plurality of packages cannot be recognized from the outputs of operation b).
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 29, 2023
January 21, 2025
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.