Dish Perception, Planning and Control for Robots

2. The system of claim 1, wherein said processor estimates the type, position, and orientation of all dishes in said cabinet.

3. The system of claim 1, wherein said dish is an article that makes contact with a food or a drink while preparing, serving, consuming, or storing of the food or the drink.

4. The system of claim 1, wherein at least one camera is mounted on the ceiling of said cabinet.

5. The system of claim 1, wherein at least one camera is mounted outside said cabinet such that said dish is visible in said image through a substantially transparent window or an opening of said cabinet.

6. The system of claim 1, wherein said robotic arm positions and orients an end effector based on said type, position, and orientation of said dish.

7. The system of claim 6, wherein said end effector comprises at least two movable fingers or a vacuum suction cup to grip said dish.

8. The system of claim 1, wherein at least one camera captures an image at a different wavelength than at least one other camera.

9. The system of claim 1, wherein said diffuse light source is either configured as a ring that surrounds a camera or configured as a diffuse illumination panel.

10. The system of claim 1, wherein said structured light source emits infrared light.

11. The system of claim 1, wherein said camera for capturing a structured image is designed to capture infrared images.

12. The system of claim 1, wherein said camera for capturing a diffuse image captures images at periodic intervals of time.

14. The method of claim 13, wherein said dish detection model classifies image data into one or more of the following classes: dish, food waste, unknown object, empty.

15. The method of claim 13, wherein said dish identification model classifies image data into one or more of the following classes: plate, bowl, cup, mug, spoon, fork, knife, chopstick.

16. The method of claim 13, wherein said dish localization model estimates the three dimensional positions of one or more points on said dish.