Augmented Reality Visualization for Robotic Picking System

2. The system according to claim 1 wherein the AR device is a headset apparatus worn by the user, or a mobile device held by the user.

3. The system according to claim 1 further comprising a pulse coder device configured to track a position of each conveyor in the robotic picking system, and a camera or other sensor configured to detect a presence and a position and orientation of the real parts on each conveyor.

4. The system according to claim 1 wherein providing preliminary boundaries for each of the one or more robots includes downloading the preliminary boundaries from a separate robotic picking system set-up application or establishing the preliminary boundaries at default locations within the robotic picking system visualization AR software application.

5. The system according to claim 1 wherein establishing the position and orientation of the AR device relative to the work cell coordinate frame includes analyzing images of one or more visual marker having a known design and placed at a known location in the work cell coordinate frame.

6. The system according to claim 1 wherein the position tracking sensors include one or more gyroscopes and one or more accelerometers, where the position tracking sensors provide signals to the processor in the AR device enabling continuous computation of changes in position and orientation of the AR device.

7. The system according to claim 1 wherein providing an AR display includes displaying the boundaries and the other virtual items in a proper size, shape and position relative to the camera images of the real items as the AR device is moved around the work cell.

8. The system according to claim 1 wherein the other virtual items include the virtual parts, part identifier blocks on the real parts, part information flags on real parts and virtual parts, coordinate frame triads, and camera position and field of view, and the display of the other virtual items is continuously updated based on the part allocation and part pick and place information received from the robot controllers.

9. The system according to claim 1 wherein instructing the one or more robot controllers to simulate virtual parts includes selecting, by the user, a virtual part feed rate and statistical distributions for positions and orientations of the virtual parts fed to the robotic picking system on an inbound conveyor.

10. The system according to claim 1 wherein receiving boundary location modifications from the user includes selecting a boundary to modify, moving the boundary a user-designated distance in an upstream or downstream direction, and accepting the boundary location modification, where selecting a boundary, moving the boundary and accepting the modification are performed by the user via AR-based gestures, menu commands or both.

11. The system according to claim 1 wherein communicating the boundary location modifications to the one or more robot controllers includes using modified locations of the boundaries by the robot controllers in computing robot pick and place operations.

12. The system according to claim 1 wherein displaying picking system performance parameters in the AR display includes displaying performance parameters and diagnostic information about any one or more robot or conveyor which is visible in the AR display at a given time.

13. The system according to claim 1 wherein the performance parameters include conveyor speed, real part rate, virtual part rate, total part rate, number of missed parts, part pick rate per robot, and part pick percentage per robot, and any combination of the performance parameters is displayable as selected by the user.

14. The system according to claim 1 wherein the boundaries include upstream, discard and downstream boundaries for each robot for each conveyor.

15. An apparatus for configuration of a visual tracking robotic picking system, said apparatus comprising an augmented reality (AR) device operated and positioned by a user and in communication with one or more robot controller, said AR device having one or more camera, position tracking sensors and a display, said AR device further including a processor and memory configured to run a robotic picking system visualization software application, where the software application is configured to interoperate with the one or more robot controller in the robotic picking system, instruct the one or more robot controllers to simulate a conveyor-based flow of virtual parts for processing by the robotic picking system, display boundaries and other virtual items in an AR display on the AR device, allow boundary modifications by the user using the AR device and communicate the boundary modifications to the one or more robot controller, and display real-time part movements and picking system performance parameters in the AR display.

17. The method according to claim 16 wherein the other virtual items include the virtual parts, part identifier blocks on the real parts, part information flags on real parts and virtual parts, and coordinate frame triads, and the display of the other virtual items is continuously updated based on the part allocation and part pick and place information received from the robot controllers.

18. The method according to claim 16 wherein the virtual parts simulated by the one or more robot controllers are based on parameters provided by the AR software application including user-defined values for a virtual part feed rate and statistical distributions for positions and orientations of the virtual parts on the inbound conveyor.

19. The method according to claim 16 wherein modifying the boundary positions by a user includes selecting a boundary to modify by the user, moving the boundary a user-designated distance in an upstream or downstream direction, and accepting the boundary location modification by the user, where selecting a boundary, moving the boundary and accepting the modification are performed by the user via AR-based gestures, menu commands or both.

20. The method according to claim 16 wherein communicating the boundary positions to the one or more robot controllers includes using modified locations of the boundaries by the robot controllers in computing robot pick and place operations.

21. The method according to claim 16 wherein displaying picking system performance parameters in the AR display includes displaying performance parameters and diagnostic information about any one or more robot or conveyor which is visible in the AR display at a given time.

22. The method according to claim 16 wherein the performance parameters include inbound conveyor speed, real part rate, virtual part rate, total part rate, number of missed parts, part pick rate per robot, and part pick percentage per robot, and any combination of the performance parameters is displayable in the AR display as selected by the user.

23. The method according to claim 16 wherein the robotic picking system includes the inbound conveyor carrying parts to be picked by the one or more robots and either an outbound conveyor or a stationary package for receiving parts placed by the one or more robots, and the boundaries include upstream, discard and downstream boundaries for each robot for each conveyor.