Robot control in working space

This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2022-035461, filed on Mar. 8, 2022. The entire contents of which are incorporated herein by reference.

One aspect of the present disclosure relates to a robot system, a planning system, a robot control method, and a planning program.

Japanese Patent No. 6860843 describes a robot system including a measurement posture storage unit that stores position information on a robot controller in a plurality of measurement robot postures provided in a robot work environment, a measurement position acquisition unit that acquires a measurement position for each of the plurality of measurement robot postures based on a detection result of a sensor, and a correction unit that corrects an operation position of a robot based on the measurement position.

A robot system according to an aspect of the present disclosure includes: a storage device configured to store a predefined work requirement that indicates a processing target region of a workpiece to be processed by a robot; and circuitry configured to: recognize an environment of a working space in which the workpiece is placed, as a work environment, wherein the work environment includes a position and a posture of the workpiece placed in the working space; identify the processing target region of the workpiece placed in the working space, based on the position and posture of the workpiece placed in the working space; generate, in real-time, a path of the robot to operate on the identified processing target region based on the work requirement and the identified processing target region; and control the robot based on the generated path.

A robot control method according to an aspect of the present disclosure is executable by a robot system including at least one processor. The method includes: storing a predefined work requirement that indicates a processing target region of a workpiece to be processed by a robot; recognizing an environment of a working space in which the workpiece is placed, as a work environment, wherein the work environment includes a position and a posture of the workpiece placed in the working space; identifying the processing target region of the workpiece placed in the working space, based on the position and posture of the workpiece placed in the working space; generating, in real-time, a path of the robot to operate on the identified processing target region based on the work requirement and the identified processing target region; and controlling the robot based on the generated path.

Anon-transitory computer-readable storage medium according to an aspect of the present disclosure stores processor-executable instructions to: retrieve a predefined work requirement that indicates a processing target region of a workpiece to be processed by a robot, from a storage device; recognize an environment of a working space in which the workpiece is placed, as a work environment, wherein the work environment includes a position and a posture of the workpiece placed in the working space; identify the processing target region of the workpiece placed in the working space, based on the position and posture of the workpiece placed in the working space; and generate, in real-time, a path of the robot to operate on the identified processing target region based on the work requirement and the identified processing target region.

In the following description, with reference to the drawings, the same reference numbers are assigned to the same components or to similar components having the same function, and overlapping description is omitted.

Overview of System

A robot system according to the present disclosure is a control system that automates various operations such as machining and assembly by causing a robot placed in an actual working space to perform a predetermined motion. The robot system generates a path of the robot based on a work requirement predefined with respect for a workpiece and a work environment that is an environment of a working space in which the workpiece is placed. The work requirement refers to information indicating a process to be performed on the workpiece. The work environment may be an actual situation of the working space. The path indicates a trajectory of the robot and constitutes at least part of a command to the robot. Accordingly, the robot system includes a planning system that is a computer system for determining a motion of the robot. The robot system generates an operation program including the path, as a command, and controls the robot based on the command. The robot processes at least one workpiece according to the control.

A series of processes for at least one workpiece is also referred to as a job. Processing for one workpiece is also referred to as a task, which is a minimum unit of processing constituting a job. The work requirement may indicate one or more tasks.

A path of the robot for executing the job can be classified into two types of paths. One is a path that is a trajectory of the robot in the task, and is referred to as a work path in the present disclosure. The other is a path connecting between a start point of a job and a first task, between two consecutive tasks, or between a last task and an end point of the job, and is referred to as an air-cut path in the present disclosure. The air-cut path connecting adjacent tasks connects the end point of a work path in a preceding task and the start point of a work path in a following task. The robot system generates individual work paths and individual air-cut paths based on the work requirement and the work environment, thereby generating the entire path of the robot for executing the job.

In some examples, the robot system sets the path of the robot based on the work requirement and the work environment, without using a teaching point with respect to the robot. With the robot system, once a user teaches a process (i.e., work requirement) to be performed on a workpiece, the robot may autonomously process the workpiece according to an actual situation (i.e., work environment). The teaching point refers to a reference point which is set to define a path, and is set manually. The teaching point is set in a robot coordinate system which is a coordinate system whose origin is set to the robot (for example, a base of the robot). The “teaching point with respect to the robot” refers to a teaching point set in the robot coordinate system in order to operating the robot.

In the related art, setting of a teaching point is also referred to as teaching. In addition, there is teaching playback as a technique of controlling a robot based on the teaching point. In these conventional techniques, a position of the robot in the robot coordinate system is required to be set. It is however difficult for a user to intuitively understand a correspondence relationship between the setting of the position and the task. In addition, in the related art, the teaching is required also to generate an air-cut path that is not directly related to a task. Furthermore, in a case where the work environment such as a relative position relationship between the robot and the workpiece changes, the user needs to recreate the air-cut path. For these reasons, the related art is not suitable for multiproduct production in which tasks or procedures are frequently changed.

As described above, in some examples, the robot system sets the path of the robot based on the work requirement and the work environment, without using the teaching point in the robot coordinate system, i.e., without requiring the teaching and the teaching playback. It may be enough for the user to instruct work on a workpiece, and the teaching is not required. Since the robot system automatically recognizes the work environment by using data obtained from a predetermined sensor or a model indicating the work environment, the robot may be operated flexibly according to the work environment. In addition, the job may be executed without positioning the workpiece or fixing the workpiece with a jig. Since the work requirement with respect to the workpiece is information that may be intuitively created by the user, the user may cause the robot to process the workpiece more intuitively without paying attention to the robot. This may reduce the load on the user. For these reasons, the robot system may promote the introduction of the robot into the multiproduct production.

In some examples, the robot system recognizes the work environment in real-time in response to detecting that the workpiece is placed in the working space. The robot system then generates, as a command, an operation program including a path in real-time, based on the work requirement for the workpiece and the work environment, and controls the robot in real-time based on the command. In this way, the robot system plans the operation of the robot online. Such online control also contributes to improving the autonomy of the robot.

Configuration of System

is a diagram showing a configuration of a robot systemaccording to some examples. The robot systemincludes a robotplaced in a working space, a motor control devicethat controls a motor of the robot, and a robot controllerthat outputs a control signal to the motor control device. The robot controllerand the motor control deviceare connected to each other via a communication network. The robot controllerconnects to a host controllerand an operation result databasevia another communication network. The communication network connecting the devices may be a wired network or a wireless network. The communication network may include at least one of the Internet and an intranet. Alternatively, the communication network may be simply implemented by one communication cable. Althoughshows one robotand one robot controller, the robot systemmay include more than one robotor more than one robot controller. One robot controllermay be connected to a plurality of the robot.

The robotis a device that receives power and performs a predetermined operation according to a purpose to perform useful work. In some examples, the robotincludes a plurality of joints and is configured to be able to perform various processes with an end effectorbeing held at the tip portion. The joint is an example of a driven object. A joint axis is set for each of the plurality of joints. Some components of the robot, such as arms, pivots, etc., rotate about the joint axes such that the robotis able to freely change the position and posture of the tip portion within a predetermined range. In some examples, the robotis a multi-axis serial link type vertical articulated robot. The robotmay be a six-axis vertical articulated robot, or a seven-axis vertical articulated robot in which one redundant axis is added to six axes. The robotmay be a movable robot, for example a robot supported by an automated guided vehicle (AGV). Alternatively, the robotmay be a stationary robot that is fixed in a predetermined place.

The end effectoris a device that acts on the workpiece, for example, a device that gives some physical change to the workpiece. The end effectormay be various devices such as a polishing tool, a welding gun, a press machine, a sealing nozzle, a screwing device, and the like.

The motor is a device that generates power for driving a driven object of the robotaccording to electric power supplied from the motor control device. The individual driven objects are operated by the individual motors such that the robotperforms a predetermined task on the workpiece with the end effector. The motor may be a rotary motor that rotates the driven object or a linear motor that moves the driven object along a straight line. The motor may be a synchronous motor or an induction motor. The motor may be a permanent magnet type synchronous motor such as a surface permanent magnet (SPM) motor or an interior permanent magnet (IPM) motor. The motor may be a synchronous motor without permanent magnets, such as a synchronous reluctance motor. The motor may be a DC motor or an AC motor.

The robotmay include at least one sensor that is a device that detects a response of the robotoperating by the power from the motor control device. The response refers to an output of the robot in response to a command that is an instruction for controlling the robot. For example, the response indicates information related to at least one of motion and a state of the robot. The response may indicate information on at least one of a motion and a state of the motor, for example, may indicate at least one of an axis velocity and a magnetic pole position of the motor. In a case where the motor is a rotation type, a rotation angle of the driven object by the motor corresponds to the “position”, and a rotation velocity of the driven object by the motor corresponds to the “velocity”. The response may indicate information on at least one of a motion and state of the driven object, for example, may indicate at least one of the position, velocity, and force of the driven object. The sensor transmits a response signal indicative of the response to the robot controller. The response may be a value obtained by the sensor or may be represented by a value calculated or processed by a predetermined calculation or algorithm. Examples of the sensor may include a rotary encoder that outputs a pulse signal having a frequency proportional to an operational speed of the motor. The rotary encoder may obtain both the position and velocity of the motor.

The motor control deviceis a device that causes the output of the motor to follow the control signal (i.e., command) from the robot controller. Based on the control signal from the robot controller, the motor control devicegenerates electric power to move the motor and supplies the electric power to the motor. The supplied electric power corresponds to a driving force command such as a torque command, a current command, or the like. The motor control devicemay be, for example, an inverter or a servo amplifier. The motor control devicemay be incorporated into the robot. The motor control devicemay include a sensor that detects a response of the motor control device.

The host controlleris a device that instructs the robot controllerto control the robot. The host controllermay be a device related to Computer Aided Engineering (CAM). In some examples, the host controllertransmits, to the robot controller, a request to cause the robotto process a workpiece placed in the working space.

The operation result databaseis a device that stores operation result data indicating that the robotor another robot has processed one or more workpieces in the past.shows the operation result databaseas an independent device, the operation result databasemay however be provided in the robot controller, the host controller, or another device. The operation result data may indicate one or more processes performed by the robotto be controlled, one or more processes performed by one or more robots other than the robot, or both of the processes. Alternatively, the operation result data may indicate one or more processes performed by a peripheral device (e.g., an end effector) of the robot. The past working space may be the working spacewhere the robotis placed or may be a different place from the working space.

The robot controlleris a device that generates the command for operating the robotand outputs the control signal based on the command. The robot controlleris an example of the robot control system. In some examples, the robot controllerincludes a requirement definition unit, a storage unit, an environment recognition unit, an order setting unit, a command generation unit, and a robot control unitas functional modules. The requirement definition unitis a functional module that defines the work requirement for the workpiece based on the operation result data. The storage unitis a functional module that stores information to generate the command, for example the defined work requirement. The environment recognition unitis a functional module that recognizes an environment of the working spacein which one or more workpieces are placed, as the work environment. The order setting unitis a functional module that sets the processing order of the one or more workpieces. The command generation unitis a functional module that generates a path of the robotfor causing the robotlocated in the working spaceto process the one or more workpieces in the processing order, based on the work requirement and the work environment, and generates a command including the path. The robot control unitis a functional module that controls the robotbased on the command.

is a diagram showing an example hardware configuration of the robot controller. The robot controllerincludes circuitry. In some examples, the circuitrycomprises at least one processor, a memory, a storage, and at least one communication port. The storageis a non-volatile storage medium and stores a program for realizing each functional module described above. The storagemay be configured by at least one hard disk or nonvolatile memory, for example. The memorytemporarily stores a program loaded from the storage, a calculation result by the processor, and the like. The memorymay comprise, for example, at least one random access memory. The processorconfigures each functional module by executing the program in cooperation with the memory. The communication portperforms data transmission with other devices in accordance with commands from the processor. For example, the robot controllerhas one communication portthat is connected to a communication network N and transmits and receives signals to and from the host controllerand the operation result database, and another communication portthat transmits and receives signals to and from the motor control device.

Each functional module of the robot controlleris implemented by causing the processoror the memoryto read a robot control program and causing the processorto execute the program. The robot control program includes code for implementing each functional module of the robot controller. The processoroperates the communication portin accordance with the robot control program and reads and writes data in the memoryor the storage.

The robot control program may be provided after being fixedly recorded in a non-transitory recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, the robot control program may be provided via a communication network as a data signal superimposed on a carrier wave.

In some examples, the robot control program includes a planning program that generates the path for the robotbased on the work requirement and the work environment. The planning program may be provided independently.

Robot Control Method

As some examples of the robot control method according to the present disclosure, some examples of processing procedure executed by the robot controllerwill be described with reference to.is a flowchart showing an example process for defining the work requirement for the workpiece, as a process flow S.is a diagram showing an example work requirement for the workpiece.is a flowchart showing an example process for generating the command to the robot, as a process flow S.is a diagram showing an example path of the robot in the working space. The robot controllerexecutes the process flow Swith respect to the definition of work requirement, and executes the process flow Swith respect to the generation of commands.

(Definition of Work Requirement)

In some examples, the work requirement is defined by a user of the robot system, and is stored in advance in the storage unitby a predetermined operation by the user. As other examples, the work requirement may be defined by the requirement definition unit. For example, the requirement definition unitmay update the work requirement defined by the user and stored in the storage unit. Some examples of the definition of work requirement by the requirement definition unitwill be described with reference to. The process flow Sis executed after the robotor another robot processes one or more workpieces in the past working space and the operation result data indicating the processing is stored in the operation result database. The operation result data may indicate a past process that included a teaching path for operating on one or more workpieces. In some examples, the process flow Sis executed for each of a plurality of types of workpiece.

In step S, the requirement definition unitacquires the operation result data corresponding to the workpiece. In some examples, the requirement definition unitaccesses the operation result databaseto retrieve the operation result data corresponding to a certain type of a workpiece WP. In a case where n workpieces WP have been processed in the past, the operation result data includes records indicating n times of processing for the workpiece WP. In this case, the requirement definition unitretrieves at least part of the records from the operation result database.

In some examples, the operation result data includes a plurality of teaching points with respect to the robotor another robot, the plurality of teaching points representing a process on a workpiece by the robot, and a position of the workpiece in a past working space where the process was performed. The “position of workpiece” is represented by a coordinate value in the robot coordinate system. For example, the position may be obtained by converting a coordinate value in a sensor coordinate system into a coordinate value in the robot coordinate system. The sensor coordinate system is a coordinate system whose origin is set to the sensor, and may be, for example, a camera coordinate system whose origin is set to a lens of a camera. The teaching point and the position of the workpiece indicated by the operation result data may be values obtained by actual processing in an actual working space or values obtained by past simulation. The simulation refers to a process of expressing the behavior of a certain system in a simulated manner on a computer.

In step S, the requirement definition unitdefines the work requirement for the workpiece based on the operation result data. In a case where a plurality of records of the operation result data are used, the requirement definition unitmay define the work requirement by a calculation including statistical processing such as regression.

In some examples, the requirement definition unitidentifies a region on the workpiece to which the processing by the robot is applied, as a processing target region, based on the teaching point indicated by the operation result data and the position of the workpiece. For example, the processing target region is a region affected by the end effector. The processing target region is a region to be processed in the workpiece, and is an example of the work requirement. Thus, the requirement definition unitdefines the processing target region as at least part of the work requirement. Each processing target region may be point-like, linear, or two-dimensional shape. The requirement definition unitidentifies one or more processing target regions for one workpiece. In some examples, the requirement definition unitconverts coordinate values indicating the position and shape of each processing target region from the robot coordinate system to a workpiece coordinate system, and identifies the processing target region by coordinate values in the workpiece coordinate system. The workpiece coordinate system refers to a coordinate system whose origin is set to workpiece.

In some examples, the requirement definition unitmay identify the processing target region further using a workpiece model, which is a model that virtually represents the workpiece. The workpiece model is a three-dimensional model indicating a structure of the workpiece and is generated by, for example, computer-aided design (CAD). Since the workpiece model is expressed using the workpiece coordinate system, the requirement definition unitmay associate the processing target region with the region model. With such association, the processing target region includes or indicates a relative position with respect to the region model. In some examples, the “relative position with respect to the workpiece model” is a position set on the workpiece model in the workpiece coordinate system.

The requirement definition unitmay identify an approach point at which the robot approaches the workpiece, as at least part of the processing target region, and define the work requirement including the approach point. The approach point refers to a location where a robot attempting to process a workpiece first contacts the workpiece.

The requirement definition unitmay define an operational speed of the robot as at least part of the work requirement, based on the teaching point indicated by the operation result data and the position of the workpiece. For example, the requirement definition unitmay estimate the operational speed of the robot in a past process based on the history of a correspondence relationship between the position of the workpiece and time, and define the operational speed as at least part of the work requirement. That is, the predefined work requirement is based, at least in part, on the operational speed of the robot in performing the past process.

The requirement definition unitmay define a force applied to the workpiece by the robot, as at least part of the work requirement, based on the teaching point indicated by the operation result data and the position of the workpiece. Examples of the force include a pressing force generated when the robot presses the workpiece. In some examples, the requirement definition unitestimates a force applied to the workpiece by the robot in a past process based on a history of correspondence relationship between the position of the workpiece and time, and specification data indicating specifications of the robot or end effector, and defines the force as at least part of the work requirement. That is, the predefined work requirement is based, at least in part, on the force applied by the robot to the one or more workpieces in performing the past process.

In step S, the requirement definition unitgenerates work requirement data indicating the defined work requirement and stores the work requirement data in the storage unit. As a result, the work requirement are updated or newly registered. In some examples, the requirement definition unitgenerates and stores the work requirement data in which the work requirement and the workpiece model are associated with each other. That is, the work requirement, the workpiece model, and the processing target region are predefined information. Both the workpiece model and the processing target region indicated by the work requirement data are defined in the workpiece coordinate system. Therefore, the work requirement, the workpiece model, and the processing target region are different techniques from the teaching point set in the robot coordinate system. In addition, the process flow Sis different from the teaching, which is conventional art.

An example of the defined work requirement will be described with reference to. This example shows a processing target regiondefined on a workpiece model. For example, the processing target regionindicates a region to be sealed. The processing target regionincludes an approach pointindicated by an X mark in.

The work requirement defined by the requirement definition unitare obtained from an operation of the robot that processed the workpiece based on the teaching point manually set in the past. The work requirement may therefore be information reflecting experience and know-how about robot control for the workpiece. In some examples, since the processing target regionincluding the approach pointis automatically defined, the user of the robot systemis able to prepare the work requirement indicated by the workpiece modeland the processing target region, once the user prepares the workpiece model. It should be noted that, as described above, the work requirement shown inmay be defined by the user.

As described above, the process flow Smay be performed for each of the plurality of types of workpiece. In this case, the storage unitmay store the work requirement data for various workpieces that may be processed by the robot. That is, the storage unitmay store the work requirement for each of the plurality of workpieces.

Generation of Command

An example process for generating a command for operating the robotwill be described with reference to. In some examples, the robot systemexecutes the process flow Sin response to receiving a predetermined request from the host controller.

The process flow Sis executed on the assumption that information for generating a command is stored in the storage unit. As described above, the storage unitstores the work requirement data. The storage unitmay further store at least one of an obstacle model and tool information. The obstacle model refers to a model of an obstacle, and is represented by, for example, a three-dimensional model indicating the structure of an obstacle. The tool information refers to information related to the end effectorof the robotand indicates, for example, the structure and specifications of the end effector

In step S, the environment recognition unitrecognizes or detects an environment of the working spacewhere at least one workpiece is placed, as the work environment. In some examples, the environment recognition unitrecognizes a current work environment in real-time. In some examples, the environment recognition unitmay recognize the work environment based on sensor data obtained from a sensor monitoring the working space. For example, the sensor is an imaging device such as a camera, and the sensor data is image data obtained from the imaging device. The image data may be a still image, a video, or a frame of video.

In some examples, the environment recognition unitrecognizes or detects a position of each of one or more workpieces as at least part of the work environment. The environment recognition unitmay further recognize a posture of each of the one or more workpieces as at least part of the work environment. The position of the workpiece refers to a location where the workpiece is placed. The posture of the workpiece refers to at least one of an orientation and an inclination of the placed workpiece. In some examples, the environment recognition unitmay recognize the position and posture of the workpiece based on the workpiece model. For example, the environment recognition unitmay analyze the image data using one or more workpiece models stored in the storage unit, identify an object that matches a certain workpiece model as a workpiece, and recognize the position and posture of the identified workpiece.