Method and apparatus for implementing a safety configuration

The instant application claims priority to International Patent Application No. PCT/EP2022/051053, filed Jan. 18, 2022, which is incorporated herein in its entirety by reference.

The present disclosure generally relates to a method and to apparatus for implementing a safety configuration in a robot system.

A safety configuration can be regarded as a set of constraints that have to be observed during operation of a robot in order to ensure health and safety of persons in the vicinity of the robot, and/or to prevent damage from unwanted interaction of the robot with surrounding hardware. In order to make efficient use of a robot, its safety configuration has to be adapted to a job the robot is to accomplish. Since the safety configuration should at no time restrict the freedom of movement of the robot more than necessary, the constraints to be observed will vary during the execution of a program, and the safety configuration will have to reflect the fact. So, whenever a robot control program is developed in order to enable the robot to accomplish a new job, developing a matching safety configuration requires a considerable amount of highly qualified labour.

3 FIG. 302 a) in a step, reading an action instruction from a manipulator control program, the instruction defining at least one of a path to be followed by the end effector, a target position to be reached by the end effector, a target pose to be assumed by the end effector and an action to be carried out by a tool wielded by the end effector, 304 b) in a step, reading at least one constraint instruction from the manipulator control program, the constraint instruction defining a constraint to be observed while carrying out an action instruction, and 306 310 308 c) in a step, carrying out the action instruction read in step a) while respecting the constraint defined in the at least one constraint instruction, or in a step, transferring the manipulator into a safety mode if violation of the at least one constraint is detected in a stepwhile carrying out the action instruction. The present disclosure describes a method and apparatus that allow a safety configuration to be developed and implemented more efficiently. A flowchart for a method in accordance with the disclosure is shown in. According to one aspect of the invention, this object is achieved by a method for implementing a safety configuration for a robot comprising a manipulator and an end effector, comprising the steps of:

Defining constraints to be observed by way of program instructions, and combining these with action instructions for the manipulator in a same program facilitates synchronization of the constraints with movements of the manipulator to which they are meant to apply, thus relieving the programmer from the task of defining conditions for the entry into force or the revocation of a constraint, and reducing the risk of the manipulator being stopped because an inadequate constraint has been violated.

For the safety of operation of the manipulator, implementation of the safety configuration must not be affected by a malfunction in the control of actions of the manipulator. Therefore, for implementing the safety configuration a safety controller distinct and independent from an operation controller controlling execution of action instructions by the manipulator should be provided.

While above step b) of reading constraint instructions may be carried out by the safety controller and the operation controller alike, thereby enabling the operation controller to take account of constraints carrying out an action instruction and thus to proactively minimize the risk of a constraint being violated, step a) of reading an action instruction is preferably carried out by the operation controller alone; by thus keeping the safety controller ignorant of action instructions, the possibility of an action instruction causing a malfunction in both controllers is avoided.

a constraint instruction of an invoking function is valid during execution of a function invoked by it, and/or a constraint instruction of an invoked function becomes void when the invoked function terminates. Grouping instructions for executing a repetitive task into functions is a well-known means for efficient and concise program writing. Conventionally, a program may comprise a main function and functions invoked directly by the main function or indirectly via one or more intermediate functions. Safety constraints may vary from one such task to another. When constraint instructions are integrated into the functions, such changes of constraints may efficiently be taken account of when, by default,

In other words, an invoked function will by default not override constraints that apply to the function invoking it, whereas, when an invoked function imposes a constraint, this constraint will no longer apply when the invoked function is terminated and execution of the invoking function is resumed.

Of course, there is always a possibility of both an invoking function and a function invoked by it comprising constraint instructions which define different values for a same parameter. In order to avoid conflicts between a first constraint set first by the invoking function and a second constraint set later by the invoked function, it can be provided that the stricter one will prevail. I.e. if a first speed limit is imposed by the invoking function, and a second, lower one is imposed by the invoked function, the second one will hold while the invoked function is being executed, whereas if the second speed limit is higher than the first, the first speed limit will continue to apply during execution of the invoked function. Thus a programmer writing the code of the invoking function will not have to bother about the possibility of an invoked function overriding necessary constraints.

On the other hand, there can be cases where an excessively strict constraint imposed by an invoking function will prevent proper operation of the invoked function. Therefore, it may be appropriate to issue a notice or a warning when the invoked function sets a less strict constraint than the one invoking it, so that a programmer will be aware of the fact and can amend constraint instructions accordingly if the constraint imposed by the invoking function causes the invoked one to malfunction.

Evidently, issuing such a notice is particularly useful while the program is still under development.

Requiring the invoking function always to set a constraint appropriately for a function to be invoked, and resetting it when that function has terminated is error-prone and is likely to produce clumsy, lengthy code, the operational safety of which is difficult to ascertain. There should therefore be a possibility of the constraint of the invoked function to prevail over that of the invoking one.

It would be desirable to be able to specify in the program whether when a specific function is invoked, constraints of the invoking function or of the invoked one shall prevail. One possibility would be to specify, in the function call instruction of the invoking function, which one of the constraints shall be applicable, but this would require rather profound adaptations of a compiler or interpreter used for converting the action instructions of the program into machine-level instructions that can be directly input into the robot by the operation controller. The effect may be achieved more conveniently by providing a constraint instruction which invalidates a previously set constraint while a subsequent action instruction is being executed.

an allowed operating region in which the manipulator is allowed to move (or, equivalently, a region which the manipulator is not allowed to enter); an allowed maximum speed of the manipulator, an allowed maximum momentum of the manipulator, an allowed maximum force applied by the manipulator to an outside object; a required minimum distance of the manipulator from an object within reach of the manipulator. Constraint instructions may define at least one of

Carrying out the action instruction by simulation in step c) may be appropriate while the program is under development. In practical operation, the action instruction will be carried out by physically moving the manipulator.

Upon violation of a constraint, the robot system will enter “safety mode”. Safety modes may be different depending on whether the program is under development or the program controls practical operation of the manipulator.

Recording details of the violation after entry into safety mode is particularly useful while the program is under development, because the recorded details may help the developer to identify a reason for the violation and remove it.

stopping the manipulator; reducing an allowed maximum speed of the manipulator; reducing an allowed maximum momentum of the manipulator; reducing an allowed maximum force applied by the manipulator to an outside object; resuming normal operation of the manipulator, querying a supervisor's decision whether to stop or to resume operation. In safety mode, a choice between at least two of the following options can be made:

Preferably, the choice will be based on a setting defined before entering safety mode.

Stopping the manipulator is an option both while the program is under development and in practical operation. Reducing an allowed maximum of one of the parameters enumerated above may be useful in practical operation in order to reduce a risk to persons in the vicinity of the manipulator while avoiding a stop of the manipulator. Violations may differ in severity, so that in case of a slight violation, reducing the allowed maximum of a parameter may be sufficient, while in case of a serious violation, a stop may be necessary. Querying a supervisor's decision whether to stop or to resume operation may be useful while the program is under development, in order to give the programmer time to study the violation and its possible causes before the manipulator moves further. In practical operation, it may enable resumption of operation if the violation was due to an unforeseeable incident such as the intrusion of a person into a safety range of the manipulator, and after the reason for the violation has been removed.

d) reading from the manipulator control program an instruction specifying a function to be invoked in case of a violation, and e) executing the specified function after entry into safety mode. This facilitates the implementation of different reactions to a violation while the program is under development, on the one hand, and in case of practical operation, on the other. The method may further comprise steps of

a manipulator, an operation controller adapted to control the manipulator to carry out an action instruction from a manipulator control program, and a safety controller adapted to detect whether a constraint is respected or not when carrying out the action instruction, wherein the safety controller is adapted to extract the constraint from a constraint instruction of the manipulator control program. According to another aspect of the invention, the above object is achieved by a robot system comprising

Preferably the robot system is adapted to carry out the method described above.

Finally, the object is achieved by a manipulator control program for the robot system defined above, the program comprising action instructions executable by the operation controller to control the manipulator to carry out the action instruction, and constraint instructions from which at least one constraint to be observed while executing the action instruction is adapted to be extracted by the safety controller.

1 FIG. 1 2 3 4 3 is a schematic plan view of an industrial robot and its working environment. In the embodiment shown here, a manipulatorof the robot has a stationary base, at least one articulated armand an end effectorat the free end of the arm, but it should be kept in mind that the method which will be described subsequently is also applicable to other types of robots, such as a gantry type robot, mobile robots etc.

1 5 6 7 8 9 8 The motion of the manipulatoris controlled by an operation controller, typically a microcomputer, in order to carry out a predefined manufacturing task, for example combining each of a series of first workpieces, supplied by a conveyor, with a second workpiece, taken from a crate. In the crate, workpiecescan have various orientations, upright, horizontal or piled up under arbitrary angles.

10 5 10 5 10 Besides operation controller s, a safety controlleris provided. Each controller,may be implemented by a processor of its own, and in the description below it will be assumed that they are, it being understood that they can easily and even more economically be implemented as distinct software modules executed by a same processor, as far as certain data quarantine measures described below are taken in order to prevent a failure state from spreading from one controller to the other, in particular from operation controllerto safety controller.

5 10 1 11 1 1 5 10 1 Both controllers,are connected to internal sensors of manipulator, in order to receive from these data relating to angles of all jointsof the manipulator, to electrical power absorbed by motors of the manipulator, etc., that enable each controller,to calculate a current pose of the manipulatorand velocities of its various components.

10 12 1 12 1 1 10 12 10 12 1 1 FIG. The safety controlleris connected to a spatially resolving sensor, typically an electronic camera, which is designed and positioned to monitor the manipulatorand its vicinity. In, the sensoris represented as a single camera. In practice it may comprise several cameras viewing the manipulatorfrom different directions, so that no object in the vicinity can be hidden from all cameras at a time by the manipulator, and so that the safety controllercan determine coordinates of an object seen by more than one camera by triangulation. Data from sensorcan be used to enable the safety controllerto detect a discrepancy between a pose calculated from the data of the internal sensors of the manipulator and a pose seen in the data from sensor, and to switch into a safety mode if a discrepancy between these poses is significant; they can further be used to detect moving objects and persons, their positions and speeds and to enter the safety mode if it judges that there is a possibility of conflict between the object or person and the manipulator.

1 1 1 4 1 In the simplest case, the safety mode can involve a complete halt of the robot. Alternatively, in the safety mode, maximum speed of the manipulatorcan be set low enough to avoid injury to a person if it comes into contact with the manipulator while the person is standing still or moving at moderate speed. This can be done by specifying a speed limit which the manipulatormust not exceed in safety mode, or, taking into account the weight and inertial moment of the manipulatorand, possibly, of an object carried by its end effector, by limiting the speed so that an allowed maximum momentum of the manipulator is not exceeded. Further alternatively, motors of the manipulator may be controlled so that a force the manipulatormight apply to an external object touched by it is limited to an allowed maximum value low enough to prevent injury if what the manipulator touches is a person. Still further alternatively, a space in which the object is detected can be excluded from a region in which the manipulator is allowed to move. Any of the above measures can be combined; in particular, there can be safety modes at various levels in which ever stricter safety measures are adopted, the closer the manipulator is to the detected object or the higher the likeliness of conflict between the two is judged to be.

1 FIG. 13 13 12 13 14 1 9 1 1 1 10 15 14 1 15 14 15 1 1 1 Inthe space in which the robot is working is partially confined by walls or barriers. Part of these walls or barriersis in the detection range of sensor. One of the wallshas a passagethrough which a moving object or person might at any time enter the vicinity of the manipulator, e.g. in order to replace the cratewhen empty, and collide with the manipulator. While an interaction between the person and the manipulatormay be desired and should not be prevented, it should be made sure that when such an interaction occurs, the manipulatoris at rest or at least moving slowly enough to prevent any risk of injury. Therefore the safety controllerdefines a forbidden zonewhich extends from the passageinto the vicinity and which it will not allow the manipulatorto enter. The dimension of the forbidden zoneis determined by the condition that whenever an object enters the vicinity through passagewith a reasonable speed, e.g. of a quickly walking human, and is detected while in the forbidden zone, it shall be possible to bring the manipulatorto rest before the object and the manipulatorcan touch each other, or at least reduce the speed of the manipulatorto a safe level.

5 4 9 a) moving the end effectorto crate; 8 9 b) identifying a second workpiecewithin cratethat is fit to be seized, and its orientation; 8 c) bringing the end effector into a pose in which the identified second workpiececan be seized, and seizing it; 7 d) moving it to conveyor; 8 6 e) bringing the end effector into a pose suitable for combining second workpiecewith a first workpiece; 6 f) identifying a first workpiece g) combining the first and second workpieces. In the present example, the task of the robot and of its operation controllercan be divided into:

5 1 Identifying steps b) and f) can be carried out by the operation controllerbased on sensor data and do not involve moving the manipulator; they will not be considered in detail here.

2 FIG. 17 5 10 5 10 5 10 illustrates code of a program recorded in a storageassociated to both controllers,. The controllers share an instruction pointer that stores the line number of an instruction to be executed, so that at a given instant a same instruction from the program is supplied to both controllers,, but is not necessarily evaluated by both. Each instruction is either an action instruction to be executed by the operation controlleror a constraint instruction defining a constraint, the observance of which is will be monitored by safety controller. The entirety of constraints valid at a given instant of operation of the robot, and the instants or conditions at which a constraint is changed from the safety configuration of the robot for the task defined in the program.

A distinguishing feature can be provided for telling apart action instructions from constraint instructions, in the present case, constraint instructions can be recognized by a first character “#”.

1 3 4 13 An initial portion of the program comprises constraint instructions that define allowed ranges for various operation parameters of the robot in program lines Lto L. For defining ranges, a command “#allowedRange” is used, arguments of which are the parameter to be defined, and a description of an allowed range of values. These can be from zero to some allowed maximum in case of the speed with which the end effectormay be moved, or in case of the force it may apply to an object it is in contact with. In case of the space the manipulator is allowed to move in, specifying ranges of x, y and z coordinates along axes perpendicular to wallswould be sufficient and would allow to define an allowed space of cuboid shape in which the manipulator can come as close to the walls as might be necessary while prohibiting a collision with them. More complexly shaped operation spaces of the manipulator might be defined by successive #allowedRange instructions, each of which defines part of a space in which the manipulator is allowed to move, or by #forbiddenRange instructions by which part of an operation space allowed by preceding instructions is taken away again.

In many cases coordinate ranges is likely to be too limiting; so a pointer “description” to any data object appropriate for describing an irregular shape may be used.

1 3 10 1 5 1 After constraints have been set by inputting constraint instructions Lto Lto safety controller, the safety controller will monitor continuously whether movements of the manipulatorcontrolled by operation controllerbased on action instructions of the program observe the constraints and will switch the manipulatorto a safety mode described above as soon as a violation of any one of these constraints is observed.

4 14 An instruction “synchPoint” in line Ldefines a program line Lwhere execution is to continue in case of a violation being detected, and where it is defined what the robot will do in safety mode. The instruction may reoccur anywhere in the program, in order to enable variable reactions to a constraint violation depending on what the robot is doing in the moment the violation occurs.

5 5 9 12 9 4 9 8 5 17 12 5 1 3 2 FIG. A first action instruction to operation controller, in line L, is to identify cratein image data from sensorand to return a position “pCrate” close to the cratefrom where the end effectorcan dive into the cratein order to seize a workpiece. Instructions making up the function “find (Crate)” invoked in line Lare stored in a region of storagestarting at line Lin. When the “find” function is invoked, the safety controller records the fact, and it also takes note of the instruction pointer returning to an instruction following Lin the main function after the function “find” has terminated, but since the function “find” has no constraint instructions, constraints set by lines Lto Lof the main function continue to hold during execution of the function.

8 6 8 4 7 In a loop repeated while there are workpiecesin the crate, the manipulator is moved to the position pCrate by the action instruction of line L, a suitable workpieceis identified, and the end effectoris moved into a position and orientation appropriate for seizing it (L), and seizes the workpiece.

8 3 7 8 7 1 3 1 7 8 8 3 2 FIG. Let us consider the case where the workpiecesare fragile, and touching one or the other of them with the force allowed in line Lmight cause damage. This problem can conveniently be dealt with by replacing lines L, Lofby a function call “seizeWorkpiece(pWorkpiece, orientation)”. Arguments of the function call can be the same as of action instruction L. The code of the invoked function comprises a constraint instruction “#allowedRange [force, 0, Fmax]” which overrides that of Land ensures, by Fmaxbeing lower than Fmax, that when action instructions that may be identical in wording to Land Lare subsequently carried out within the function, workpieceswill be touched so softly that damage is prevented. When the function “seizeWorkpiece” terminates, the safety controller will replace constraint settings made in it by those of the main function, so that when execution of the main function continues, the definition of Lis valid again.

8 1 7 9 14 3 14 7 7 2 With the workpieceseized, the manipulatorhas to move towards conveyor. While in case of approaching the crate, the possibility of a person rushing in though passageand bumping into the manipulator could not be excluded, and the choice of allowed maximum speed vmax in Lhad to take account of this possibility, there is no way by which a person could run into the manipulator while it is moving away from passage. Therefore, exceptionally, a speed exceeding vmax can be allowed for the approach to the conveyor. Analogous to the case discussed above, such a higher speed could be allowed by programming the approach to the conveyoras a call of a function in which the limit set by Lis locally overruled by a higher limit.

2 9 1 1 10 7 1 6 1 8 6 11 8 9 Such a way of programming is risky, however, in that a programmer might unintentionally release a constraint by writing a function call without being aware that the called function will override a constraint to which the calling function is subject. In order to avoid such a possibility, the constraint imposed by Lis released momentarily, by the constraint instruction of L: the prefix “#once” ensures that the higher speed limit defined by “Sallowed Range [speed, 0, vmax]” (wherein vmax>vmax) will apply to the next action instruction L, which is the instruction to move to the conveyor. Here the manipulatorwill wait for the next workpieceto appear, and the loop ends by the manipulatormounting the workpieceheld by it to said next workpiece(L). The loop is repeated while there are workpiecesleft in the crate.

8 6 1 3 2 1 2 2 FIG. Let us also consider the case where the workpiecesare resistant to a high force and their installation in workpiecerequires application of a force Fmaxthat is higher than the force allowed in L. Just as discussed above for the speed, the constraint of Lmight be temporarily overridden by a command “#once #allowedRange [force, 0, Fmax]” in the main function, but that would require the programmer dealing with programming the overall movement of the manipulator to consider details of the installation process. This may easily lead to programming errors, and it makes adapting the program to variations of the workpieces complicated. Therefore, in this case, releasing the constraint of Lby imposing a less strict constraint on force in a function governing the installation process would be the preferred option. The problem of preventing an unintended release of constraints in such a case can be dealt with in the process of developing the program of, as will be explained below.

If the program has been drawn up properly, violations of constraints should not occur during normal operation. They may occur, however, while the program is under development, and a substantial part of the development effort is to adapt the movements the manipulator must perform in order to accomplish its task and the safety configuration to each other so that the task can be accomplished while all safety requirements for persons and hardware are met.

2 FIG. 1 1 5 1 1 5 10 1 12 1 While the program ofis under development, the physical manipulatorcan be replaced by a software model which, when fed the instructions which the physical manipulatorwould receive from controller, calculates the pose the manipulatorwould assume and the power its motors would consume if it carried out these instructions, returns resulting angle and power data as data from the internal sensors of the manipulatorto controllersand, and injects a calculated image of the manipulatorand its vicinity into sensor. So damage to the manipulatoror to persons can be prevented while operational safety of the program is not yet certified.

10 11 1 1 10 20 10 10 2 FIG. In the development stage, whenever a function is invoked, the safety controllercompares constraints valid in the invoking function to those of the invoked function, and issues a warning if constraints of the invoked function are less strict than those of the invoking function. Thus, when the function installwp( ) invoked in line Lof the main function ofcomprises a constraint instruction “#allowedRange [force, 0, Fmax”] in which Fmax>Fmax, the safety controllerissues a warning to the programmer via a display, and requires the programmer to confirm that loosening of the constraint is intentional. If it isn't, the programmer will have to amend the program, if the programmer confirms to the safety controllerthat loosening the constraint is intended, the safety controllerenables further execution of the program.

1 1 1 Addresses specified in #synchPoint instructions will in most cases be modified when development is finished, and the program is employed in production mode. While in the development stage, code accessed via a in #synchPoint instruction may specify debugging functions such as a storage dump or step-by-step operation, allowing (simulated) movement of the manipulatorto continue, the safety mode entered in case of a violation in production mode will actually cause the manipulatorto stop, to slow down or to limit movements of the manipulatorin some other appropriate way.

All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

The use of the terms “a” and “an” and “the” and “at least one” and similar referents in the context of describing the invention (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The use of the term “at least one” followed by a list of one or more items (for example, “at least one of A and B”) is to be construed to mean one item selected from the listed items (A or B) or any combination of two or more of the listed items (A and B), unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.

Preferred embodiments of this invention are described herein, including the best mode known to the inventors for carrying out the invention. Variations of those preferred embodiments may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the invention to be practiced otherwise than as specifically described herein. Accordingly, this invention includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the invention unless otherwise indicated herein or otherwise clearly contradicted by context.

1 manipulator 2 base 3 arm 4 end effector 5 operation controller 6 first workpiece 7 conveyor 8 second workpiece 9 crate 10 safety controller 11 joint 12 sensor 13 wall, barrier 14 passage 15 forbidden zone 16 person 17 storage 18 display