System and Method for Independent Safety Control

The present disclosure generally relates to an independent vehicle safing (making a vehicle safe) system.

An autonomous vehicle (AV) or a manual vehicle (MV) may include a vehicle safing system incorporated therein. For example, an AV may include an automated driving system having a safing system built therein. The system may include a plurality of sensors, computer hardware assets, and software assets employed to assess the driving environment and detect unsafe situations. If an unsafe driving situation is detected by the automated driving system, the AV may constrain operations within a safe driving envelope. For example, the AV may slow down, stop, or implement avoidance steering if an unsafe driving situation is detected. Similarly, a manual assistive drive system may be employed in a MV. As such, though a MV is driven under the control of the operator, a manual assistive drive system may take over vehicle control from the operator to constrain operations within a safe driving envelope.

Existing safing systems, whether employed in an AV or an MV, can be quite complex. Further, in addition to the complexity of existing safety systems, it can be difficult and/or costly to validate such systems. Often, an AV system or a manual assistive drive system is “custom” built for a particular make and model of a vehicle. Accordingly, the safing system employed, whether integrated into an AV system or designed as a manual assistive drive system, is generally validated for the particular make and model in which it was custom built. As such, if a manufacturer would like to employ a safing system for another make and model, the manufacturer generally custom builds another safing system, which in turn is validated for this other make and model. Accordingly, the custom nature of these safing systems and the accompanying need to revalidate such systems can be costly.

There is a need, therefore, for a safing system that can be employed on a variety of vehicles without the need to redesign and revalidate the safing system for each vehicle.

Referring to the discussion that follows and the Figures, illustrative approaches to the disclosed systems and methods are described in detail. Although the Figures represent some possible approaches, the Figures are not necessarily to scale and certain features may be exaggerated, removed, or partially sectioned to better illustrate and explain the present disclosure. Further, the descriptions set forth herein are not intended to be exhaustive, otherwise limit, or restrict the claims to the precise forms and configurations shown in the Figures and disclosed in the following detailed description.

With reference now to, a block diagram of an exemplary vehiclehaving an exemplary independent autonomous safety system (IAVSS)(a.k.a. a safing system or safety system) incorporated therein is shown. The vehicleincludes a plurality of sensorsfor tracking vehicle location and/or obstacle detection. These sensorsmay include, for example, some combination of video, range, inertial measurement units (IMUs), and/or inertial navigation sensors (INSs). Further, such sensors may include, for example, light detection and ranging (LIDAR) sensors, radar sensors, and/orD stereo wide sensors to name a few. Any sensor that gather details about the environment in which the vehicleoperates is contemplated.

The vehiclealso includes a steering control module, dashboard host control interface (HCI), one or more vehicle to vehicle (V2V), vehicle to infrastructure (V2I), and vehicle to pedestrian (V2P) transceivers, a sensor bus, a controller area network (CAN) bus, and controller. If the vehicleis an autonomous vehicle (AV), the controllermay then be an AV controller. Alternatively, if the vehicleis a manual vehicle (MV), the controllerwould be a MV controller, which may or may not be coupled to an assistive drive system.

The controlleris in communication with the sensorsvia the sensor bus. Further, the controlleris communicatively coupled to the CAN bus, which in turn is in communication with microcontrollers and/or other devices for controlling the vehicle. Still further, the controllermay be communicatively coupled to a self-location subsystemif the vehicleis an AV.

Also shown in the vehicleare speed, brake, acceleration, and transmission control system.

The IAVSSis communicatively coupled to an emergency stop system, the controller, the sensors, and the self-location subsystem(if there is one). The IAVSSmay be comprised of a software stack and/or one or more hardware modules. The IAVSSoperates independently from vehicle planning systems (if any) and other control systems. For example, the IAVSSis independent from the controller. As such, the IAVSSis not incorporated into the controller, but rather is independent from, and operates in parallel to, the controller. Since the IAVSSoperates in parallel to the controller, the IAVSSmay more easily be incorporated into a vehicle. That is, it may be configured to work with already existing controllers from other vehicles (not shown), whether it is an AV or MV.

The IAVSSis configured to override any driver or autonomous system if an unsafe driving condition is identified. Further, once the IAVSSidentifies an unsafe driving condition, the IAVSSinitiates mitigation behavior. The mitigation behavior may include, for example, vehicle stopping and/or shutdown, starting and moving, steering away and/or around obstacles, signaling a breakdown, calling for help, accelerating to avoid and unsafe condition, communicating with infrastructure systems (e.g., emergency services), and/or slowing to safe speeds due to weather, traffic and/or obstacles. Other mitigation behaviors are contemplated.

Effectively, the IAVSSoperates as a “safe sensor.” If the vehicle is an AV, and the IAVSSidentifies or senses an unsafe driving condition, the IAVSSinjects a “safety assessment” message into the AV suite to initiate mitigation behavior. Similarly, if the vehicle is an MV, and the IAVSSidentifies or senses an unsafe driving condition, the IAVSSinjects a “safety assessment” message into an MV control system or assistive drive system (e.g., an advanced driver assistive systems (ADS)) to initiate mitigation behavior. For example, the safety assessment message may ensure a driver does not disobey traffic laws (e.g., take the vehicle to speeds above posted speed limits).

With reference now to, an exemplary block diagram of the IAVSSis shown. The IAVSSincludes a world model, a localization module, and a safety aggregator module.

Referring to, The world modelreceives data (a.k.a. sensor output) from one or more of the plurality of sensorsto at least create a model of obstacles and terrain the vehicles is, or will be, encountering. One or more of these sensorsmay also provide data regarding details identified in traffic control systems (e.g., the state of traffic lights, traffic flow state set forth in signage, and/or yield and speed limit rules set forth in signage).

The world modelalso receives driving inputs. These inputs may include, for example, driver command inputs (e.g., turning, braking, and/or accelerating) if the vehicle is an MV. Alternatively, the inputs may include AV driving command inputs from an AV suite if the vehicle is an AV. Effectively, the world model modulefuses sensor data and driving inputs to create a world model that details the driving environment including obstacles and terrain around the vehicle.

Whether the vehicle is an AV or a MV, the world modelhelps ensure that the driving inputs/commands do not cause the vehicle to disobey traffic laws (e.g., speed limits) and/or implement an unsafe driving condition (e.g., entering a prohibited area or a space already occupied by a vehicle or other obstacle).

With the information gathered via the world model, the IAVSSmay continuously do the following: assess the health of AV hardware if implemented (e.g., controller), other controls, and applicable software; assess the position and velocity of the vehiclein relation to the terrain and objects (e.g., vehicles, pedestrian(s), barriers, debris, and etc.) along the driving path; help predict potential collisions or unsafe driving events/conditions in the driving path; carry out proximity checking of objects close to the vehicle within predetermined safety zones around the vehicle in the direction of travel; verification that the vehicle is, and is planning to, operate in safely traversable area (in contrast to in areas denied by safety concerns or traffic rules consistent with vehicle operating rules); and check for system wide commanded slowdowns and/or stops (i.e., commands from a central traffic manager to stop and/or slowdown). If the vehicle is an MV, the IAVSSmay also check for manually directed emergency stops while also modeling vehicle environment in light of driving inputs.

As mentioned above, the IAVSSalso includes the localization modulethat provides information to the world model. Generally, the localization moduledetermines vehicle location, speed, acceleration, and/or direction of travel. It is noted that the localization moduleof the IAVSSis distinct from a localization module or subsystemthat may be included in an AV or MV.

The localization moduleof the IAVSScan make localization determinations through a variety of techniques and combinations thereof. For example, the localization modulemay rely on vehicle position data from a global positioning system (GPS) provided by a variety of countries and companies worldwide. One or more of the exemplary sensorsmay include a GPS unit that gathers position data from a GPS to provide location information to the localization module.

One or more INSs and/or IMUs may also be employed in conjunction with the with the GPS to provide, for example, location, acceleration, speed, and direction information to the localization module. Often, an INS system and/or IMU system may be used in conjunction with a GPS system to obtain a more accurate measurement of vehicle location along with a more accurate measurement of speed, acceleration, and direction. While the terms “speed” and “acceleration” may be used colloquially herein, it is understood technical terms such as “velocity” and “acceleration” are vectors and, therefore, inherently include direction information therein.

In yet another example, one or more of the sensorsmay be employed to determine wheel revolutions to measure turn angle, speed, distance traveled, and/or etc., which in turn is provided to the localization module. In other words, odometry techniques may also be employed to determine turn angle, speed, distance traveled, direction, and etc.

In still yet another example, the localization modulemay rely on landmark sightings to determine vehicle location, direction and etc. For example, a database(or other memory) that is accessible by the location modulemay include detailed information of landmarks (e.g., location information of mountains and/or other physical terrain characteristics, buildings, road signs, stop lights, intersections, warehouse markers and/or structural elements, and etc.). A landmark may be identified via one or more of the sensors. As such, the landmark location information in the databasemay be employed to determine the vehicle's location. Such a technique may, for example, be helpful to determine vehicle location when GPS information is not available (e.g., when GPS signal is blocked or when a GPS unit is defective).

Several localization techniques are described above, but it is understood that the localization modulemay rely on other localization techniques not discussed.

With continued reference to, and as mentioned above, the IAVSSincludes a safety aggregator (SA) modulein addition to the localization moduleand the world module. The SA moduleidentifies unsafe driving conditions. As such, the SA module may identify sources of failure or potential failure. For example, the SA modulemay identify failure of hardware systems (e.g., sensors or other hardware systems), failure of software modules implementing vehicle autonomy and/or sensor operation, and failure of internal diagnostics performed by vehicle equipment (e.g., sensors, engine controls, transmission controls, various control actuators, steering, braking, acceleration, and/or etc.).

The SA modulemay identify the failure or potential failure based on internal assessments. As discussed above, the world model modulefuses sensordata to continually update a model of the vehicle in its surroundings. The world model modulemay compare outputs from various sensors to measure sensor error. If the world model moduleidentifies a sufficient error (i.e., an error greater than a predefined threshold or an error that may result in an unsafe driving condition) associated with a sensor, the world model modulemay be configured to provide a fault condition to the SA module. In turn, the SA modulemay ensure that data from faulty sensor is locked out of the world model module. As such, the world model modulemay not employ the faulty sensor data in the sensor data fusion process employed to create the world model.

Further, if the SA moduledetermines that that enough sensor data is faulty (or missing), the SA module may provide a notification to a user and/or provide commands to the controllerto initiate safing actions (e.g., slowing the vehicle or stopping the vehicle).

The SA modulemay also be configured to receive fault notifications. That is, the SA modulemay receive fault notifications from hardware modules and sensors (e.g., one or more of the sensorsof) that incorporate their own fault detection systems. If, based on received fault notification(s), the SA moduleidentifies an unsafe driving condition, the SA modulemay control the vehicle via commands sent to the controllerand/or simply notify the operator or user of the vehicle.

Further, the SA modulemay identify vehicle commands that will or may result in some type of unsafe driving condition. For example, if an operator initiates a command to emergency stop (E-Stop), the SA modulemay determine if such an E-stop command initiated by the driver will or may result in an unsafe driving condition. Further, the SA modulemay determine or identify unsafe driving speeds and a dangerous vehicle location relative to fixed and/or moving objects in the environment.

With reference to both, it is again noted that the IAVSSis connected in parallel to the vehicle controller(whether it is an MV controller, an AV controller, or some combination of both) and it may have access to one or more of the sensors. In another example, the IAVSSmay be part of a system that employs its own sensors (not shown) and it may employ those sensors or some combination of its own sensors and vehicle sensorsto identify system, vehicle, and/or driver failures. In either case, the IAVSSis operated in parallel to the operating systems (e.g., the controller) of the vehicle.

Regardless of the sensors employed, since the IAVSSsits in parallel to the vehicle controller, the IAVSS(e.g., via the SA module) is able to take control of the vehiclevia the CAN busif a safety violation or other unsafe driving condition is identified via the SA module. The unsafe driving condition may, for example, be the result of an internal vehicle systems failure and/or a pending unsafe driving event (e.g., a potential collision due to leaving an authorized driving area such as entering an intersection incorrectly, heading for a potential collision, and/or changing lanes incorrectly).

The IAVSSand its accompanying SA moduleimplements a level autonomy to effect safety responses, while the controller (MV or AV)implements driving functions.

It is noted that the techniques and acts of the IAVSSmay be incorporated into one or more computer readable storage mediums, which may be carried out via one or more processors. That is, instructions employed to carry out acts of the IAVSSmay be stored on the one or more storage mediums. Alternatively, a hardware/software implementation may be carried out such that one or more processorsand software module(s) are built up to carry out the actions of the IAVSS.

Still further, in some examples, the IAVSScould be incorporated into a vehicle controller (e.g., the controllerof) or an AV suite that may be employed. In any of the examples, the IAVSSoperates in parallel to any existing controller system so that the IAVSSserves as the arbitrator of safety.

The IAVSSdiscussed above may be deployed in a variety of vehicles. For example, the vehiclemay be a construction vehicle, airport support vehicle, material handling vehicle, and/or a military vehicle-to name a few. Since the IAVSSoperates in parallel to any existing AV system, revalidation of the safing system may not be needed. Further, since the IAVSSoperates in parallel to any existing AV system, it may be added at a later date.

Referring now to, an exemplary techniquefor enhancing vehicle safety is shown. Process control begins at block, where identifying an unsafe driving condition of a vehicle is carried out. As discussed above with respect to, there may be a variety of events that may cause an unsafe driving condition. For example, malfunction(s) in an autonomous driving system, errors made by a driver, incorrect sensor data or lack thereof, and/or threats imposed by driving terrain or other objects (e.g., other vehicles and/or animals) serve as just a few examples of what can cause an unsafe driving condition. The exemplary IAVSSofmay be employed to identify the unsafe driving condition(s).

With continued reference to the technique of, after identification of the unsafe driving condition, process control proceeds to block, where overriding at least one driving input when the unsafe driving condition is identified is carried out. The driving input(s), whether they come from a driver or an AV system, are intended to be provide to a driving control system to controls the vehicle. Process control overrides the driving input(s) at blockand, as such, prevents the driving inputs from controlling the vehicle.

After the driving input(s) are overridden, or while the driving input(s) are being overridden, process control carries out providing at least one new input (a.k.a. new driving input(s)) to the driving control system to control the vehicle at block. The at least one new input causes the driving control system to change at least one of the direction and the speed of the vehicle to remove or mitigate the unsafe driving condition. The new input may be provided to the driving control system via, for example, a controller area network (CAN bus) that allows communications between microcontrollers.

Techniquecontinues at decision block, where it is determined whether or not to continue the autonomous safety monitoring technique. If the technique is continued, process control proceeds to blockas identification of unsafe driving conditions continues.

If, on the other hand, it is determined not to continueidentifying unsafe driving conditions, process control proceeds to an END. One example of such a decision could simply be the vehicle being turned off.

The identification of unsafe driving conditionsin the exemplary techniqueofmay be carried in a variety of ways. For example,illustrates an exemplary techniquefor identifying unsafe driving conditions.

The exemplary techniqueincludes monitoring driving inputs at block, monitoring sensor data from a plurality of sensors at block, and monitoring any AV driving system or assistive driving system the vehicle may employ at block. In some instances, the vehicle may not include an AV driving system or an assistive driving system. As such, techniques for identifying unsafe driving conditions need not employ the monitoring set forth in block.

Nonetheless, while monitoring the driving inputs at block, process control determines whether any of the driving inputs will create an unsafe driving condition at block.

If the driving inputs will not createan unsafe driving condition, process control proceeds to blockand the driving input(s) are not overridden. Alternatively, if the driving input(s) will createan unsafe driving condition, the driving input(s) are overridden (see block,). For example, an IAVSS module may determine that, based on the sensor inputs, one or more driving inputs may cause a vehicle to be set on a collision course with another vehicle or object. In such an instance, process control causes at least some driving input(s) (e.g., steering, acceleration, and/or braking) to be overridden.

As mentioned above, the exemplary techniquefor identifying an unsafe driving condition also includes monitoring sensor data from a plurality of sensors (e.g., one or more of the sensorsof) at block. At block, process control determines if any of the sensors are defective and, if so, if the defective sensor(s) will produce an unsafe driving condition. That is, if there is one or more defective sensors, the techniquedetermines if the defective sensor may create an unsafe driving condition. An IAVSS may determine if a sensor is defective. Additionally, the sensor may have its own system to determine if it is defective, in which case the sensor system(s) would notify the IAVSS it is defective.

If it is determined that the defective sensor(s) would not createan unsafe driving condition, process control proceeds to blockand the driving inputs are not overridden. For example, one or more sensors may be producing minor senor errors that do not impact the driving safety of the vehicle.

On the other hand, it may be determined that one or more defective sensors do impactthe driving safety of the vehicle. In such a scenario, process control may cause at least some of the current driving inputs to be overridden (see, e.g., blockof). For example, if the vehicle is an AV vehicle, and it is determined one or more defective sensors could cause an unsafe driving condition, the IAVSS may override the AV system and safely bring the vehicle to stop.

As also mentioned above, in addition to monitoring driving inputsand sensor data, the techniqueofalso includes monitoring any AV driving system that may exist at block. The AV driving system may include an AV suite including hardware and/or software. Effectively, the components that control the autonomous driving of the AV vehicle are monitored.

While monitoring the AV drive system, process control proceeds to blockwhere it is determined whether or not the AV driving system has a fault. Sensor datamay be employed to make the determination at block. If the AV Drive system does nothave a fault, process control proceeds to blockand driving inputs are not overridden.

Alternatively, if it is determined that the AV driving system has a fault, at least one driving input may be overridden (e.g., blockof). An IAVSS may, for example, be monitoring the AV drive system and determine that the AV drive system's world model module and/or localization module is producing errors. As such, the IAVSS may determine that the AV drive system includes a fault, thus causing any drive inputs from the AV system to be overridden.

It is noted that a world model module and localization module of an AV system is distinct from a world model module (e.g., the world model moduleof) and localization module (e.g., the localization moduleof) of an IAVSS (e.g., IAVSS,). As discussed above, with regard to an AV, the IAVSS operates in parallel to the AV driving suite. Accordingly, the IAVSS serves as the arbiter of safety.