Vehicle Control System

This application claims priority to Japanese Patent Application No. 2025-045693 filed on Mar. 19, 2025. The disclosure of the above-identified application, including the specification, drawings, and claims, is incorporated by reference herein in its entirety.

The present disclosure relates to a technology for controlling a vehicle function in response to an operation of a portable terminal by a driver.

Japanese Patent No. 6124841 (JP 6124841 B) discloses a portable device including an acceleration sensor that detects an acceleration along three axes.

In addition, Japanese Patent No. 5955254 (JP 5955254 B) discloses a system that remotely operates a work vehicle using a portable information terminal. The portable information terminal includes a display that displays a driving operation screen that simulates a driving operation region of the work vehicle. A pseudo-operation device (on-screen button incorporated into the driving operation screen) is displayed on the driving operation screen in a disposition corresponding to a disposition of an operation device. An operation input of the pseudo-operation device can be made by designating the on-screen button.

A method of using an operation button displayed on a screen of a portable terminal for operating a vehicle function using the portable terminal by a driver of a vehicle (for example, an automobile) (for example, refer to JP 5955254 B) is known. However, with this method, visual confirmation of a touch position on the screen by the driver is needed. In addition, as the number of vehicle functions to be operated increases, the number of operation buttons on the screen increases. As a result, complicated display on the screen may be caused or the driver may focus on the screen during vehicle driving.

The present disclosure has been made in view of the problem, and an object of the present disclosure is to provide a vehicle control system in which a driver can operate a vehicle function using a portable terminal without focusing on a screen of the portable terminal.

A vehicle control system according to the present disclosure includes a vehicle and a portable terminal. The vehicle includes a plurality of in-vehicle devices configured to implement a vehicle function and a control device. The portable terminal is operated by a driver of the vehicle for operating the vehicle function. The portable terminal includes one or more sensors configured to detect information on an orientation of the portable terminal and an acceleration applied to the portable terminal as sensor information, and a processor. The processor is configured to execute operation request specifying processing of specifying an operation request of the driver regarding the vehicle function based on the sensor information, and function control processing of generating a control command in response to the specified operation request and transmitting the generated control command to the vehicle. The control device is configured to control an in-vehicle device corresponding to the vehicle function related to the control command in response to the control command received from the portable terminal.

According to the present disclosure, the vehicle function can be operated by the driver performing an action, such as changing the orientation of the portable terminal or knocking the portable terminal. Therefore, the driver can operate the vehicle function using the portable terminal without focusing on the screen of the portable terminal.

Embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 FIG. 1 1 10 20 20 10 20 is a schematic diagram showing an example of a configuration of a vehicle control systemaccording to the present embodiment. The vehicle control systemincludes a portable terminaland a vehicle. The vehicleis, for example, an automobile. The portable terminalis operated by the driver for operating a plurality of vehicle functions Fv of the vehicle.

10 20 20 10 20 20 10 20 20 The portable terminalcommunicates with the vehicle. In the present embodiment, the driver rides in the vehicleand operates the vehicle function Fv using the portable terminalthat communicates with the vehiclewirelessly or via a wired connection. However, the driver does not always have to ride in the vehiclein a case of operating the vehicle function Fv, and the vehicle function Fv may be operated using the portable terminalthat communicates with the vehiclewirelessly outside the vehicle.

20 20 20 10 The vehicle function Fv includes a basic vehicle function Fv1 related to vehicle traveling, such as running, turning, and stopping. That is, the driver operates the vehicle(drives the vehicle) related to the traveling of the vehicleusing the portable terminal. In addition, the vehicle function Fv includes an auxiliary or additional vehicle function Fv2. Examples of the vehicle function Fv2 include blinking/turning off of a turn signal, blinking/turning off of a hazard lamp, switching of a shift range, sounding/stopping of a horn, blinking/turning off of a headlight, operation/stop of an air conditioner, and operation/stop of a wiper.

10 10 11 12 13 14 15 11 10 15 12 20 10 10 3 3 FIGS.A andB The portable terminalis, for example, a portable terminal, such as a smartphone or a tablet terminal. The portable terminalincludes, for example, a touch panel, a communication device, a processor, a storage device, and sensors. The touch panelis provided on one surface of the portable terminaland includes a display screen and a touch sensor included in the sensors. The touch sensor is configured to detect a touch of the driver on the display screen. The communication devicecommunicates with the vehicle. A shape of the portable terminalis not particularly limited, but the portable terminalis provided in a plate shape (for example, a rectangular plate shape) in which one side is a short direction and the other side is a long direction (see).

13 10 13 13 13 14 13 14 13 14 The processor (processing circuit)executes various types of processing for operating the vehicle functions Fv using the portable terminal. Examples of the processorinclude a general-purpose processor and a specific-purpose processor. In addition, examples of the processorinclude a central processing unit (CPU), a graphics processing unit (GPU), and an application specific integrated circuit (ASIC). Further, examples of the processorinclude a field-programmable gate array (FPGA). The storage devicestores various types of information needed for the processing by the processor. Examples of the storage deviceinclude a volatile memory, a non-volatile memory, a hard disk drive (HDD), and a solid state drive (SSD). The processorexecutes a computer program. The computer program is stored in the storage device. Alternatively, the computer program may be recorded in a computer-readable recording medium or may be provided via a network.

15 10 10 10 The sensorsinclude a sensor for detecting information on the orientation of the portable terminal(more specifically, a tilt direction and a tilt angle of the portable terminal) and an acceleration applied to the portable terminal. The sensor is configured to include, for example, an acceleration sensor and a gyro sensor, and is, for example, a six-axis motion sensor in which a three-axis acceleration sensor and a three-axis gyro sensor are combined. The information on the orientation and the acceleration detected by the sensor (orientation information Isp and acceleration information Isa) corresponds to "sensor information Is" according to the present disclosure.

15 15 In addition, the sensorsmay include, for example, at least one of a microphone or an image sensor (camera). That is, the sensor information Is may include information (sound information Iss) of the microphone and information (image information Isi) of the image sensor. Further, the sensor information Is may include information (screen touch information Ist) indicating whether the screen is touched by the touch sensor. The sensorscorrespond to an example of "one or more sensors" according to the present disclosure.

20 21 22 23 24 21 10 22 22 20 20 22 The vehicleincludes a communication device, a plurality of in-vehicle devices, sensors, and a control device. The communication devicecommunicates with the portable terminal. The in-vehicle devicesare devices for implementing the vehicle functions Fv. Specifically, the in-vehicle devicesinclude a traveling device for implementing the basic vehicle function Fv1 related to the vehicle traveling. The traveling device includes a drive device, a steering device, and a braking device. The drive device includes one or both of an electric motor and an internal combustion engine for driving the vehicle. The steering device includes an electric motor that turns the wheels. The braking device includes a brake actuator for braking the vehicle. In addition, the in-vehicle devicesinclude various devices for implementing the auxiliary or additional vehicle function Fv2. Examples of the various devices include a light device (turn signal, hazard lamp), a shift device (device for switching the shift range), a horn device, a headlight device, an air conditioner device, and a wiper device.

23 20 20 20 The sensorsinclude, for example, a recognition sensor, a vehicle state sensor, and a position sensor. The recognition sensor (for example, a camera and a radar) recognizes a situation around the vehicle. The vehicle state sensor (for example, a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor) detects a state of the vehicle. The position sensor (for example, a global navigation satellite system (GNSS) receiver) detects a position and an orientation of the vehicle.

24 24 25 25 26 26 25 26 25 25 13 26 14 The control deviceis a computer that controls the vehicle functions Fv. The control deviceincludes one or more processors(hereinafter, simply referred to as a processor) and one or more storage devices(hereinafter, simply referred to as a storage device). The processorexecutes various types of processing related to the control of the vehicle functions Fv. The storage devicestores various types of information needed for the processing by the processor. A configuration example of the processoris the same as that of the processordescribed above. In addition, a configuration example of the storage deviceis the same as that of the storage devicedescribed above.

10 10 10 20 20 22 10 The driver uses the portable terminalto operate the vehicle functions Fv. The portable terminalgenerates a control command INS in response to the operation of the portable terminalby the driver, and transmits the generated control command INS to the vehicle. The vehiclecontrols the in-vehicle devicein response to the received control command INS. As a result, the control of the vehicle functions Fv using the portable terminalis realized.

10 Here, it is considered to use an operation button displayed on a screen of the portable terminal for the driver to operate the vehicle function Fv using the portable terminal. However, with this method, visual confirmation of a touch position on the screen by the driver is needed. In addition, as the number of vehicle functions Fv to be operated increases, the number of operation buttons on the screen increases. As a result, complicated display on the screen may be caused or the driver may focus on the screen during vehicle driving.

10 13 10 10 20 24 25 20 22 Therefore, in the present embodiment, the portable terminal(processor) executes "operation request specifying processing" and "function control processing". In the operation request specifying processing, the portable terminalspecifies an operation request Rm of the driver regarding the vehicle function Fv based on the sensor information Is. In the function control processing, the portable terminalgenerates the control command INS in response to the specified operation request Rm, and transmits the generated control command INS to the vehicle. The control device(processor) of the vehiclecontrols the in-vehicle devicecorresponding to the vehicle function Fv related to the control command INS in response to the received control command INS.

2 FIG. 10 10 31 32 33 24 20 34 13 25 is a diagram showing an example of a functional block related to the control of the vehicle functions Fv using the portable terminalaccording to the present embodiment. As the functional block, the portable terminalincludes a filter processing unit, an operation determination unit, and a function controller. The control deviceof the vehicleincludes a vehicle controller. These functional blocks are implemented by software at the time when a program related to the present control is executed by the processoror.

31 32 31 31 10 10 15 10 32 5 FIG.A The processing of the filter processing unitand the operation determination unitcorresponds to the "operation request specifying processing". The sensor information Is (for example, the orientation information Isp and the acceleration information Isa) is input to the filter processing unit. The filter processing unitextracts operation information Im (that is, information on an operation M intended by the driver for operating the vehicle function Fv) of the driver related to the vehicle function Fv from the sensor information Is. More specifically, the operation information Im is different from a change in the sensor information Is caused by gripping the portable terminalor moving the driver (holder), or the like. The operation information Im is obtained by observing an obvious motion of the portable terminalcaused by the actuation of the driver via the sensors. An example of the operation information Im is "right corner of the portable terminalis knocked". The extracted operation information Im is output to the operation determination unit. A method of extracting the operation information Im will be described later with reference to.

32 10 31 32 5 5 FIGS.B andC The operation determination unitdetermines the operation M intended by the driver based on the extracted operation information Im. For example, it is assumed that the operation information Im indicating that "the right corner of the portable terminalis knocked" is extracted N times (N: predetermined value) within a predetermined time by the filter processing unit. In this case, the operation determination unitdetermines (specifies) that the operation request Rm of the driver is "switching of blinking/turning off of the right turn signal". A method of determining the operation request Rm will be described later with reference to.

14 10 10 The storage deviceof the portable terminalstores a correspondence relationship C between the "operation M of the portable terminal" by the driver and an "actuation O of the vehicle function Fv" as a database.

3 3 FIGS.A andB 4 4 FIGS.A andB 1 1 10 1 2 2 10 2 1 2 are diagrams showing an example of a correspondence relationship Cbetween an operation Mof the portable terminalduring a horizontal holding and an actuation Oof the vehicle function Fv.are diagrams showing an example of a correspondence relationship Cbetween an operation Mof the portable terminalduring a vertical holding and an actuation Oof the vehicle function Fv. The driver has previously learned such a correspondence relationship Cand C.

10 10 1 10 2 10 1 1 2 10 3 10 2 1 10 20 11 3 FIG.A 4 FIG.A 4 FIG.A e e e A way of holding the portable terminalin a case where the driver operates the vehicle functions Fv may be selectable between "horizontal holding" and "vertical holding". As shown in, the horizontal holding corresponds to a holding method in which both endsand(see) of the portable terminalin the longitudinal direction Dare held by both hands Hand H. On the other hand, as shown in, the vertical holding corresponds to a holding method in which one endof the portable terminalin the short direction Dis held by one hand (for example, H) of the operator. For example, the driver operates the portable terminalto select one of the horizontal holding or the vertical holding before starting driving of the vehicle. In this case, the driver selects, for example, one of a horizontal holding selection button and a vertical holding selection button displayed on the touch panel.

13 1 31 1 10 1 1 32 1 1 3 FIG.B 3 FIG.B In a case where the horizontal holding is selected by the driver, the processoruses the correspondence relationship Cshown in. Specifically, the filter processing unitextracts the operation information Im related to the operation Mof the portable terminalincluded in the correspondence relationship Cfrom various types of sensor information Is.shows examples of operations M1A to M1M that are specific examples of the operation M. The operations M1A to M1C and M1L, M1M can be extracted from the acceleration information Isa. The operations M1D, M1E can be extracted from the orientation information Isp. The operations M1F, M1G can be extracted from the acceleration information Isa and the screen touch information Ist. The operation M1H can be extracted from the screen touch information Ist. The operations M1I, M1J can be extracted from the orientation information Isp and the screen touch information Ist. The operation M1K can be extracted from the sound information Iss. The operation determination unitdetermines (specifies) the operation request Rm of the driver (in other words, the actuation Oof the vehicle function Fv desired by the driver) based on the extracted operation information Im and the correspondence relationship C.

1 1 1 10 10 1 10 2 10 11 3 4 3 FIG.B 3 FIG.A 2 FIG. 3 FIG.A 3 FIG.A The contents of each specific example of the operation Mand the actuation Oare as described in, and the repeated description thereof will be omitted here. However, the contents of each specific example will be described as follows. Each of the "right-left" and "up-down" in each operation Mcorresponds to the right-left and up-down of the paper surface of. The "right side surface" of the portable terminalin the operation M1A may include the "right corner" as in the example shown in(the same applies to the "left side surface" in the operation M1B). The "forward tilt" in the operation M1D corresponds to tilting the portable terminalin the rotation direction Rin, and the "backward tilt" in the operation M1E corresponds to tilting the portable terminalin the rotation direction R. The "right back" in the operation M1F corresponds to a portion on the right side of the paper surface on a surface (back surface) opposite to a front surface of the portable terminalon which the touch panelis provided (the same applies to the "left back" in the operation M1G). The "clockwise" in the operation M1I corresponds to the rotation direction Rin, and the "counterclockwise" in the operation M1J corresponds to the rotation direction R.

13 2 31 2 10 2 2 32 2 2 4 FIG.B 4 FIG.B At the time when the vertical holding is selected by the driver, the processoruses the correspondence relationship Cshown in. Specifically, the filter processing unitextracts the operation information Im related to the operation Mof the portable terminalincluded in the correspondence relationship C.shows examples of operations M2A to M2M that are specific examples of the operation M. The operations M2A to M2E and M2L can be extracted from the acceleration information Isa. The operations M2F, M2G and M2I, M2J can be extracted from the orientation information Isp and the screen touch information Ist. The operation M2H can be extracted from the screen touch information Ist. The operation M2K can be extracted from the sound information Iss. The operation M2M can be extracted based on, for example, the image information Isi together with the acceleration information Isa and the orientation information Isp. The operation determination unitdetermines (specifies) the operation request Rm of the driver (in other words, the actuation Oof the vehicle function Fv desired by the driver) based on the extracted operation information Im and the correspondence relationship C.

2 2 2 10 11 10 5 10 6 10 7 4 FIG.B 4 FIG.A 4 FIG.A 4 FIG.A The contents of each specific example of the operation Mand the actuation Oare as described in, and the repeated description thereof will be omitted here. However, the contents of each specific example will be described as follows. Each of the "right-left" and "up-down" in each operation Mcorresponds to the right-left and up-down of the paper surface of. The "back" in the operations M2D and M2E corresponds to a portion on the surface (back surface) opposite to the front surface of the portable terminalon which the touch panelis provided. The "forward tilt" in the operations M2F and M2G corresponds to tilting the portable terminalin the rotation direction Rin. The "tilt to the right" in the operation M2I corresponds to tilting the portable terminalin the rotation direction Rin, and the "tilt to the left" in the operation M2J corresponds to tilting the portable terminalin the rotation direction R.

1 2 10 10 13 10 It is assumed that the driver does not intend the operation Mand Mbut drops the portable terminal. As a result, it is assumed that the sensor information Is (the acceleration information Isa and/or the orientation information Isp) indicating that the portable terminalreceives a strong impact or rotates violently is received. In this case, the processormay execute processing of stopping the control of the vehicle function Fv using the portable terminal.

2 FIG. 6 FIG. 32 33 33 33 33 20 In, the operation request Rm from the operation determination unitis input to the function controller. The processing of the function controllercorresponds to the "function control processing". The function controllergenerates the control command INS (for example, a blinking command of the right turn signal) in response to the specified operation request Rm. The function controllertransmits the generated control command INS to the vehicle. A method of generating the control command INS will be described later with reference to.

33 34 20 34 22 34 The control command INS from the function controlleris input to the vehicle controlleron the vehicleside. The vehicle controllercontrols the in-vehicle devicecorresponding to the related vehicle function Fv in response to the received control command INS. For example, in a case where the control command INS is the blinking command of the right turn signal, the vehicle controllercontrols the turn signal device such that the right turn signal blinks.

5 FIG.A 2 FIG. 31 31 31 31 20 10 20 10 20 10 20 is a diagram for supplementarily describing details of the filter processing unitshown in. As described above, the filter processing unitextracts the operation information Im of the driver from various types of sensor information Is. For example, the filter processing unitselectively or in combination uses a plurality of filters A to C for extracting the operation information Im. Examples of the filters include a frequency-dependent filter, such as a low-pass filter or a band-pass filter, and a neural network model generated by machine learning. Various parameters of each filter are input to each of the filters together with the sensor information Is. The filter processing unitmay be configured to switch at least one of a type of the filter to be used or a parameter thereof according to a type (the acceleration information Isa, the orientation information Isp, the sound information Iss, and the like) of the input sensor information Is. In addition, in an example in which the driver rides in the vehicleand operates the portable terminal, at least one of the filters may be configured to have a function of separating an influence of the motion of the vehicleon the sensor information Is. The determination of the driver's boarding by the portable terminalcan be performed, for example, by using a seat sensor or a seat belt wearing sensor mounted in the vehicle, or by using a determination result of the connection of the portable terminalto the device mounted in the vehicle.

5 5 FIGS.B andC 2 FIG. 3 FIG.B 4 FIG.B 32 32 are diagrams for supplementarily describing details of the operation determination unitshown in. As described above, the operation determination unitdetermines the operation M intended by the driver (that is, specifies the operation request Rm) based on the extracted operation information Im. The operation request Rm of the driver may include operation requests Rm1 and Rm2. The operation request Rm1 corresponds to an operation request directly related to the vehicle traveling, such as running, turning, and stopping (that is, the operation request regarding the basic vehicle function Fv1). For example, the operation request Rm1 corresponds to the operations M1F, M1G and M1I, M1J inand the operations M2F, M2G and M2I, M2J in. The operation request Rm2 corresponds to an operation request that is not directly related to the vehicle traveling (that is, the operation request regarding the auxiliary or additional vehicle function Fv2).

32 32 32 5 FIG.B 5 FIG.C The operation determination unitspecifying the operation request Rm based on the extracted operation information Im may include, as shown in, a condition that the operation information Im is extracted together with the touch of the display screen by the driver. In this case, the operation determination unitmay include determining whether the operation request Rm1 (first operation request) is present. The operation determination unitspecifying the operation request Rm may include determining whether the operation request Rm2 (second operation request) is present without the condition that the operation information Im is extracted together with the touch of the display screen as shown in.

6 FIG. 2 FIG. 6 FIG. 32 33 33 10 13 20 20 is a diagram for supplementarily describing details of the operation determination unitshown in. As described above, the function controllergenerates the control command INS in response to the specified operation request Rm. More specifically, as shown in, the vehicle state information Iv may be input to the function controllertogether with the operation request Rm. The vehicle state information Iv is information indicating a current control state of the vehicle function Fv related to the input operation request Rm. The portable terminal(processor) communicates with the vehicleand acquires the vehicle state information Iv related to the input operation request Rm (for example, switching of blinking/turning off of the right turn signal) from the vehicle.

6 FIG. 33 As shown in, the vehicle state information Iv is, for example, "the vehicle function Fv is in an operation state". In this example, the function controllergenerates the control command INS (for example, a turn-off command of the right turn signal) for stopping the vehicle function Fv.

33 In another example, the vehicle state information Iv is "the vehicle function Fv is in a stop state". In this example, the function controllergenerates the control command INS (for example, a blinking command of the right turn signal) for operating the vehicle function Fv.

20 33 In still another example, the vehicle state information Iv is "the control state of the vehicle function Fv has reached an upper limit (for example, the speed or the steering angle of the vehiclehas reached the upper limit)". In this example, the function controllerlimits the control command value (for example, the target speed and the target steering angle) such that the upper limit is not exceeded.

20 33 In still another example, the vehicle state information Iv is "the change rate of the control state of the vehicle function Fv has reached an upper limit (for example, the change rate of the speed or the steering angle of the vehiclehas reached the upper limit)". In this example, the function controllerlimits the control command value (for example, the target speed and the target steering angle) such that the upper limit of the change rate is not exceeded.

33 22 As described above, the function controllermay generate the control command INS for the in-vehicle devicerelated to the operation request Rm in consideration of the current control state of the vehicle function Fv related to the operation request Rm together with the operation request Rm.

10 10 10 10 As described above, according to the present embodiment, the vehicle function Fv can be operated by the driver performing an action, such as changing the orientation of the portable terminalor knocking the portable terminal. Therefore, the driver can operate the vehicle function Fv using the portable terminalwithout focusing on the screen of the portable terminal.

32 32 5 5 FIGS.B andC In addition, as described above, the operation determination unitspecifying the operation request Rm may include determining whether the operation request Rm1 directly related to the vehicle traveling is present, on a condition that the operation information Im is extracted with the screen touch. Further, the operation determination unitmay include determining whether the operation request Rm2 that is not directly related to the vehicle traveling is present, without the condition that the operation information Im is extracted with the screen touch (refer to). By using the screen touch information Ist in combination with the acceleration information Isa or the orientation information Isp in this way, it is possible to appropriately suppress the erroneous operation of the vehicle function Fv.

33 6 FIG. In addition, as described above, the function controllermay generate the control command INS in consideration of the vehicle state information Iv (the current control state of the vehicle function Fv) together with the operation request Rm (see). As a result, it is possible to appropriately generate the control command INS.