Portable terminal and vehicle control system

This application claims priority to Japanese Patent Application No. 2023-137213 filed on Aug. 25, 2023, incorporated herein by reference in its entirety.

The present disclosure relates to a technique of operating a vehicle using a portable terminal.

Japanese Patent No. 6986518 discloses a system that controls entry and exit of a vehicle into and from a parking lot by operating a portable terminal via wireless communication. In this system, an image to be displayed on a touch panel of the portable terminal is changed according to whether the orientation of the portable terminal is landscape or portrait.

In a system that operates a vehicle using a portable terminal, the manner of holding the portable terminal and the operation mode of the portable terminal with good operability may be different depending on the preference of an operator. Further, even for the same operator, the desired manner of holding the portable terminal and operation mode of the portable terminal may be different depending on the situation at the time of operating the vehicle, for example. Therefore, it is considered that the ability to appropriately change the manner of holding the portable terminal and the operation mode of the portable terminal according to the preference of the operator leads to improving the operability of the vehicle using the portable terminal.

The present disclosure has been made in view of the above issue, and has an object to provide a technique capable of improving the operability of a vehicle using a portable terminal.

An aspect of the present disclosure provides a portable terminal that is operated by an operator for operation of a vehicle and that performs wireless communication with the vehicle.

The portable terminal includes a sensor, a touch panel, and a processor.

The sensor detects an inclination direction and an inclination angle of the portable terminal.

The touch panel includes a display screen and detects a touch by the operator on the display screen.

The processor is configured to:

According to the present disclosure, it is possible to improve the operability of a vehicle using a portable terminal.

1. A Vehicle Control System Comprising:

is a schematic diagram illustrating an example of a configuration of a vehicle control systemaccording to an embodiment. The vehicle control systemincludes a portable terminal (or simply a terminal)and a vehicle. The terminalis operated by an operator for operation (remote driving) of the vehicle. The terminalperforms wireless communication with the vehicle.

The terminalincludes, for example, a touch panel, a communication device, a processor, a storage device, and sensors. The touch panelis formed on one plate surface of the terminaland includes a display screen and a touch sensor. The touch sensor is configured to be able to detect a touch of an operator on a display screen. The communication deviceis configured to be able to communicate with the vehiclevia the wireless communication network. The terminalis, for example, a smart phone or a tablet terminal, and is formed in a plate shape (for example, a rectangular plate shape) in which one side is a longitudinal direction and the other side is a lateral direction (for example, refer to). More specifically, for example, the touch panelhas a rectangular shape having a longitudinal direction and a lateral direction, and the longitudinal direction of the touch panelcoincides with the longitudinal direction of the terminal.

The processorexecutes various processes for operating the vehicleusing the terminal. The storage devicestores various kinds of information necessary for processing by the processor. More specifically, the processorexecutes various processes using various programs related to the operation of the vehicle. The various programs may be stored in the storage deviceor may be recorded in a computer-readable recording medium. The sensorsinclude, for example, an inclination angle sensor and a position sensor. The inclination angle sensor detects an inclination direction and an inclination angle (posture) of the terminal. The inclination angle sensor includes, for example, a six-axis gyro sensor. The tilt angle of the terminalis used for the operation of the vehiclesusing the “tilt operation A” described later. The position sensor includes a GNSS (Global Navigation Satellite System) receiver and detects a position and an orientation of the terminal.

The vehicleincludes a communication deviceand an in-vehicle electronic control unit (an in-vehicle ECU)and a traveling system. The communication devicecommunicates with the outside of the vehicle. For example, the communication devicecommunicates with the terminalvia the wireless communication network.

The in-vehicle ECUincludes a processor (in-vehicle processor) and a storage device. The in-vehicle ECUgenerates the “vehicle control amount C” according to the “travel control information Itc” generated by the terminal. As will be described later, the in-vehicle ECUincludes a first control amount calculation unit, a second control amount calculation unit, and a control amount arbitration unitas a functional configuration related to generation of the vehicle control amount C. The vehicle control amount C is output to the traveling system.

The traveling systemincludes a traveling device, sensors, and a ECU. The traveling deviceincludes a steering device, a driving device, and a braking device. The steering device includes an electric motor that steers wheels. The drive device includes one or both of an electric motor and an internal combustion engine for driving the vehicle. The braking device includes a brake actuator for braking the vehicle. The sensorsinclude a recognition sensor, a vehicle state sensor, and a position sensor. The recognition sensor is, for example, a camera, and recognizes a situation around the vehicle. The vehicle state sensor detects a state of the vehicle. The vehicle state sensor includes, for example, a speed sensor, an acceleration sensor, a yaw rate sensor, and a steering angle sensor. The position sensor detects a position and an azimuth of the vehicle. For example, the position sensor includes a GNSS receiver. ECUcontrols the traveling of the vehiclein accordance with the vehicle control amount C from the in-vehicle ECU.

Note that, unlike the embodiment illustrated in, the function of the in-vehicle ECUmay be provided in the terminal. That is, the terminalmay generate the vehicle control amount C after generating the travel control information Itc, and may output (transmit) the generated vehicle control amount C to the vehicle.

2. Vehicle Travel Control Using Portable Terminal

2-1. Holding the Terminal and Operating Method

are each a diagram for explaining the first and second holding methods and the first and second operating methods of a portable terminalshown in.

As shown in, the first holding method (or the vertical holding method) corresponds to the holding method in which one endof the lateral direction Dof the terminalis held by one hand Hof the operator. On the other hand, as shown in, the second holding method (or lateral holding) corresponds to a method of holding both endandof the terminalin the longitudinal direction Dwith both hands Hand H. In addition, in the present embodiment, from the viewpoint of fail-safe, the control of the travel (steering and speed) of the vehicleby the first and second operation methods is enabled on condition that the operator continuously touches the touch panel.

2-1-1. Vertical Hold and First Operating Method

First, a first operation method corresponding to the vertical holding uses a “tilt operation A” to control the steering and the velocity of the vehicles. Here, the tilt operation Arefers to an operation of tilting the terminalby an operator.

Specifically, in the first operation method, the steering of the vehiclesis controlled in accordance with a tilt operation Aabout a rotational axis parallel to the longitudinal direction D. The rotational axis is parallel to a center line L(see) extending through the center P of the terminaland in the longitudinal direction D. More specifically, according to the first operation method, the vehicleis steered to the right in accordance with a tilt operation Ain which the operator rotates the terminalalong the rotational direction RR with respect to a predetermined reference state. The reference state referred to here is, for example, a state in which the center line Lis level. More specifically, in the steering control, the target steering angle is set to be larger, for example, as the change in the tilt angle of the rotational direction RR is larger. The target steering angle corresponds to an example of the vehicle control amount C described later with reference to. Similarly, the vehicleis steered leftward in response to a tilt operating Ain which the operator rotates the terminalalong the rotational direction RL.

In the first operation method, the velocity of the vehicleis controlled in accordance with a tilt operation Aabout a rotational axis that is parallel to the lateral direction D. The rotational shaft is parallel to a center line L(see) that passes through the center P of the terminaland extends in the lateral direction D. Here, “speed control of the vehicle” includes control of acceleration and deceleration of the vehicle. In addition, the control of the deceleration includes control of the braking. More specifically, according to the first operation method, the speed (forward speed) of the vehicleis controlled in accordance with the tilt operation Ain which the operator rotates the terminalso that the endof the terminalfarther from the operator is lowered with respect to the predetermined reference state (that is, along the rotational direction RF). The reference state referred to here is, for example, a state in which the center line Lis level. More specifically, the larger the variation of the tilt angle of the rotational direction RF by the tilt operation Ais, the higher the target vehicle speed or the target acceleration is. The target vehicle speed and the target acceleration correspond to an example of the vehicle control amount C described later together with. In addition, the retraction rate of the vehiclesmay be controlled in response to a tilt operation Ain which the operator rotates the terminalsuch that the endincreases (i.e., along the rotational direction RR). Further, by additionally using the shift range data of the vehicles, the control of the backward speed may also be executed based on the tilt operation Afor rotating the terminalin the rotational direction RF.

During the selection of the first mode of operation, the processorgenerates a travel control information Itc based on the tilt operation Adescribed above. The travel control information Itc is information for controlling the travel of the vehiclesusing the terminal. The travel control information Itc generated during the selection of the first operating mode is sensor information based on the detected value of the tilt angle sensor. More specifically, the travel control information Itc is an inclination angle of each inclination direction (the rotational direction RR or RL, and the rotational direction RR or RL; see) and each inclination direction of the terminal.

2-1-2. Lateral Holding and Second Operation Method

Next, the second operation method corresponding to the lateral holding uses the “tilt operation A” for the steering control, but uses the “touch operation A” instead of the tilt operation Afor the vehicle speed control.

Specifically, in the second operation method, the steering of the vehiclesis controlled in accordance with a tilt operation Aabout a rotational axis that is parallel to the thickness-direction Dof the terminal. The rotating shaft is parallel to a center line L(see) that passes through the center P of the terminaland extends in the thickness-direction D. More specifically, according to the second operation method, the vehicleis steered to the right in accordance with a tilt operation Ain which the operator rotates the terminalalong the rotational direction RR (refer to) with respect to a predetermined reference state. Then, the vehicleis steered leftward in accordance with a tilt operation Ain which the operator rotates the terminalalong the rotational direction RL (refer to). Here, the reference state is, for example, a state in which the center line Lis parallel to the vertical plane.

In addition, the second operation method controls the velocity of the vehicles based on the touch operation A. The touch operation Acorresponds to an operation of touching (for example, tapping) a predetermined position on the display screen of the touch panel. The speed of the vehicleis controlled in accordance with the touch position. In, an image IM that is an exemplary image for displaying a touched position is shown. The image IM is displayed at a position where touching by a finger of one hand His accepted, for example. As an example, the image IM includes an image IMc of a circle mark and an image IMy of a plurality of arrows differing in size, together with image IMv indicating the orientation of the vehicles. The image IMc is touched when the vehiclesare stopped. The plurality of images IMy indicates that the target vehicle speed is higher as the arrow is larger. According to the second operating method, the operator can control the velocity of the vehiclesby changing the touched position during IM of images. For example, the operator can start the vehicleby changing the touch position from the image IMc to any of the image IMy. Conversely, by changing the touched position from any of the image IMy to the image IMc, the vehiclescan be decelerated (including braking) and stopped. In addition, the operator can accelerate the vehicleby changing the touch position in the image IMy indicating the larger arrow, and conversely, can decelerate the vehicleby changing the touch position in the image IMy indicating the smaller arrow. The speed control of the vehiclebased on the touch-operation Amay be used not only for the control of the forward speed but also for the control of the backward speed.

During the selection of the second mode of operation, the processorgenerates a travel control information Itc based on the tilt operation Aand the touch operation Adescribed above. The travel control information Itc generated during the selection of the second operating method is a combination of sensor information based on the detection value of the tilt angle sensor and sensor information based on the detection value of the touch sensor of the touch panel. More specifically, the travel control information Itc includes the tilt direction (rotational direction RR or RL; see) of the terminal, the tilt angle of the tilt direction, and the touch position of the touch panel.

2-2. Selecting the Operation Method

The processorof the terminalselects one of the first and second operating methods based on the orientation information Io indicating the orientation of the terminal. The orientation information Io is either “vertical (information indicating the orientation of the terminalwhen held vertically)” or “horizontal (information indicating the orientation of the terminalwhen held horizontally)”. The selection of such an operation method can be performed, for example, as in the following first selection example or second selection example.

2-2-1. First Selection Example

is a diagram illustrating an example of a screen display of the touch panelused in the first selection example of the operation method of the terminal.

In the embodiment illustrated in, two selection buttons Band Brelated to the operation method are displayed on the display screen together with the icons of the vehiclesto be operated. The selection button Bis a button for accepting selection of the first operating method (vertically held) by the operator. The selection button Bis a button for accepting selection of the second operating method (lateral holding) by the operator. The operator can select a desired operating method by touching the selection button Bor B. In addition, in this first selection, the “vertical” or “horizontal” corresponding to the selection button Bor Bselected by the operator corresponds to the orientation information Io.

is a flow chart illustrating an example of a process related to acquiring the orientation-information Io according to the first selection example. For example, the processing of this flowchart starts when the operator operates the terminalto display the screen display shown inon the touch panel, and ends when the operator turns off the screen display. In addition, the terminalmay be configured such that the operator can display the screen display illustrated inonly while the vehicle is stopped. That is, the terminal(the processor) may accept switching (updating) of the direction (i.e., the vertical holding or the horizontal holding) of the terminalby the operator on condition that the vehicleis stopped.

At the beginning of the process of the present flow chart, an initial value is set as the orientation information Io. The initial value is, for example, a predetermined “vertical” or “horizontal”. Alternatively, the initial value may be, for example, “vertical” or “horizontal” corresponding to the operation method used in the previous vehicle operation.

In S, the processorof the terminalwaits for a user's touch (e.g., tapping) (standby). Next, in S, the processordetermines whether or not a touch on the touch panelby the operator has been detected. Consequently, if the operator's touch is not detected (S; No), the process returns to S. On the other hand, if the operator touches (S; Yes), the process proceeds to S.

In S, the processordetermines whether or not the touched position by the operator is the vertical holding position (selection button B). Consequently, if the vertical holding position is touched (S; Yes), the processorproceeds to Sand updates the orientation information Io to “vertical”. That is, the processorselects the first operation method corresponding to the vertical holding. The updated orientation information Io is stored in the storage device. Thereafter, the process returns to S. In addition, in accordance with the selection of the first operation method, the processorperforms display related to the first operation method using the touch panel. The display includes, for example, a display indicating that the current operation method is the first operation method, and a display of vehicle information received from the vehicle.

On the other hand, when the touched position is not the vertically held position (S; No), the process proceeds to S. In S, the processordetermines whether or not the touched position is the horizontal holding position (selection-button B). Consequently, if the horizontal holding position is touched (S; Yes), the processorproceeds to Sand updates the orientation information Io to “lateral”. That is, the processorselects the second operation method corresponding to the lateral holding. The updated orientation information Io is stored in the storage device. Thereafter, the process returns to S. In addition, in accordance with the selection of the second operation method, the processorperforms display related to the second operation method using the touch panel. The display includes, for example, a display indicating that the present operation method is the second operation method, a display of the vehicle information received from the vehicle, and a display of the image IM shown in.

When the touch position is neither the vertically held position nor the horizontal holding position (S;No), that is, when the orientation information Io corresponding to the latest touch position indicates an abnormal value, the process returns to Swithout updating the orientation information Io.

2-2-2. Second Selection Example

The second selection example is different from the first selection example in the method of acquiring (updating) the orientation information Io. In the second selection example, the processorupdates the orientation information Io in the held direction when the terminalis held in the fixed direction for a predetermined time or longer under the condition that the operator is not touching the touch panel. That is, in the second selection example, the orientation information Io is automatically updated by the processorwithout accepting the selection of the holding method by touching by the operator.

is a flow chart illustrating an example of a process related to acquiring the orientation information Io according to the second example of selection. For example, the processing of this flowchart is repeatedly executed during a period in which the operator operates the terminalfor the operation of the vehicle. Note that the processing of the flowchart illustrated inand the processing of the present flowchart may be executed in parallel. That is, the first selection example and the second selection example may be used in combination for acquiring (updating) the orientation information Io and selecting the operating method.

In S, the processordetermines whether or not a touch on the touch panelby the operator is detected. Consequently, if the operator touches (S; Yes), the process proceeds to S.

In S, the processorholds the present orientation information Io stored in the storage device. That is, the processorholds the present orientation information Io on condition that the touch on the touch panelis continued. As a result, the 1 or 2 operation method currently selected is held

On the other hand, if the operator is not detected (S;No), the process proceeds to S. In S, the processordetermines whether the orientation of the terminaldetected by the tilt angle sensor differs from the present orientation information Io. Here, the “detected direction” is one of “vertical” and “horizontal” described above. Whether the direction is “vertical” or “horizontal” can be determined based on, for example, the inclination angle of the respective rotational directions of the terminal(refer to).