Tracking system and tracking method

The disclosure relates to a tracking system; particularly, the disclosure relates to a tracking system and a tracking method.

In order to bring an immersive experience to users, various technologies, such as augmented reality (AR) and virtual reality (VR), are constantly being developed. AR technology allows users to bring virtual elements to the real-world. VR technology allows users to enter a whole new virtual world to experience a different life. Wearable devices are often used to provide this kind of immersive experience.

In addition, to provide an immersive in-vehicle user experience, a wearable device is further integrated with a vehicle. For the purpose of the integration, the wearable device may obtain information from the vehicle by communicating with the advanced driver assistance system (ADAS). That is, the wearable device may obtain either engine status from the engine of the vehicle or may obtain tracking information from the tracking system of the vehicle. However, the setup of the integration is usually complicated and not user-friendly. Further, the tracking system of the vehicle might be expensive.

The disclosure is direct to a tracking system and a tracking method, so as to track a position of a wearable device inside a vehicle.

In this disclosure, a tracking system is provided. The tracking system includes a first tracking device, a second tracking device, and a wearable tracking device. The first tracking device is disposed on a vehicle and is configured to obtain map information and first measurement information. The second tracking device is disposed on the vehicle and is configured to obtain second measurement information. The wearable tracking device is disposed on a user in the vehicle and is configured to obtain third measurement information. Further, the wearable tracking device is configured to obtain location position information of the user based on the map information, the first measurement information, the second measurement information, and the third measurement information. Furthermore, the local position information indicates a user position of the user within the vehicle.

In this disclosure, a tracking method is provided. The tracking method includes: obtaining, through a first tracking device disposed on a vehicle, map information and first measurement information; obtaining, through a second tracking device disposed on the vehicle, second measurement information; obtaining, through a wearable tracking device disposed on a user in the vehicle, third measurement information; and obtaining local position information of the user based on the map information, the first measurement information, the second measurement information, and the third measurement information, wherein the local position information indicates a user position of the user within the vehicle.

Based on the above, according to the tracking system and the tracking method, a user friendly and low cost integration system of the wearable device in the vehicle is achieved.

To make the aforementioned more comprehensible, several embodiments accompanied with drawings are described in detail as follows.

Reference will now be made in detail to the exemplary embodiments of the disclosure, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numbers are used in the drawings and the description to refer to the same or like components.

Certain terms are used throughout the specification and appended claims of the disclosure to refer to specific components. Those skilled in the art should understand that electronic device manufacturers may refer to the same components by different names. This article does not intend to distinguish those components with the same function but different names. In the following description and rights request, the words such as “comprise” and “include” are open-ended terms, and should be explained as “including but not limited to . . . ”.

The term “coupling (or connection)” used throughout the whole specification of the present application (including the appended claims) may refer to any direct or indirect connection means. For example, if the text describes that a first device is coupled (or connected) to a second device, it should be interpreted that the first device may be directly connected to the second device, or the first device may be indirectly connected through other devices or certain connection means to be connected to the second device. The terms “first”, “second”, and similar terms mentioned throughout the whole specification of the present application (including the appended claims) are merely used to name discrete elements or to differentiate among different embodiments or ranges. Therefore, the terms should not be regarded as limiting an upper limit or a lower limit of the quantity of the elements and should not be used to limit the arrangement sequence of elements. In addition, wherever possible, elements/components/steps using the same reference numerals in the drawings and the embodiments represent the same or similar parts. Reference may be mutually made to related descriptions of elements/components/steps using the same reference numerals or using the same terms in different embodiments.

It should be noted that in the following embodiments, the technical features of several different embodiments may be replaced, recombined, and mixed without departing from the spirit of the disclosure to complete other embodiments. As long as the features of each embodiment do not violate the spirit of the disclosure or conflict with each other, they may be mixed and used together arbitrarily.

In order to bring an immersive experience to users, various technologies, such as augmented reality (AR) and virtual reality (VR), are constantly being developed. AR technology allows users to bring virtual elements to the real-world. VR technology allows users to enter a whole new virtual world to experience a different life. Wearable devices are often used to provide this kind of immersive experience.

In addition, to provide an immersive in-vehicle user experience, a wearable device is further integrated with a vehicle. It is noted that, while the vehicle is moving, the sensor of the wearable device in the vehicle may also generates sensing values that indicates the wearable device is moving. However, the wearable device may stay still while the sensing values are generated. Therefore, there is a need to integrating the wearable device with the vehicle to prevent the false detection.

For the purpose of the integration, the wearable device may obtain information from the vehicle by communicating with the advanced driver assistance system (ADAS). That is, the wearable device may obtain either engine status from the engine of the vehicle or may obtain tracking information from the tracking system of the vehicle. However, the setup of the integration is usually complicated and not user-friendly. Further, the tracking system of the vehicle might be expensive. Hence, how to develop a user friendly and low cost integration system of the wearable device in the vehicle is becoming an issue to work on.

is a schematic diagram of a tracking system according to an embodiment of the disclosure. With reference to, a tracking systemincludes a first tracking device, a second tracking device, and a wearable tracking device. The first tracking device is disposed on a vehicle (not shown) and is configured to obtain map information and first measurement information M1. The second tracking deviceis disposed on the vehicle and is configured to obtain second measurement information M2. The wearable tracking deviceis disposed on a user (not shown) in the vehicle and is configured to obtain third measurement information. Further, the wearable tracking deviceis configured to obtain location position information LP of the user based on the map information, the first measurement information M1, the second measurement information M2, and the third measurement information. Furthermore, the local position information LP indicates a user position of the user within the vehicle.

It is noted that, the first tracking devicemay be mounted rigidly on a fixed position of the vehicle that stays still while the vehicle is moving. For example, the first tracking devicemay be mounted on the bonnet of the vehicle, the glovebox of the vehicle, or the windscreen of the vehicle, but this disclosure is not limited thereto. Further, the second tracking devicemay be mounted rigidly on a fixed position within the vehicle that stays still while the vehicle is moving. In addition, the second tracking deviceon the fixed position is able to be detected by the wearable tracking device. Furthermore, the wearable tracking deviceis disposed on the user in the vehicle.

In one embodiment, the vehicle includes, for example, a car, a bus, a bumper car, an amusement facility, a full flight simulator, or other similar objects capable of carrying people. This disclosure is not limited thereto.

In one embodiment, the tracking systemmay include a controller. The controller may be disposed in the first tracking device, the second tracking device, the wearable tracking device, or other devices according to design need. While it is assumed in the following embodiments for the sake of convenience in explanation that the controller is disposed in the wearable tracking device, it is to be noted that the controller is not limited to being disposed in the wearable tracking device.

In one embodiment, the controller includes, for example, a microcontroller unit (MCU), a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a programmable controller, a programmable logic device (PLD), other similar devices, or a combination of the devices. The disclosure is not limited thereto. In addition, in an embodiment, each of functions of the controller may be achieved as multiple program codes. The program codes are stored in a memory, and executed by the controller. Alternatively, in an embodiment, each of the functions of the controller may be achieved as one or more circuits. The disclosure does not limit the use of software or hardware to achieve the functions of the controller.

In one embodiment, each of the first tracking device, the second tracking device, and the wearable tracking devicemay include a sensor. In one embodiment, the sensor includes, for example, an inertial measurement unit (IMU) sensor, an accelerometer, a gyroscope, other similar devices, or a combination of these devices. This disclosure is not limited thereto. In one embodiment, each of the first measurement information M1, the second measurement information M2, and the third measurement information may include an inertial measurement value of an IMU sensor. Each of the inertial measurement value may include changes in six degrees of freedom (DOF) and the six degrees of freedom comprises three translation values corresponding to three perpendicular axes and three rotation values corresponding to the three perpendicular axes. For example, the inertial measurement value includes three linear accelerations and three angular velocities. However, this disclosure is not limited thereto.

In one embodiment, the second tracking deviceincludes, for example, a tracker, a light emitting device (LED), or other similar devices capable of providing a tracker pattern for the wearable tracking deviceto detect/recognize. For example, a plurality of LEDs of the second tracking devicemay be arranged in a specific pattern. However, this disclosure is not limited thereto. In one embodiment, the tracker pattern includes, for example, at least one of a predetermined pattern, an ava group of the university of Cordoba (ArUco) marker, and a light emitting device, but this disclosure is not limited thereto.

In one embodiment, the wearable tracking deviceincludes, for example, a wearable head-mounted display (HMD), wearable glasses (e.g., AR/VR goggles), a graspable device (e.g., handheld device), an electronic device, other similar devices, or a combination of these devices. However, the disclosure is not limited thereto.

In this manner, the tracking systemis able to determine the position of the user in the vehicle while the vehicle is moving. Therefore, the convenience of integration of the wearable tracking devicein the vehicle and the cost of integration of the wearable tracking devicein the vehicle is reduced.

is a schematic diagram of a vehicle positioning scenario according to an embodiment of the disclosure. With reference toand, in a vehicle positioning scenariothe first tracking devicemay include a location sensor.

In one embodiment, the map information of the first tracking devicemay include a simultaneous localization and mapping (SLAM) map and the SLAM map may be obtained based on the location sensor. Further, the wearable tracking deviceis configured to obtain global position information GP of the vehicle based on the SLAM map. It is worth mentioned that, the global position information GP of the vehicle indicates a location of the vehicle in the world.

In one embodiment, the location sensorincludes, for example, a camera, a light detection and ranging (LiDAR) device, a global positioning system (GPS) device, a radar, infrared sensor, an ultrasonic sensor, other similar devices, or a combination of these devices. The disclosure is not limited thereto. That is, the SLAM map may be obtained through a camera, a LiDAR device, a GPS device, a radar, infrared sensor, an ultrasonic sensor, other similar devices, or a combination of these devices. The disclosure is not limited thereto.

In one embodiment, the first tracking devicemay include an exterior camera and the exterior camera is configured to obtain a plurality of exterior images outside the vehicle. Based on the plurality of exterior images, the SLAM map may be obtained. That is, a location of the vehicle may be obtained. Further, a movement of the vehicle may be also obtained based on the difference between the plurality of exterior images at a previous moment and the plurality of exterior images at a present moment.

In this manner, based on the location sensorof the first tracking device, the location and the movement of the vehicle in the world may be obtained.

is a schematic diagram of a calibration scenario according to an embodiment of the disclosure.is a schematic diagram of a tracking scenario according to an embodiment of the disclosure. With reference toto,depicts a calculation process of a tracker-vehicle pose relationship R_TV between the second tracking deviceand the vehicle (the first tracking device) anddepicts a tracking process of a user-tracker pose relationship R_UT between the user (the wearable tracking device) and the second tracking device, wherein the pose relationship comprise a relative translation (position) relationship and a relative rotation relationship, but this disclosure is not limited thereto.

Referring to, in a calibration scenarioA, the first measurement information M1 and the second measurement information M2 are provided to a tracker-vehicle calibrationto obtain the tracker-vehicle pose relationship R_TV. The tracker-vehicle calibrationmay be performed by the controller of the wearable tracking device, but this disclosure is not limited thereto. It is noted that, it is described in the embodiment for the sake of convenience in explanation that the second tracking devicemay be a tracker, but this disclosure is not limited thereto.

In one embodiment, the tracker-vehicle pose relationship R_TV may indicate a relationship between a rotation value of the first tracking deviceand a rotation value of the second tracking device. That is, with the rotation value of the first tracking deviceand the rotation value of the second tracking device, the tracker-vehicle calibrationis configured to calibrate the relative rotation relationship from the first tracking deviceto the second tracking device.

Specifically, the first tracking deviceis configured to obtain a first rotation value of a first fixed position of the vehicle and the second tracking deviceis configured to obtain a second rotation value of a second fixed position within the vehicle. The first rotation value of the first tracking devicemay be represented byω and the second rotation value of the second tracking devicemay be represented byω, wherein ω stands for a rotation value, V stands for the vehicle (the first tracking device), and T stands for the tracker (the second tracking device). The first rotation valueω and the second rotation valueω may satisfy a rotation transformation equation

wherein

stands for the relative rotation relationship from the tracker to the vehicle. That is,

also stands for the tracker-vehicle rotation relationship in the pose relationship R_TV. Hence, the tracker-vehicle rotation relationship

which is part of the tracker-vehicle pose relationship R_TV may be obtained based on the first rotation valueω (the first measurement information M1) and the second rotation valueω (the second measurement information M2).

In one embodiment, the tracker-vehicle pose relationship R_TV may indicate a relationship between a translation value of the first tracking deviceand a translation value of the second tracking device. That is, with the translation value of the first tracking deviceand the translation value of the second tracking device, the tracker-vehicle calibrationis configured to calibrate the relative translation relationship from the first tracking deviceto the second tracking device.

Specifically, the first tracking deviceis configured to obtain a first acceleration value of the first fixed position of the vehicle and the second tracking deviceis configured to obtain a second acceleration value of the second fixed position within the vehicle. In detail, a first translation value of the first tracking devicein a global coordinate may be represented byDand a second translation value of the second tracking devicein the global coordinate may be represented byD, wherein G stands for the global coordinate, D stands for a translation value, V stands for the vehicle (the first tracking device), and T stands for the tracker (the second tracking device). The first translation valueDand the second translation valueDmay satisfy a translation transformation equation

wherein

stands for the relative translation relationship from the vehicle V to the tracker T, and

stands for the relative rotation relationship from the global coordinate G to the vehicle V. That is,

also stands for the tracker-vehicle translation relationship in the pose relationship R_TV.

Further, the translation transformation equation may be differentiated twice to obtain an acceleration transformation equation

whereinastands for a first acceleration value of the vehicle V under the global coordinate G,astands for a second acceleration value of the tracker T under the global coordinate G, a stands for an acceleration value, [ω]stands for representing a 3D vector (e.g., a rotation value) in a form of outer product and {dot over (ω)} stands for a first differential form of a 3D vector (e.g., a rotation value). Notably, the second acceleration value under the global coordinate G (a) may be also obtained based on the second acceleration value under a coordinate of the tracker T (a), and satisfy a conversion equation

wherein g stands for the acceleration of gravity. Therefore, the second acceleration value (of the tracker T) (a) may further satisfy an equation

In this manner, the relative translation relationship

between the first translation value of the first tracking deviceand the second translation value of the second tracking devicemay be obtained based on the relative rotation relationship

Hence, the tracker-vehicle translation relationship

which is part of the tracker-vehicle pose relationship R_TV, may be obtained based on the first acceleration valuea, the first rotation value ω(ω) (the first measurement information M1), and the second acceleration valuea(the second measurement information M2).