Systems and methods for driving. A driving data set for each of plurality of human-driven vehicles is determined. For each driving data set, exterior scene features of an exterior scene of the respective vehicle are extracted from the exterior image data. A driving response model is trained based on the exterior scene features and the vehicle control inputs from the selected driving data sets.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method, comprising: obtaining driving data sets for respective vehicles, each of the driving data sets comprising synchronized interior image data and exterior image data, wherein the synchronized interior image data and exterior image data are associated with vehicle control inputs; for each of the driving data sets, determining a driving quality metric that indicates a driving quality; selecting one of the driving data sets, having the driving quality metric satisfying a condition; and providing the exterior image data from the selected one of the driving data sets for training a driving response model based on the exterior image data from the selected one of the driving data sets.
2. The method of claim 1 , further comprising: for each of the driving data sets: determining driver attention based on the interior image data of the corresponding driving data set; and determining a region of interest (ROI) based on driver attention.
3. The method of claim 1 , wherein the driving response model is trained based on an exterior scene feature extracted from the exterior image data of the selected one of the driving data sets.
4. The method of claim 1 , wherein one of the driving quality metrics is determined based on the interior image data of the corresponding driving data set.
5. The method of claim 1 , wherein one of the driving quality metrics comprises an attentiveness score, determined based on driver gaze, wherein the driver gaze is extracted from the interior image data of the corresponding driving data set.
6. The method of claim 5 , wherein the attentiveness score is determined based on the driver gaze and a region of interest identified in the exterior image data of the corresponding driving data set.
7. The method of claim 1 , wherein one of the driving quality metrics comprises a driver score.
8. The method of claim 1 , wherein one of the driving quality metrics is determined based on a comparison of driver behavior, indicated by the corresponding driving data set, with an expected driving behavior.
9. The method of claim 1 , wherein one of the driving quality metrics comprises a ride comfort metric.
10. The method of claim 1 , wherein at least one of the driving data sets is obtained by: detecting a predetermined driving event; and recording the synchronized interior image data and exterior image data for the at least one of the driving data sets after detecting the predetermined driving event.
11. The method of claim 1 , wherein the synchronized interior image data and exterior image data of the driving data sets are obtained by a remote computing system, and wherein the act of determining the driving quality metrics, the act of selecting the one of the driving data sets, and the act of providing the exterior image data from the selected one of the driving data sets for training the driving response model, are performed by the remote computing system.
12. The method of claim 1 , wherein the driving response model is trained for different driving events using different respective ones of the driving data sets.
13. The method of claim 1 , further comprising extracting exterior scene feature from the exterior image data of one of the driving data sets by: determining a point cloud based on the exterior image data of the one of the driving data sets; and extracting the exterior scene feature from the point cloud.
14. The method of claim 1 , wherein the act of obtaining driving data sets and the act of determining driving quality metrics for the respective driving data sets are performed by a vehicle onboard system.
15. The method of claim 1 , wherein the driving response model is trained based on the exterior image data of the selected one of the driving data sets, and also based on the vehicle control inputs associated with the selected one of the driving data sets.
16. The method of claim 1 , wherein the driving response model is trained based on the exterior image data of the selected one of the driving data sets, and also based on driver behavior that is determined based on the interior image data of the selected one of the driving data sets.
17. A system, comprising: an interior camera configured to generate interior image data of a vehicle; an exterior camera configured to generate exterior image data of the vehicle; and a processing system configured to: obtain a driving data set for a driving session that includes the interior image data and the exterior image data provided by the interior camera and the exterior camera, respectively, wherein the interior image data and the exterior image data are synchronized with each other and are associated with vehicle control inputs; determine a driving quality metric for the driving data set, the driving quality metric indicating a driving quality; and providing the driving data set for training a driving response model, wherein the driving response model is based on an external scene feature extracted from the exterior image data and a driver behavior extracted from the interior image data.
18. The system of claim 17 , further comprising: a scanning model configured to determine a region of interest (ROI) for the exterior image data; and a gaze detector configured to extract an eye gaze of a driver from the interior image data; wherein the system is configured to use the ROI and the eye gaze to determine the driving quality metric for the driving data set.
19. The system of claim 17 , further comprising: the driving response model, wherein the system is configured to generate a notification when a driving behavior differs from an expected driving behavior.
20. The system of claim 17 , wherein the driving response model is also based on the vehicle control inputs.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
February 27, 2019
July 19, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.