Systems and methods use cameras to provide autonomous navigation features. In one implementation, top-down refinement in lane marking navigation is provided. The system may include one or more memories storing instructions and one or more processors configured to execute the instructions to cause the system to receive from one or more cameras one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line, update a model of the lane marking based on odometry of the one or more cameras relative to the roadway, refine the updated model of the lane marking based on an appearance of dashes derived from the received one or more images and a spacing between dashes derived from the received one or more images, and cause one or more navigational responses in the vehicle based on the refinement of the updated model.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A computer system comprising: one or more processors configured to execute instructions to cause the system to: receive wirelessly, via a data interface, a plurality of images captured by an image acquisition device comprising at least three cameras, each of the least three cameras being configured to capture a distinct field of view, the plurality of images being of a roadway in a vicinity of a remote vehicle; analyze the plurality of images to detect one or more segments of lane markings by; deriving information from images captured by each of the at least three cameras, determining whether derived information from different cameras of the at least three cameras is consistent, and excluding the derived information from a selected camera, of the at least three cameras when determining the derived information from the selected camera is consistent with derived information from another camera; generate a model based on the analysis of the plurality of images, the model representing the one or more detected segments refine the model by performing a top-down modeling process based on: an appearance of the one or more segments, the appearance comprising at least one of dimensions, texture, sharpness, or brightness of the one or more segments, and a spacing calculated between segments derived from the plurality of images; and transmit wirelessly the refined model to at least one vehicle for navigating the roadway based on refined model.
2. The system of claim 1 , wherein: the data interface comprises a wireless link for receiving image data from the image acquisition device; and the one or more processors and the vehicle are in different locations.
3. The system of claim 1 , wherein the plurality of images comprise a first set of images and a second set of images, the first set of images being received before the second set of images.
4. The system of claim 1 wherein to generate the model, the instructions cause the system to: identify locations of the line markings at a first time; and update the model based on odometry from the image acquisition device, the model comprising estimated updated locations of the line markings at a second time.
5. The system of claim 4 , wherein: the model comprises a multi-frame model; and the top-down modeling process comprises refining the multi-frame model based on the appearance of the one or more segments.
6. The system of claim 4 , wherein to generate the model, the instructions cause the system to: identify a plurality of candidate dashes from the plurality of images; and calculate spacing information between the plurality of candidate dashes.
7. The system of claim 1 , wherein the instructions further cause the system to: generate navigational instructions based on the refined model; and transmit wirelessly the navigational instructions to navigation systems positioned on a plurality of vehicles.
8. The system of claim 7 , wherein the navigation instructions comprise: instructions to generate control signals for a throttling system; instructions to generate control signals for a braking system; or instructions to generate control signals for a steering system.
9. The system of claim 1 , wherein to analyze the plurality of images, the instructions cause the system to: execute a stereo image analysis module; and combine the plurality of images using the stereo image analysis module.
10. The system of claim 1 , wherein to analyze the plurality of images, the instructions cause the system to execute a monocular image analysis module.
11. The system of claim 1 , wherein to analyze the plurality of images, the instructions cause the system to: detect features in the plurality of images, the features comprising at least one of vehicles, pedestrians, road signs, and highway exit ramps.
12. A computer implemented method comprising: receiving wirelessly, via a data interface, a plurality of images captured by an image acquisition device wherein the image acquisition device comprises at least three cameras, each of the least three cameras being configured to capture a distinct field of view, the plurality of images being of a roadway in a vicinity of a remote vehicle; analyzing the plurality of images to detect one or more segments of lane markings by: deriving information from images captured by each of the at least three cameras, determining whether derived information from different cameras of the at least three cameras is consistent, and excluding the derived information from a selected camera of the at least three cameras when determining the derived information from the selected camera is consistent with derived information from another camera; generating a model based on the analysis of the plurality of images, the model. representing the one or more detected segments, p‘refining the model by performing a top-down modeling process based on: an appearance of the one or more segments, the appearance comprising at least one of dimensions, texture, sharpness, or brightness of the one or more segments, and a spacing calculated between segments derived from the plurality of images; and transmitting wirelessly the refined model to at least one vehicle for navigating the roadway based on the refined model.
13. The method of claim 12 , wherein: the data interface comprises a wireless link for receiving image data from the image acquisition device; and the one or more processors and the vehicle are in different locations.
14. The method of claim 12 , further comprising: generating navigational instructions based on the refined model; and transmitting wirelessly the navigational instructions to navigation systems positioned on a plurality of vehicles.
15. The method of claim 14 , wherein: generating the model comprises: updating the model based on odometry from the image acquisition device, the updated model comprising estimated updated locations of the line markings at a second time, identifying a plurality of candidate dashed from the plurality of images, and calculating spacing information between the plurality of candidate dashes; the model comprises a multi-frame model; and the top-down modeling process comprises refining the multi-frame model based on the appearance of the one or more segments.
16. The method of claim 12 , wherein analyzing the plurality of images comprises: executing a stereo image analysis module; and combining the plurality of images using the stereo image analysis module.
17. A non-transitory computer-readable medium storing instructions that, when executed by a processor, perform operations comprising: receiving wirelessly, via a data interface, a plurality of images captured by an image acquisition device wherein the image acquisition device comprises at east three cameras, each of the east three cameras being configured to capture a distinct field of view, the plurality of images being of a roadway in a vicinity of a remote vehicle; analyzing the plurality of images to detect one or more segments of lane markings by: deriving information from images captured by each of the at least three cameras, determining whether derived information from different cameras of the at least three cameras is consistent, and excluding the derived information from a selected camera of the at least three cameras when determining the derived information from the selected camera is consistent with derived information from another camera; generating a model based on the analysis of the plurality of images, the model representing the one or more detected segments; refining the model by performing a top-down modeling process based on: an appearance of the one or more segments, the appearance comprising at least one of dimensions, texture, sharpness, or brightness of the one or more segments, and a spacing calculated between segments derived from the plurality of images; and transmitting wirelessly the refined model to at least one vehicle for navigating the roadway based on the refined model.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
June 10, 2019
August 25, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.