To provide a road marking image processing device which performs highly accurate road marking recognition by creating a high-quality synthesized bird's-eye image that has no influence from the camera parameter error and noise such as the reflection light from the road surface, lens extraneous matter, shadow of the own vehicle, and the like. A new synthesized range identifying module creates a bird's-eye image of only an area not contained in a synthesized bird's-eye image created based on the road images up to the one captured previously from the current road image. An image synthesizing module synthesizes the bird's-eye image and the synthesized bird's-eye image created based on the road images up to the one captured previously to create a new synthesized bird's-eye image.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A road marking image processing method for recognizing a road marking within a captured image of a road surface, comprising: capturing a road image of a rear side of a traveling path from a vehicle; measuring a movement amount of the vehicle; identifying a road surface range corresponding to the movement amount as a new synthesized range from the road image, and creating a bird's-eye image for the new synthesized range; and shifting position of a synthesized bird's-eye image that is being stored by length corresponding to the movement amount, synthesizing the synthesized bird's-eye image to the bird's-eye image created for the new synthesized range to create a new synthesized bird's-eye image, and outputting the created new synthesized bird's-eye image.
A method for recognizing road markings in images captured from a moving vehicle. A camera on the vehicle captures images of the road behind the vehicle. The method measures how much the vehicle has moved. It identifies the portion of the current road image that corresponds to this movement as a "new synthesized range". A bird's-eye view image is created for this new range. A stored bird's-eye view image (representing a compilation of previous road images) is shifted to align with the new movement. The new bird's-eye view image is combined with the shifted, stored bird's-eye view image to create an updated, synthesized bird's-eye view image, which is then output.
2. The road marking image processing method as claimed in claim 1 , comprising: judging presence of a noise within the road image based on presence of a change in the image, and specifying a noise region when judged that there is a noise; and identifying the road surface range by avoiding the specified noise region when identifying the road surface range.
The road marking image processing method of the previous description also includes noise reduction. It checks the captured road image for noise by detecting changes within the image. If noise is detected, the method identifies the noisy region. When identifying the road surface range corresponding to the vehicle's movement, the noisy region is avoided. This ensures that the bird's-eye view image is created without including the noise, resulting in a cleaner image for road marking recognition.
3. The road marking image processing method as claimed in claim 2 , comprising identifying the road surface range within a closed region that is set in advance within an image region of the road image when identifying the road surface range.
The road marking image processing method of the previous descriptions includes further refinement of the road surface identification. When identifying the road surface range (even when avoiding noise), the method only considers a pre-defined, closed region within the road image. This pre-defined region limits the area that is analyzed, focusing on the most relevant part of the image and improving processing speed and accuracy.
4. The road marking image processing method as claimed in claim 1 , comprising recognizing a road marking within the synthesized bird's-eye image.
The road marking image processing method described previously includes a step to recognize road markings. This step is performed on the synthesized bird's-eye view image (the combined image created from previous and current road images). This allows the system to identify road markings like lane lines and symbols using the clear, accumulated view of the road surface.
5. A non-transitory computer readable recording medium storing a road marking image processing program for recognizing a road marking within a captured image of a road surface, which causes a computer to execute: a function which acquires a road image from an imaging module which is loaded on a vehicle to capture the road image of a rear side of a traveling path; a function which acquires a movement amount value of the vehicle; a function which identifies a road surface range corresponding to the movement amount as a new synthesized range from the road image, and creates a bird's-eye image for the new synthesized range; and a function which shifts position of a synthesized bird's-eye image that is being stored by length corresponding to the movement amount, and synthesizes the synthesized bird's-eye image to the bird's-eye image created for the new synthesized range to create a new synthesized bird's-eye image.
A computer program stored on a non-transitory medium (like a hard drive or flash drive) is used to recognize road markings in images. When executed, the program takes road images from a camera on the back of a moving car, and gets data of how far the car has travelled. It identifies part of the current road image corresponding to the vehicle's movement as a "new synthesized range" and makes a bird's-eye view image of it. It then shifts the stored bird's-eye view image to align with the new movement, combines this with the new range image, and creates an updated synthesized bird's-eye view image.
6. The non-transitory computer readable recording medium storing the road marking image processing program as claimed in claim 5 , which causes the computer to execute: a function which judges presence of a noise within the road image based on presence of a change in the image, and specifies a noise region when judged that there is a noise; and a function which identifies the road surface range by avoiding the specified noise region.
The road marking image processing program from the previous description also has noise reduction capabilities. It checks the captured road image for noise by detecting changes in the image. If noise is present, the program identifies the noisy region. The program avoids this noise region when identifying the road surface range.
7. The non-transitory computer readable recording medium storing the road marking image processing program as claimed in claim 6 , which causes the computer to execute a function which identifies the road surface range within a closed region that is set in advance within an image region of the road image.
The road marking image processing program described in the previous two claims further refines road surface identification. When identifying the road surface range (even while avoiding noise), the program only analyzes a pre-defined, closed region within the road image, thus improving processing speed and accuracy.
8. The non-transitory computer readable recording medium storing the road marking image processing program as claimed in claim 5 , which causes the computer to execute a function which recognizes a road marking within the synthesized bird's-eye image.
The road marking image processing program from previous claims includes a function to identify road markings. This identification is done using the combined synthesized bird's-eye view image.
9. A road marking image processing device which recognizes a road marking within a captured image of a road surface, comprising: an imaging device configured for placement on a vehicle and configured to capture a road image of a rear side of a traveling path of the vehicle; a sensor configured to measure a movement amount of the vehicle along the traveling path; a memory configured to store a synthesized bird's-eye image; and a processor configured to perform operations of: identifying a road surface range corresponding to the movement amount of the vehicle as a new synthesized range from the road image; creating a bird's-eye image for the new synthesized range; shifting a position of the synthesized bird's-eye image by a length corresponding to the movement amount of the vehicle; synthesizing the synthesized bird's-eye image to the bird's-eye image created for the new synthesized range to create a new synthesized bird's-eye image, and outputting the created new synthesized bird's-eye image; recognizing a road marking based on the new synthesized bird's-eye image; and outputting a result of the recognized road marking based on the new synthesized bird's-eye image.
A road marking image processing device that recognizes road markings from a vehicle. It has a camera to capture road images from the rear side, and a sensor to measure vehicle movement. The device includes memory to store a synthesized bird's-eye view image. A processor performs the following steps: identifies the area corresponding to vehicle movement in the current image, creates a bird's-eye view image of that area, shifts the stored bird's-eye view image to match vehicle movement, combines both bird's-eye images into a new one, and outputs the new bird's-eye image. This new image is used to recognize road markings, and the result is output.
10. The road marking image processing device as claimed in claim 9 , comprising a noise region specifying module implemented by the processor which judges presence of a noise within the road image based on presence of a change in the image, and specifies a noise region when judged that there is a noise, wherein the road surface range is identified by avoiding the noise region that is specified by the noise region specifying module.
The road marking image processing device of the previous description has a noise reduction system. The processor analyzes the road image for noise, detecting changes within the image. If noise is detected, the processor marks the noisy region and avoids that region when identifying the road surface range.
11. The road marking image processing device as claimed in claim 10 , wherein the noise region specifying module judges an image region with no luminance change within the road image as having a noise that is generated due to an extraneous matter attached to a lens of the imaging module.
The road marking image processing device of the previous description further specifies that the noise detection system identifies areas of the road image with little to no change in brightness as noise caused by dirt on the camera lens.
12. The road marking image processing device as claimed in claim 10 , wherein the noise region specifying module judges an image region whose positional change is smaller than other regions in a circular high-luminance region within the road image as a noise that is generated due to reflected light from the road surface captured into the image.
The road marking image processing device of the previous description specifies that the noise detection system recognizes reflected light as noise. If a bright, circular area in the image shifts less than other areas, it's identified as reflected light.
13. The road marking image processing device as claimed in claim 10 wherein the noise region specifying module judges an image region whose positional change is smaller than other regions in a low-luminance region located on the vehicle side within the road image as a noise that is generated due to a shadow of the vehicle captured into the image.
The road marking image processing device of the previous description specifies that the noise detection system recognizes the vehicle's shadow as noise. Low-brightness areas on the vehicle's side of the image that shift less than other areas are considered shadows and are excluded.
14. The road marking image processing device as claimed in claim 10 , wherein the new synthesized range identifying module identifies the road surface range within a closed region that is set in advance within an image region of the road image.
In the road marking image processing device of previous descriptions, the system only analyzes a pre-defined, closed region within the road image when identifying the road surface range. This limits processing to the most important part of the image and makes it faster and more accurate.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
October 1, 2008
August 6, 2013
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.