A computer includes a processor and a memory storing instructions executable by the processor to receive radar data including a radar pixel having a radial velocity from a radar; receive camera data including an image frame including camera pixels from a camera; map the radar pixel to the image frame; generate a region of the image frame surrounding the radar pixel; determine association scores for the respective camera pixels in the region; select a first camera pixel of the camera pixels from the region, the first camera pixel having a greatest association score of the association scores; and calculate a full velocity of the radar pixel using the radial velocity of the radar pixel and a first optical flow at the first camera pixel. The association scores indicate a likelihood that the respective camera pixels correspond to a same point in an environment as the radar pixel.
Legal claims defining the scope of protection, as filed with the USPTO.
2. The computer of claim 1, wherein the instructions further control a vehicle including the radar and the camera based on the full velocities.
3. The computer of claim 1, wherein mapping the radar pixels to the image frame includes applying a geometric transformation to the radar pixels.
4. The computer of claim 1, wherein inputs to the trained network include the radar data, the image frame, and optical flow of the image frame.
5. The computer of claim 4, wherein the radar data are accumulated over a time interval.
6. The computer of claim 1, wherein the trained network is a convolutional neural network.
7. The computer of claim 1, wherein the trained network is an encoder-decoder network.
8. The computer of claim 1, wherein each projected full velocity is calculated using the radial velocity of the respective radar pixel and a respective optical flow at the respective camera pixel.
9. The computer of claim 1, wherein the image frame is a first image frame, the camera data include a second image frame, and the first optical flow for each first camera pixel represents shifting between the first image frame at the first camera pixel and the second image frame.
10. The computer of claim 1, wherein the respective regions have preset positions in the image frame relative to the respective radar pixels.
11. The computer of claim 10, wherein the respective regions are preset shapes in the image frame, and the preset shapes have a greater height than width.
12. The computer of claim 10, wherein the respective regions are preset shapes in the image frame, and the preset shapes extend a greater distance vertically upward than downward from the radar pixel.
13. The computer of claim 1, wherein the instructions further include instructions to, for the respective regions, upon the greatest association score being below a threshold association score, refrain from calculating the full velocity of the respective radar pixel.
14. The computer of claim 1, wherein, for the respective regions, calculating the full velocity of the respective radar pixel is performing a closed-form calculation.
15. The computer of claim 1, wherein the full velocities of the radar pixels are calculated using only the radar data and the camera data.
16. The computer of claim 1, wherein the first camera pixel is at an updated location in the image frame from a location to which the radar pixel was mapped in the image frame.
17. The computer of claim 1, wherein the radar data are single-row data, and the radar data are sparser horizontally than the camera data are.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
July 23, 2021
December 10, 2024
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.