Automatic calibration between laser and vision sensors carried by a mobile platform, and associated systems and methods are disclosed herein. A representative method includes evaluating depth-based feature points obtained from the laser sensor with edge information obtained from the vision sensor and generating calibration rules based thereon.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A computer-implemented method for calibrating at least a laser unit and a vision unit, both carried by a common mobile platform, the method comprising: combining temporally sequenced sets of point information obtained from the laser unit to form a point cloud in a reference system associated with the mobile platform based on one or more transformation rules, wherein the one or more transformation rules specify transformations between reference systems associated with the laser unit at different points in time within a time period to a target reference system associated with the laser unit at a target point in time; extracting a plurality of feature points from the point cloud, wherein the feature points correspond to a threshold discontinuity in depth measurement; evaluating the feature points with edge information obtained from the vision unit based at least in part on a target function, the target function defined at least in part by positions of the feature points when projected to a reference system associated with the vision unit; generating at least one calibration rule for calibration between the laser unit and the vision unit based at least in part on evaluating the feature points with the edge information; and causing calibration between the laser unit and the vision unit using the at least one calibration rule.
2. The method of claim 1 , further comprising converting an image obtained from the vision unit into a grayscale image.
3. The method of claim 2 , further comprising determining the edge information based at least in part on a difference between at least one pixel of the grayscale image and one or more pixels within a threshold proximity of the at least one pixel.
4. The method of claim 3 , wherein evaluating the feature points with the edge information comprises projecting the feature points to respective positions in an image obtained from the vision unit.
5. The method of claims 1 , wherein generating at least one calibration rule comprises optimizing the target function.
6. The method of claim 5 , wherein optimizing the target function is based at least in part on an exhaustion method.
7. The method of claim 5 , wherein optimizing the target function comprises optimizing the target function in accordance with at least six degrees of freedom.
8. A non-transitory computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors associated with a mobile platform to perform actions, the actions comprising: combining temporally sequenced sets of point information obtained from a laser unit to form a point cloud in a reference system associated with the mobile platform based on one or more transformation rules, wherein the one or more transformation rules specify transformations between reference systems associated with the laser unit at different points in time within a time period to a target reference system associated with the laser unit at a target point in time; extracting a plurality of feature points from the point cloud, wherein the feature points correspond to a threshold discontinuity in depth measurement; evaluating the feature points with edge information obtained from a vison unit based at least in part on a target function, the target function defined at least in part by positions of the feature points when projected to a reference system associated with the vision unit; generating at least one calibration rule for calibration between the laser unit and the vision unit based at least in part on evaluating the feature points with the edge information; and causing calibration between the laser unit and the vision unit using the at least one calibration rule.
9. The computer-readable medium of claim 8 , wherein the actions further comprise transforming the plurality of feature points based at least in part on a set of transformation rules.
10. The computer-readable medium of claim 9 , wherein the set of transformation rules is at least partially defined in accordance with a position and orientation of the vision unit relative to the mobile platform.
11. The computer-readable medium of claim 9 , wherein the set of transformation rules comprises a transformation matrix.
12. The computer-readable medium of claim 8 , wherein the reference system associated with the mobile platform comprises a coordinate system.
13. The computer-readable medium of claim 8 , wherein extracting the plurality of feature points is based at least in part on one or more depth differences between points within the point cloud.
14. The computer-readable medium of claim 13 , wherein extracting the plurality of feature points is further based on a relationship between the one or more depth differences and the threshold discontinuity.
15. The computer-readable medium of claim 8 , wherein the laser unit comprises at least one laser sensor that has a field of view (FOV) smaller than at least one of 360 degrees, 180 degrees, 90 degrees, or 60 degrees.
16. The computer-readable medium of claim 8 , wherein the vision unit includes a monocular camera.
17. A vehicle including a programmed controller that at least partially controls one or more motions of the vehicle, wherein the programmed controller includes one or more processors configured to: combine temporally sequenced sets of point information obtained from a laser unit to form a point cloud in a reference system associated with the vehicle based on one or more transformation rules, wherein the one or more transformation rules specify transformation rules between reference systems associated with the laser unit at different points in time within a time period to a target reference system associated with the laser unit at a target point in time; extract a plurality of feature points from the point cloud, wherein the feature points correspond to a threshold discontinuity in depth measurement. evaluate the feature points with edge information obtained from a vision unit based at least in part on a target function, the target function defined at least in part by positions of the feature points when projected to a reference system associated with the vision unit; generate at least one calibration rule for calibration between the laser unit and the vision unit based at least in part on evaluating the feature points with the edge information; and cause calibration between the laser unit and the vision unit using the at least one calibration rule.
18. The vehicle of claim 17 , wherein extracting the plurality of feature points comprises transforming a subset of the point cloud based at least in part on a set of transformation rules.
19. The vehicle of claim 18 , wherein the set of transformation rules is at least partially defined in accordance with a position and orientation of the vision unit relative to the vehicle.
20. The vehicle of claim 18 , wherein the one or more processors are further configured to select the subset of the point cloud based at least in part on one or more depth differences between points within the point cloud.
21. The vehicle of claim 20 , wherein selecting the subset of the point cloud comprises selecting a subset of points based at least in part on one set of the temporally sequenced sets of point information.
22. The vehicle of claim 17 , wherein the one or more processors are further configured to convert an image obtained from the vision unit into a grayscale image.
23. The vehicle of claim 22 , wherein the one or more processors are further configured to determine the edge information based at least in part on a difference between at least one pixel of the grayscale image and one or more pixels within a threshold proximity of the at least one pixel.
24. The vehicle of claim 17 , wherein the at least one calibration rule includes a rule for transformation between a reference system associated with the laser unit and the reference system associated with the vision unit.
25. The vehicle of claim 17 , wherein the one or more processors are further configured to detect a difference between (a) the generated at least one calibration rule with (b) one or more previously generated calibration rules.
26. The vehicle of claim 17 , wherein the vehicle corresponds to at least one of an unmanned aerial vehicle (UAV), a manned aircraft, an autonomous car, a self-balancing vehicle, or a robot.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
October 11, 2017
October 8, 2019
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.