The invention relates to a real-time object detection and 3D localization method based on a single frame image. Comprising following steps: S1: inputting a 2D RGB image; S2: performing feature extraction on the 2D RGB image, extracting features of a deep network and a shallow network respectively; S3: carrying out 2D object detection and applying to subsequent modules; S4: estimating vertices, instance-level depth and center point of a 3D-box respectively; S5: adding a regularization term for maintaining horizontal locality into prediction of center point of a 3D-box to constrain and optimize the prediction of center point of the 3D-box; and S6: outputting a 2D RGB image with a 3D-box tag in combination with predictions of all modules. The invention increases the speed of model training convergence and the accuracy of 3D object detection and localization, and meets the accuracy requirements of an Advanced Driver Assistant System (ADAS) with a low hardware cost.
Legal claims defining the scope of protection, as filed with the USPTO.
3. The real-time object detection and 3D localization method based on a single frame image according to claim 2, wherein in formula (1), when depth distances of two targets are similar and the targets are more adjacent on horizontal, a weight su will be greater; and when depth distance of the target pair is larger or horizontal distance difference of the target pairs is greater, the weight sij will be smaller.
4. The real-time object detection and 3D localization method based on a single frame image according to claim 2, wherein the loss function of target confidence is a combination of a softmax function and a cross entropy; and the loss function of a 2D-Box is calculated by an L1 distance loss function.
6. The real-time object detection and 3D localization method based on a single frame image according to claim 1, wherein the 3D-box will be represented by a 3D center point of an object and coordinate points of 8 vertices of the 3D-box.
9. The real-time target detection and 3D positioning method based on a single frame image according to claim 8, wherein the embedded system is Jetson AGX Xavier.
10. The real-time target detection and 3D positioning method based on a single frame image according to claim 1, wherein the instance-level depth information is data obtained by predicting depth zg of the center point of the 3D-box through an instance-level depth prediction module, i.e., after a feature map is divided into grids, the depth prediction module only predicts a target depth of a grid having a distance from an instance less than a distance threshold σscope.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
June 25, 2021
April 2, 2024
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.