A method includes detecting at least one region of interest in a frame of image data. One or more patches of interest are detected in the frame of image data based on detecting the at least one region of interest. A model including a deep convolutional neural network is applied to the one or more patches of interest. Post-processing of a result of applying the model is performed to produce a post-processing result for the one or more patches of interest. A visual indication of a classification of defects in a structure is output based on the result of the post-processing.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method comprising: detecting at least one region of interest in a frame of image data; detecting one or more patches of interest in the frame of image data based on detecting the at least one region of interest; applying a model comprising a deep convolutional neural network to the one or more patches of interest; performing post-processing of a result of applying the model to produce a post-processing result for the one or more patches of interest; and outputting a visual indication of a classification of defects in a structure based on the result of the post-processing, wherein the classification distinguishes between normal edges of the structure and cracks of the structure.
2. The method of claim 1 , wherein detecting the one or more patches of interest comprises applying a threshold on a percentage of pixels with edges in a given patch.
3. The method of claim 1 , wherein the post-processing comprises aggregating classifications from each of the one or more patches and smoothing the classifications to identify dominant classifications.
4. The method of claim 1 , wherein the visual indication comprises a classification heat map overlaid upon the image data to highlight location and severity of the defects.
5. The method of claim 1 , wherein the method is performed in part using cloud computing resources.
6. The method of claim 1 , wherein the image data is received from a boroscope camera.
7. The method of claim 1 , wherein the model is trained using a plurality of image frames comprising a plurality of defects labeled on a patch or pixel basis.
8. The method of claim 1 , wherein the image data comprises at least one channel per frame.
9. The method of claim 1 , wherein the deep convolutional neural network comprises a plurality of pairs of convolution layers and pooling layers, and at least one of the convolution layers comprises a plurality of kernels with a pixel stride and pixel edge padding.
10. The method of claim 9 , wherein the deep convolutional neural network comprises three of the pairs of convolution layers and pooling layers, and a third pooling layer of the pairs is connected to a soft-max layer configured to provide a defect classification value for each of the one or more patches of interest in the frame.
11. A system comprising: a camera or a database of images; and a processing system operable to: detect at least one region of interest in a frame of image data from the camera or the database of images; detect one or more patches of interest in the frame of image data based on detecting the at least one region of interest; apply a model comprising a deep convolutional neural network to the one or more patches of interest; perform post-processing of a result of applying the model to produce a post-processing result for the one or more patches of interest; and output a visual indication of a classification of defects in a structure based on the result of the post-processing, wherein the classification distinguishes between normal edges of the structure and cracks of the structure.
12. The system of claim 11 , wherein detection of the one or more patches of interest comprises application of a threshold on a percentage of pixels with edges in a given patch.
13. The system of claim 11 , wherein the post-processing comprises aggregation of classifications from each of the one or more patches and smoothing the classifications to identify dominant classifications.
14. The system of claim 11 , wherein the visual indication comprises a classification heat map overlaid upon the image data to highlight location and severity of the defects.
15. The system of claim 11 , wherein the processing system interfaces with cloud computing resources to perform a portion of the processing.
16. The system of claim 11 , wherein the camera is a boroscope camera.
17. The system of claim 11 , wherein the model is trained using a plurality of image frames comprising a plurality of defects labeled on a patch or pixel basis.
18. The system of claim 11 , wherein the image data comprises at least one channel per frame.
19. The system of claim 11 , wherein the deep convolutional neural network comprises a plurality of pairs of convolution layers and pooling layers, and at least one of the convolution layers comprises a plurality of kernels with a pixel stride and pixel edge padding.
20. The system of claim 19 , wherein the deep convolutional neural network comprises three of the pairs of convolution layers and pooling layers, and a third pooling layer of the pairs is connected to a soft-max layer configured to provide a defect classification value for each of the one or more patches of interest in the frame.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
May 16, 2016
December 8, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.