Methods and systems are provided for generating a multi-label classification system. The multi-label classification system can use a multi-label classification neural network system to identify one or more labels for an image. The multi-label classification system can explicitly take into account the relationship between classes in identifying labels. A relevance sub-network of the multi-label classification neural network system can capture relevance information between the classes. Such a relevance sub-network can decouple independence between classes to focus learning on relevance between the classes.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
5. The media of non-transitory computer-readable claim 4, wherein the error is determined using binary cross-entropy loss.
6. The media of non-transitory computer-readable claim 1, wherein the plurality of channel-wise attention maps are generated using a relevance sub-network to process the plurality of class-wise feature maps and enable determination of relevance between the plurality of classes based at least in part on class-independent features of the image.
7. The media of non-transitory computer-readable claim 1, wherein the plurality of channel-wise attention maps are generated using a relevance sub-network to process the plurality of convolutional feature maps and enable determination of relevance between the plurality of classes based at least in part on local detailed information relating to features of the image.
8. The media non-transitory computer-readable of claim 1, wherein the coupling of combined class-wise feature maps with the plurality of channel-wise attention maps produces a category-wise feature map that dynamically re-weights the combined class-wise feature maps based on the plurality of channel-wise attention maps.
12. The computer-implemented method of claim 9, wherein the plurality of class-wise feature maps have a decreased spatial resolution, the decreased spatial resolution being a spatial resolution of the convolutional feature maps.
This invention relates to computer vision and deep learning, specifically improving the efficiency of convolutional neural networks (CNNs) by reducing computational complexity in feature extraction. The problem addressed is the high computational cost and memory usage associated with processing high-resolution feature maps in CNNs, which limits their scalability and real-time performance. The method involves generating a plurality of class-wise feature maps from convolutional feature maps, where the class-wise feature maps have a decreased spatial resolution compared to the original convolutional feature maps. The decreased spatial resolution matches the spatial resolution of the convolutional feature maps, meaning the feature maps are downsampled to a lower resolution to reduce the number of spatial dimensions while preserving essential features. This reduction in spatial resolution minimizes computational overhead and memory requirements during subsequent processing steps, such as classification or object detection. The method may also include generating a plurality of class-wise feature maps from convolutional feature maps, where the class-wise feature maps have a decreased spatial resolution compared to the original convolutional feature maps. The decreased spatial resolution matches the spatial resolution of the convolutional feature maps, meaning the feature maps are downsampled to a lower resolution to reduce the number of spatial dimensions while preserving essential features. This reduction in spatial resolution minimizes computational overhead and memory requirements during subsequent processing steps, such as classification or object detection. The method may also include generating a plurality of class-wise feature maps from convolutional feature maps, wh
13. The computer-implemented method of claim 9, wherein the plurality of channel-wise attention maps are generated using a relevance sub-network to process the plurality of class-wise feature maps and enable determination of relevance between the plurality of classes based at least in part on class-independent features of the image.
14. The computer-implemented method of claim 9, wherein the plurality of channel-wise attention maps are generated using a relevance sub-network to process the plurality of convolutional feature maps and enable determination of relevance between the plurality of classes based at least in part on local detailed information relating to features of the image.
15. The computer-implemented method of claim 9, wherein coupling of the category-wise feature map with the plurality of channel-wise attention maps dynamically re-weights the category-wise attention map based on the channel-wise attention maps.
17. The system of claim 16, wherein the output is a plurality of labels each based on a probability for a class based on the coupled category-wise feature map and the plurality of channel-wise attention maps in relation to a predefined threshold level score.
18. The system of claim 16, wherein the output further comprises a heat map based at least in part on the coupled category-wise feature map and the plurality of channel-wise attention maps, wherein the heat map provides a visualization mapping locations of a feature of the image corresponding to one of the plurality of labels.
19. The system of claim 16, wherein the plurality of channel-wise attention maps are generated using a relevance sub-network to process the plurality of class-wise feature maps and enable determination of relevance between the plurality of classes based at least in part on class-independent features of the image.
20. The system of claim 16, wherein the plurality of channel-wise attention maps are generated using a relevance sub-network to process the plurality of convolutional feature maps and enable determination of relevance between the plurality of classes based at least in part on local detailed information relating to features of the image.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
July 11, 2019
November 8, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.