Multi-Granularity Part Sampling Attention for Fine-Grained Visual Classification

Fine-grained visual classification aims to classify similar sub-categories with the challenges of large variations within the same sub-category and high visual similarities between different sub-categories. Recently, methods that extract semantic parts of the discriminative regions have attracted increasing attention. However, most existing methods extract the part features via rectangular bounding boxes by object detection module or attention mechanism, which makes it difficult to capture the rich shape information of objects. In this paper, we propose a novel Multi-Granularity Part Sampling Attention (MPSA) network for fine-grained visual classification. First, a novel multi-granularity part retrospect block is designed to extract the part information of different scales and enhance the high-level feature representation with discriminative part features of different granularities. Then, to extract part features of various shapes at each granularity, we propose part sampling attention, which can sample the implicit semantic parts on the featuremaps comprehensively. The proposed part sampling attention not only considers the importance of sampled parts but also adoptsthe part dropout to reduce the overfitting issue. In addition, we propose a novel multi-granularity fusion method to highlight theforeground features and suppress the background noises with the assistance of the gradient class activation map. Experimentalresults demonstrate that the proposed MPSA achieves state-ofthe-art performance on four commonly used fine-grained visualclassification benchmarks. The source code is publicly available at https://github.com/mobulan/MPSA.