QCS: Feature Refining from Quadruplet Cross Similarity for Facial Expression Recognition

Facial expression recognition faces challenges where labeled significantfeatures in datasets are mixed with unlabeled redundant ones. In this paper, weintroduce Cross Similarity Attention (CSA) to mine richer intrinsic informationfrom image pairs, overcoming a limitation when the Scaled Dot-Product Attentionof ViT is directly applied to calculate the similarity between two differentimages. Based on CSA, we simultaneously minimize intra-class differences andmaximize inter-class differences at the fine-grained feature level throughinteractions among multiple branches. Contrastive residual distillation isutilized to transfer the information learned in the cross module back to thebase network. We ingeniously design a four-branch centrally symmetric network,named Quadruplet Cross Similarity (QCS), which alleviates gradient conflictsarising from the cross module and achieves balanced and stable training. It canadaptively extract discriminative features while isolating redundant ones. Thecross-attention modules exist during training, and only one base branch isretained during inference, resulting in no increase in inference time.Extensive experiments show that our proposed method achieves state-of-the-artperformance on several FER datasets.