Prompting for Multimodal Hateful Meme Classification

Hateful meme classification is a challenging multimodal task that requirescomplex reasoning and contextual background knowledge. Ideally, we couldleverage an explicit external knowledge base to supplement contextual andcultural information in hateful memes. However, there is no known explicitexternal knowledge base that could provide such hate speech contextualinformation. To address this gap, we propose PromptHate, a simple yet effectiveprompt-based model that prompts pre-trained language models (PLMs) for hatefulmeme classification. Specifically, we construct simple prompts and provide afew in-context examples to exploit the implicit knowledge in the pre-trainedRoBERTa language model for hateful meme classification. We conduct extensiveexperiments on two publicly available hateful and offensive meme datasets. Ourexperimental results show that PromptHate is able to achieve a high AUC of90.96, outperforming state-of-the-art baselines on the hateful memeclassification task. We also perform fine-grained analyses and case studies onvarious prompt settings and demonstrate the effectiveness of the prompts onhateful meme classification.