Multi-step Error Minimization

Multi-step Error Minimization (MEM) was published in 2024 by the Institute of Information Engineering of the Chinese Academy of Sciences, Nanyang Technological University, National University of Singapore, and Sun Yat-sen University in the paper “Multimodal Unlearnable Examples: Protecting Data against Multimodal Contrastive Learning” proposed a new optimization procedure for generating multimodal non-learnable examples. It extends the error minimization (EM) framework to optimize image noise and additional text triggers, thereby expanding the optimization space and effectively misleading the model to learn shortcuts between noise features and text triggers.

The research team adopted projected gradient descent to solve the noise minimization problem, and used HotFlip to approximate the gradient and replace words to find the best text trigger. Extensive experiments have demonstrated the effectiveness of MEM, and the retrieval results after protection are almost half of random guessing, and it has high portability between different models.