Parameter-efficient Fine-tuning

Parameter Efficient Fine-tuning (PEFT) is a technique used in Natural Language Processing (NLP) to improve the performance of a pre-trained language model on a specific downstream task. It involves reusing the parameters of a pre-trained model and fine-tuning it on a smaller dataset, which can save computational resources and time compared to training the entire model from scratch. PEFT achieves this efficiency by freezing certain layers of the pre-trained model and fine-tuning only the last few layers that are specific to the downstream task. This way, the model can be adapted to new tasks with less computational overhead and fewer labeled examples. Although PEFT is a relatively new concept, updating the last layers of a model has been practiced in computer vision since the introduction of transfer learning.

In 2024, Sun Maosong's team from the Department of Computer Science at Tsinghua University proposed a related research result "Efficient parameter fine-tuning for large-scale pre-trained language models", the paper was published in Nature Machine Intelligence. The research results define and describe the Delta Tuning problem, and review previous research through a unified framework. Delta Tuning methods can be divided into three groups: Addition-based, Specification-based, and Reparameterization-based methods. The research team also proposed a theoretical framework from the perspectives of optimization and optimal control to guide subsequent structural and algorithmic design, and conducted comprehensive experimental comparisons and performance analysis on more than 100 NLP tasks.