Few-Shot Learning is a technique in large language model prompting that involves providing a small number of examples in the prompt, allowing the model to learn the current task through in-context learning, typically without updating the model weights. Zero-shot, one-shot, and few-shot describe common distinctions based on the number of examples in the prompt. It is important to note that in a broader sense, few-shot learning also includes training or adaptation methods such as meta-learning and fine-tuning, which are not all equivalent to in-context learning.

In 2020, Brown et al. in the paper 《Language Models are Few-Shot Learners》 brought this approach to the forefront with GPT-3: the model had 175 billion parameters, ten times the size of the largest previous non-sparse language models; all tasks were performed without gradient updates or fine-tuning, with task descriptions and examples included purely as text in the prompt. It demonstrated strong performance in translation, question answering, cloze tests, and tasks requiring on-the-fly reasoning such as unscrambling words and using new terms, sometimes approaching state-of-the-art levels achieved by models fine-tuned for single tasks.

This also explains why the same model’s performance can vary significantly under different prompts—in-context learning is highly sensitive to “how examples are presented.” It shifted the paradigm from “training a model for each task” to “using one model to adapt to all tasks via prompting,” directly giving rise to prompt engineering and in-context learning. The limitations are equally evident: performance is sensitive to prompt wording and example selection, both the choice and ordering of examples can affect results, the number of examples is constrained by the context window, and complex reasoning tasks still require more structured approaches.