One-sentence Summary

Google Research and Cornell University propose a "Sleep" paradigm for LLMs that combines Memory Consolidation via Knowledge Seeding with Generalized Distillation (on-policy distillation and RL-based imitation) and a Dreaming phase using RL-generated synthetic data for self-improvement, enabling continual learning and knowledge transfer and yielding gains on long-horizon, continual learning, knowledge incorporation, and few-shot generalization tasks.

Key Contributions

• A Sleep paradigm for large language models replaces the static train/test cycle with a continuous wake-sleep lifecycle, where Knowledge Seeding consolidates short-term in-context knowledge into stable long-term parameters and a Dreaming phase enables self-improvement.
• A Generalized Distillation process for Knowledge Seeding combines on-policy distillation with reinforcement-learning-based imitation learning. A smaller teacher generates masked sequences, and the student is rewarded for predicting continuations close to the teacher's outputs, transferring fragile in-context memories to higher-capacity parameters.
• The Dreaming phase uses random low-rank expert selection in a Mixture of Experts to mix knowledge across irrelevant experts and reinforcement learning to generate a synthetic data curriculum for self-improvement without human supervision. Experiments show consistent gains on long-context understanding, factual knowledge incorporation, few-shot reasoning, and continual learning.

Introduction

Large language models (LLMs) remain static after deployment, causing their knowledge to become outdated. Continual parameter updates can alleviate this but often lead to catastrophic forgetting of previously learned tasks. In-context learning offers a lightweight way to adapt, yet it only stores information temporarily within the context window. Prior attempts at online memory consolidation, such as nested learning, transfer knowledge to more stable modules but retain it at the same abstraction level without compression, and the process is retrieval-dependent and context-bound. The authors introduce a sleep paradigm for LLMs that alternates between active learning and offline consolidation phases. During sleep, the model uses periodic parameter (de)activation and knowledge seeding to distill short-term memories into lower-frequency, long-term parameters, and employs dreaming via synthetic data generation to self-improve while controlling interference.

Method

The authors propose the Continuum Memory System (CMS) to address catastrophic forgetting in continual learning. The CMS architecture consists of a sequence model, such as an attention mechanism, followed by a chain of MLP layers. Each layer in this chain is associated with a specific update frequency. The sequence model acts as short-term memory with high-frequency updates, while the subsequent MLP blocks form a spectrum of memory modules, ranging from higher-frequency shorter-term memories to lower-frequency longer-term memories.

To manage the lifecycle of a continual learner, the authors divide the process into two critical phases: active (wake) time and sleep time.

During the active or wake time, the model processes new external input data. Online consolidation occurs here through end-to-end learning, where earlier, faster-updating layers transfer knowledge to later, slower-updating layers. In contrast, during sleep time, the model stops receiving external data and focuses on processing existing knowledge to consolidate memories and self-improve.

Memory consolidation is an iterative operation that transfers fragile, short-term memories from higher-frequency blocks into more stable, lower-frequency parameters. To prevent catastrophic forgetting during this transfer, the authors introduce parameter expansion. Specifically, before updating a slower memory block, a new low-rank expert is added to its set of parameters. This allows the model to shape new connections and increase its capacity without interfering with previously stored knowledge.

Following parameter expansion, the authors employ a compute-consolidate-update protocol called Knowledge Seeding. In this step, the knowledge from a higher-frequency sender block is distilled into the expanded parameters of the lower-frequency receiver block. This is formulated as a distillation problem where a smaller-state teacher model transfers knowledge to a larger-capacity student model. The process involves an on-policy distillation objective that mixes teacher-generated and student-generated data, followed by a reinforcement learning-based Learning to Imitate method to align the student sampling process with the teacher.

Finally, the sleep paradigm includes a Dreaming phase, analogous to the REM sleep stage in humans. During dreaming, the model generates synthetic data to self-improve. To ensure these dreams are beneficial and maintain diversity, the model samples from its existing knowledge space while incorporating random irrelevant knowledge. The generated dreams are evaluated based on their potential to improve downstream performance, and the top-scoring dreams are selected for supervised fine-tuning, allowing the model to explore novel syntheses of memories and further strengthen its capabilities.

Experiment

The experiments evaluate the proposed sleep mechanism across continual learning, long-context understanding, language acquisition, reasoning, knowledge incorporation, and few-shot learning. The memory consolidation phase alone improves class-incremental learning and long-context behavior by converting prompt-level adaptation into durable parametric memory, with more consolidation stages yielding better performance. When combined with self-improvement, the full sleep process achieves state-of-the-art results in knowledge incorporation and few-shot reasoning, and ablation studies confirm that all design components contribute positively.

The full Sleep method achieves the highest average@16 scores on AIME-24 and HMMT-25, outperforming its ablations and the OPSD baseline. On AIME-25, the Semantic Reward variant alone reaches the best result, marginally ahead of the full Sleep model. Removing the expansion component from Sleep or adding it to OPSD consistently shifts performance, indicating expansion contributes positively. Sleep attains the top average@16 on AIME-24 (79.2) and HMMT-25 (46.1), surpassing all other configurations. The Semantic Reward variant achieves the highest AIME-25 score (69.2), slightly exceeding the full Sleep method (69.0).

Sleep, a memory consolidation method, achieves the highest average@16 scores on all three mathematical reasoning benchmarks with Qwen3-1.7B, outperforming supervised fine-tuning (SFT), GRPO, and OPSD. SFT degrades performance relative to the base instruct model on two of the three datasets, while GRPO and OPSD provide progressive gains. The results indicate that consolidation improves reasoning beyond standard training and reinforcement learning approaches. Sleep yields the best results across AIME-24, AIME-25, and HMMT-25, surpassing both GRPO and OPSD. Supervised fine-tuning underperforms the base instruct model on AIME-24 and HMMT-25, suggesting naive fine-tuning can harm reasoning. GRPO improves over the base model, and OPSD further lifts scores, but Sleep provides the largest consistent boost.

Sleep-based memory consolidation substantially improves knowledge incorporation over SEAL and fine-tuning baselines. A four-level memory system yields the highest accuracy, with both gradient-based selection and random expert components contributing to performance. Gains are largest in the single-passage setting, while continued pretraining shows consistent but more modest improvements. Sleep with a four-level memory system achieves the highest accuracy in both single-passage and continued pretraining settings. Removing gradient-based selection or the random expert lowers performance, confirming the value of these design choices. The base model and fine-tuned model without dreaming perform similarly, while SEAL and sleep methods bring large gains.

On few-shot abstract reasoning, Sleep achieves the highest success rate at 80%, substantially outperforming the next-best method SEAL (72.5%). Standard in-context learning (ICL) and test-time training (TTT) fail to solve the task, with 0% and 10% success respectively, underscoring the importance of memory consolidation for this challenge. Sleep attains an 80% success rate, surpassing SEAL by 7.5 percentage points and leaving ICL and TTT far behind. ICL yields zero success and TTT only 10%, showing that without consolidation the model cannot solve these few-shot abstract reasoning tasks.

The Sleep memory consolidation method is evaluated across mathematical reasoning, knowledge incorporation, and few-shot abstract reasoning tasks. It consistently outperforms supervised fine-tuning, reinforcement learning, and other consolidation baselines, with expansion and multi-level memory components contributing positively. Consolidation proves essential for abstract reasoning, where standard in-context learning and test-time training fail, and overall, Sleep provides robust gains by enhancing reasoning and knowledge retention beyond conventional training approaches.