One-sentence Summary

The RecGPT Team presents RecGPT-V3, a stateful recommender that employs a Memory Hub to distill user history into condensed memory units, a Hybrid-modal Foundation Model to reason jointly over natural language tags and Semantic IDs, and Latent Intent Reasoning to internalize verbose rationales into latent tokens, reducing token cost by $200\times$200×; deployed on Taobao's "Guess What You Like" feed, it improves GMV by $+3.97\%$+3.97% while cutting serving resource consumption by $52.4\%$52.4%.

Key Contributions

• A Memory Hub maintains a structured, evolving user memory that condenses long-horizon behavior into memory units, reducing user-modeling computation by 55.8%.
• A Hybrid-modal Foundation Model enables the LLM to reason jointly over natural language tags and Semantic IDs, opening a high-bandwidth channel from intent to item grounding and overcoming the tag-to-item bottleneck.
• Latent Intent Reasoning compresses verbose chain-of-thought into compact learnable latent tokens, cutting output token cost by 200× while preserving decodable explanations; the full RecGPT-V3 system improves IPV by +1.28%, CTR by +1.00%, TC by +1.97%, and GMV by +3.97% and reduces serving resource consumption by 52.4%.

Introduction

Traditional recommender systems predict the next interaction from co-occurrence patterns in historical logs, missing the underlying user intent and reinforcing filter bubbles. While large language models can reason about intent, earlier LLM-based approaches like RecGPT-V1 and RecGPT-V2 suffered from stateless behavior modeling that reprocesses the full history on every request, a lossy tag-to-item bottleneck that limits retrieval precision, and costly explicit chain-of-thought generation. The authors address these limitations with RecGPT-V3, which introduces a stateful Memory Hub that distills and evolves user memory, a hybrid-modal foundation model that jointly reasons over natural language and semantic item IDs to ground intent, and latent intent reasoning that compresses verbose rationales into compact, decodable tokens, achieving both accuracy gains and significant compute savings at industrial scale.

Method

The authors introduce a Memory Hub to transform stateless full-sequence encoding into stateful memory-driven reasoning. This module addresses the high computational cost of reprocessing long behavioral sequences by consolidating them into a compact set of schema-defined memory units. As shown in the figure below, the Memory Hub operates through two primary mechanisms: Structured Behavior Compression and Evolving Memory Curation.

Structured Behavior Compression performs an initial consolidation of the user's full historical behavior sequence $\mathcal{B} = \{b_1, b_2, \dots, b_N\}$B={b1​,b2​,…,bN​} into a compact set of structured memory units $\mathcal{M} = \{m_1, m_2, \dots, m_K\}$M={m1​,m2​,…,mK​} with $K \ll N$K≪N. Each unit centers on a behavior pattern identifier and a preference summary, complemented by metadata like representative behavior indices and temporal profiles. Evolving Memory Curation then keeps this memory current by periodically integrating newly observed behaviors $\Delta \mathcal{B}^{(t, t+\delta)}$ΔB(t,t+δ). It defines an incremental update function $\mathcal{G}$G that selectively updates existing memory units based on relevant new behaviors and extracts new patterns from unmatched behaviors, yielding a condensed, traceable, and evolvable user representation without requiring full re-encoding.

To overcome the bottleneck of text-only LLM experts relying on coarse item tags, the authors propose a Hybrid-Modal Recommendation Foundation Model. This architecture extends the Qwen3-14B backbone with Semantic IDs (SIDs), which are discrete codes encoding item semantics, serving as a second modality alongside natural language. The overall pipeline is illustrated in the following figure.

The model employs a hybrid tokenization scheme where multimodal item features including text, image, and side information are fused via a multimodal encoder and quantized into hierarchical discrete codes using Residual Quantization. This creates a two-level codebook structure, adding 65,536 new SID tokens to the vocabulary. The training procedure consists of two stages. First, Continual Pre-training grounds the SID tokens in natural language space using SID-grounding data mixed with general-domain data to prevent catastrophic forgetting. Second, Instruction Tuning teaches the model to apply these tokens across diverse tasks, including bidirectional translation between SIDs and text, sequential recommendation directly in SID space, and general-domain tasks to preserve pre-existing capabilities.

To retain the benefits of explicit chain-of-thought reasoning without the prohibitive token cost of autoregressive generation, the authors develop Latent Intent Reasoning. This approach internalizes verbose reasoning traces into a short sequence of learnable latent tokens. The framework is depicted in the figure below.

The reasoning internalization mechanism compresses an explicit trace $R$R into latent tokens $\boldsymbol{z} = (z_1, \dots, z_K)$z=(z1​,…,zK​) through a deterministic positional partition. The post-training pipeline consists of two stages. Stage 1, Explicit-to-Implicit CoT Alignment, distills reasoning from a strong teacher into explicit traces and then compresses them into latent tokens. This is achieved through a multi-task curriculum involving latent token warm-up and multi-granularity alignment, where the model reconstructs masked segments of the trace from the latent tokens at single-segment, multi-segment, and full-trace levels. Stage 2, Reinforcement Learning from Ranking Feedback, optimizes the latent model directly against online business objectives. Instead of sparse accuracy rewards, the authors use a dense CTRScore derived from the production ranking model, combined with constrained reward shaping for alignment, diversity, and length, optimized via GRPO.

Experiment

RecGPT-V3 is evaluated on Taobao’s homepage recommendation, where online A/B tests confirm its superiority over RecGPT-V2 across engagement and transaction metrics. The memory hub compresses behavioral sequences with high accuracy, reducing the Global Planner’s compute cost by 55.8%, while mixing general-domain data during foundation model training prevents catastrophic forgetting and boosts downstream recommendation quality. Latent reasoning condenses chain-of-thought into ten latent tokens, achieving comparable performance with a 3.46× inference speedup and yielding a 52.4% overall system resource saving. Further analysis shows that semantic IDs and text tags are complementary modalities, and a case study demonstrates how memory compression and latent reasoning enable precise, explainable recommendations.

A structured memory unit compresses a user's behavioral pattern into two core fields: a categorical behavior pattern label and a natural language preference summary. The example shown for a K-pop fandom illustrates how the unit also records supportive metadata such as representative behavior indices, preferred brands, and temporal activity. The 'Behavior Pattern' field is a categorical label, like 'K-pop Fandom', drawn from a predefined taxonomy of e-commerce behavioral archetypes. The 'Preference Summary' describes the user's full-lifecycle engagement, collecting tendencies, and item preservation habits within that pattern.

Incremental memory curation selectively updates existing user preference units when new evidence arrives, retains unchanged ones that lack relevant new behaviors, and creates new units for previously unseen interests. The examples illustrate how a baby-care unit shifts from infant to toddler needs, a cooking unit evolves toward premium cookware, and a pet parenting unit emerges from novel behaviors. Existing units are updated only when new behaviors align with their pattern, as seen when the infant care unit shifts to 6–12 month items and adds toddler toys. Units with no relevant new interactions remain unchanged, such as outdoor running and photography, which are retained as-is. Entirely new interests, like a first-time pet owner, lead to creation of a fresh memory unit rather than forcing a match to existing ones.

Instruction tuning for SID alignment includes bidirectional translation tasks between SIDs and text (titles, tags, commodity categories) and a sequential recommendation task performed purely in SID space. Among these, the next-click SID prediction task receives the largest share of tuning data, while a separate portion of general-domain data preserves the model's broader language abilities. The sid2sid sequential recommendation task has the highest proportion (20%) among SID-related tasks, teaching the model collaborative filtering signals directly from tokenized click histories. Bidirectional title mapping (sid2title and title2sid) together account for 28.5% of the SID-tuning data, emphasizing the core semantic link between item identifiers and text. Tag-to-SID mapping is the least frequent SID alignment task (6.2%), while an independent 20% of instruction data is dedicated to general-domain tasks to maintain reasoning and instruction-following capabilities.

The table illustrates how latent reasoning tokens replace segments of an explicit chain-of-thought trace, allowing the model to reconstruct masked content from context. Compared to Semantic IDs, text tags show higher cross-user overlap and broader embedding dispersion, while SIDs form tighter clusters, reflecting their complementary world-knowledge coverage and collaborative specificity. Hybrid retrieval that combines both modalities outperforms either alone, confirming their synergistic benefit. Latent tokens substitute the full explicit reasoning trace, significantly reducing inference cost while still enabling reconstruction of the original rationale. Text tags have noticeably higher cross-user overlap than Semantic IDs, indicating they capture shared world knowledge rather than user-specific signals. Embedding projections show tag-matched items scatter across diverse regions, whereas SID-matched items concentrate in tight neighborhoods. Hybrid retrieval using both text tags and SIDs achieves a higher hit rate than using either modality independently, demonstrating complementary retrieval strengths.

The Stage 1 training mixture combines a large share of general reasoning data (69.58%) to prevent catastrophic forgetting, a smaller set of reasoning alignment tasks (21.43%) that compress explicit chain-of-thought into latent tokens, and a tag prediction component (8.43%) that supervises only output tokens to keep latent positions from collapsing. General reasoning data dominates the mixture at 69.58%, acting as a safeguard against forgetting prior knowledge. Reasoning alignment tasks collectively make up 21.43%, with full-trace reconstruction as the largest subtask (8.42%). Tag prediction is allocated only 8.43% and its supervision is restricted to output tokens, preventing latent token collapse.

The experiments evaluate a memory-augmented framework that compresses user behavior into structured units with categorical pattern labels and preference summaries, then incrementally curates them by updating existing units, retaining unchanged ones, or creating new ones for novel interests. Instruction tuning aligns semantic IDs (SIDs) with text through bidirectional translation and next-click prediction, while latent reasoning tokens replace explicit chain-of-thought traces to significantly reduce inference cost while preserving reconstruction of the rationale. Hybrid retrieval combining text tags and SIDs achieves higher hit rates than either modality alone, and a training mixture dominated by general reasoning data with smaller reasoning alignment and tag prediction components prevents catastrophic forgetting and latent token collapse.