One-sentence Summary

Researchers from Tsinghua University’s Institute for AI Industry Research (AIR) and ByteDance Seed propose Subspace-Aligned Rewiring (SAR), a post-hoc editing method that extracts a compact reasoning-effective spectral core from reinforcement learning updates, preserving over 99% of performance with $\sim 0.58\%$∼0.58% of parameters while improving exploration, purifying cross-domain interference, and enabling superior model merging.

Key Contributions

• Subspace-Aligned Rewiring (SAR) extracts compact reasoning cores from RL updates by projecting them into the base model's spectral space, preserving over 99% of math performance with as little as 0.58% of parameters and improving six of seven agentic coding benchmarks.
• SAR improves high-k exploration, surpassing full-parameter RL in Pass@k scaling, and in mixed-domain RL it purifies updates to release suppressed coding capability while maintaining math reasoning and instruction following.
• SAR enables model merging across math and code experts, producing merged models that outperform the best single-domain experts and previous merging baselines on both AIME and LiveCodeBench at multiple scales.

Introduction

Reinforcement learning (RL) post-training has become a key method for improving large language models on tasks like math, coding, and agentic workflows by optimizing against task-level feedback. However, the black-box nature of full-parameter RL provides limited control over the induced parameter changes, leading to two practical bottlenecks: reward optimization can narrow the policy to a small set of high-reward trajectories, suppressing exploration and test-time scaling, and updates from multiple domains can interfere destructively in parameter space, causing cross-domain performance trade-offs. The authors address these issues by proposing a geometric perspective in which the base model's singular value decomposition (SVD) defines a coordinate system of pretrained "atomic skills." They show that RL primarily reconnects these skills through a compact, non-diagonal rewiring matrix rather than arbitrary parameter drift. This insight yields Subspace-Aligned Rewiring (SAR), a post-hoc editing method that projects dense RL updates onto the base model's spectral manifold, enabling extraction of a compact reasoning-effective component, purification of multi-domain updates to reduce interference, and improved expert model merging.

Method

The authors introduce a geometric framework to understand and extract reasoning capabilities elicited by outcome-reward reinforcement learning (RL) from a pretrained model. The core idea is that the reasoning-effective part of an RL update resides within the pretrained model's spectral subspace, and can be isolated via a projection-based algorithm called Subspace-Aligned Rewiring (SAR).

The framework begins by analyzing a linear layer's weight matrix $W_0 \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$W0​∈Rdout​×din​ via singular value decomposition (SVD):

$y_{\text{base}} = W_0 x = U \Sigma V^\top x = \sum_{i=1}^r \sigma_i u_i (v_i^\top x)$ybase​=W0​x=UΣV⊤x=∑i=1r​σi​ui​(vi⊤​x)

Here, the right singular vectors $v_i$vi​ act as feature detectors (information read-in), and the left singular vectors $u_i$ui​ define output directions (information read-out), scaled by singular values $\sigma_i$σi​. The authors argue that the pretrained model's functional capacity is concentrated in the principal spectral manifold $\mathcal{S}_r(W_0) = \operatorname{span}(U \otimes V)$Sr​(W0​)=span(U⊗V), which serves as a coordinate system of latent skills.

When a model is fine-tuned with outcome-reward RL, the resulting weight matrix $W_{\text{RL}}$WRL​ yields an update $\Delta W = W_{\text{RL}} - W_0$ΔW=WRL​−W0​. The central hypothesis is that the reasoning-relevant component of this update can be recovered by projecting $\Delta W$ΔW onto the pretrained spectral subspace. The update is decomposed as $\Delta W = \Delta W^* + \Delta W_\perp$ΔW=ΔW∗+ΔW⊥​, where $\Delta W^*$ΔW∗ is the subspace-aligned component and $\Delta W_\perp$ΔW⊥​ is the residual outside the manifold. Using projection operators $P_U = U U^\top$PU​=UU⊤ and $P_V = V V^\top$PV​=VV⊤, the reasoning-effective update is extracted as:

$\Delta W^* = P_U \Delta W P_V = U (U^\top \Delta W V) V^\top = U M V^\top$ΔW∗=PU​ΔWPV​=U(U⊤ΔWV)V⊤=UMV⊤

The matrix $M = U^\top \Delta W V \in \mathbb{R}^{r \times r}$M=U⊤ΔWV∈Rr×r is called the Rewiring Matrix. Unlike the original diagonal $\Sigma$Σ, $M$M is generally dense, capturing cross-dimensional interactions between pretrained singular vectors. In practice, the method first extracts a low-rank approximation of $\Delta W$ΔW and then projects it onto the SVD subspace, combining parameter compression with spectral alignment.

The rewired forward pass becomes $W^* = W_0 + \Delta W^* = U(\Sigma + M)V^\top$W∗=W0​+ΔW∗=U(Σ+M)V⊤, leading to:

$y_{\text{rewired}} = \sum_{i=1}^r \left[ \underbrace{(\sigma_i + M_{ii})(v_i^\top x)}_{\text{Rescaling}} + \underbrace{\sum_{j \neq i} M_{ij}(v_j^\top x)}_{\text{Rewiring}} \right] u_i$yrewired​=∑i=1r​​Rescaling(σi​+Mii​)(vi⊤​x)​​+Rewiringj=i∑​Mij​(vj⊤​x)​​​ui​

This reveals a mechanistic distinction: diagonal entries $M_{ii}$Mii​ simply rescale existing capabilities, while off-diagonal entries $M_{ij}$Mij​ enable a single output direction $u_i$ui​ to be synthesized from multiple distinct input features $v_j$vj​ ($j \neq i$j=i). This transition from one-to-one associative mapping to many-to-one logical synthesis provides a geometric basis for reasoning, allowing the model to compose independent latent premises into a shared output representation. SAR thus retains the structural rewiring responsible for relational integration while discarding update components that fall outside the pretrained spectral manifold.

The SAR framework offers a practical interface: reasoning updates can be compressed, filtered, or merged by operating on the compact rewiring matrix rather than the full parameter update. This geometric perspective is later applied to cross-domain generalization, where projecting multi-domain RL updates or merged expert models onto the base spectral manifold helps retain shared reasoning structure and filter incompatible residual directions.

Experiment

The experiments evaluate Spectral Attention Rewiring (SAR), a method that extracts a compact, reasoning-effective spectral update from reinforcement learning (RL) fine-tuned models. Across diverse model scales and families on mathematical reasoning (AIME), coding (LiveCodeBench, agentic benchmarks), and instruction following (IFEval), SAR preserves full RL reasoning gains while retaining only 1–10% of the pretrained spectral rank, and it improves exploration at larger sampling budgets without retraining. In cross-domain settings, projecting multi-task RL updates or merging domain experts through this spectral geometry reduces task interference, yielding merged models that surpass single-domain experts and releasing suppressed cross-domain capabilities, demonstrating that the transferable reasoning signal is concentrated in the base model’s spectral manifold.

Spectral rewiring preserves nearly all RL reasoning gains on AIME benchmarks while using only a tiny fraction of the pretrained spectral rank. Across model scales, projected models match full RL performance within evaluation variance, and strong reasoning recipes require as little as 1% retained rank. This shows that RL-induced improvements concentrate in a compact spectral subspace rather than needing the full parameter update. For the 1.5B model, retaining just 1% of the spectral rank recovers the full RL AVG@32 score on AIME 2024 (40.21% vs. 40.31%) and AIME 2025 (28.96% vs. 29.58%), with identical or slightly higher Pass@32. Strong post-trained reasoning recipes like DeepScaleR and Polaris saturate at 1–10% retained rank, while eliciting reasoning from a less specialized base model requires a larger 30% rank. The compact spectral update translates to storage ratios as low as ~0.58% of total parameters for the 1.5B model, indicating that most of the unconstrained RL update is unnecessary for the measured reasoning gains.

The spectral rewiring matrix M is highly compact, using only about 0.58% of total parameters for the 1.5B model and 0.64% for the 32B model. For the 4B model, which requires a larger projected rank, M still accounts for just 7.75% of parameters. This confirms that reasoning-effective RL updates are concentrated in a small spectral subspace, making them efficient to store and transfer. For the 1.5B and 32B models, the rewiring matrix M occupies only ~0.58% and ~0.64% of total parameters, respectively. The 4B model needs a higher projected rank (10%), resulting in a larger M at ~7.75% of total parameters, still a fraction of the full model size.

Applying a top-1% spectral-rank projection to the coding post-training delta improves six of seven open agentic coding benchmarks, yielding an average relative gain of 2.52% and a maximum improvement of 25.10%. Only MSWE-Bench with the CodeAct agent shows a marginal decline of 0.33%, confirming that the compact spectral update broadly transfers to applied coding workflows. SAR boosts TerminalBench 2.0 by 25.10%, the largest relative gain across all evaluated agentic coding benchmarks. Six out of seven benchmarks improve, with an average increase of 2.52%; the sole regression is a 0.33% drop on MSWE-Bench using the CodeAct agent.

Spectral reasoning rewiring extracts compact, reasoning-effective parameters that, when merged with a code expert, produce models surpassing the best single-domain experts on both math and code benchmarks at 1.5B and 14B scales. This approach reduces cross-domain interference compared to naive weight averaging, enabling stronger and more balanced transfer across domains. SAR merging simultaneously exceeds the best single-domain math and code experts at the 1.5B scale. At 14B, SAR provides balanced cross-domain transfer, improving code performance beyond the code expert while preserving the math expert’s advantage.

The experiments evaluate spectral rewiring for preserving RL reasoning gains on math benchmarks, transferring coding improvements to agentic tasks, and merging reasoning with code experts. Across model scales, projecting RL updates to as little as 1% of the pretrained spectral rank recovers full reasoning performance, showing that improvements concentrate in a compact subspace. This compactness also enables efficient storage and cross-domain transfer, where a top-1% spectral projection boosts most agentic coding benchmarks and merged models surpass single-domain experts while reducing interference.