One-sentence Summary

Rutgers University, Ant Group, and collaborators propose SLMREC, a knowledge distillation method that empowers small language models for sequential recommendation by pruning redundant intermediate LLM layers, achieving state-of-the-art performance with only $13\%$13% of the parameters of LLM-based models alongside up to $6.6\times$6.6× training and $8.0\times$8.0× inference speedups, and they provide a theoretical justification for why small language models can perform comparably to large ones in this setting.

Key Contributions

• The paper empirically analyzes LLM depth for sequential recommendation, showing through large-scale industry experiments that most intermediate layers are redundant and that models retaining only the remaining layers still deliver strong performance.
• The paper introduces SLMREC, a knowledge distillation framework that compresses the feature extraction behavior of a deep LLM into a compact small language model, and this method remains compatible with other post-training efficiency techniques such as quantization and pruning.
• Results show that SLMREC reaches the best performance while using only 13% of the parameters of LLM-based sequential recommenders, accelerating training by up to 6.6x and inference by up to 8.0x, and the paper also provides a theoretical account of why small language models can match larger ones in this task.

Introduction

Sequential recommendation systems aim to model users' evolving interests from historical interaction data. Traditional methods, ranging from LSTM and GRU to Transformer-based encoders, have hit a performance plateau, with model sizes typically under 0.1 billion parameters. Large Language Model (LLM) based recommenders, both generation-based and embedding-based, have recently driven significant gains, improving benchmarks by nearly 20% over traditional approaches. However, these gains come with a steep cost: LLMRec models often exceed 7 billion parameters, roughly 70 times larger than traditional models, which introduces severe hardware demands and makes real-world deployment inefficient, especially when processing billions of daily traffic logs and handling new items.

A key limitation of prior work is that scaling laws in the recommendation domain have only been tested on models below 1 billion parameters, far smaller than frontier LLMs like GPT-3's 175 billion. Additionally, those studies focused on test loss rather than ranking-based metrics, limiting practical relevance. The authors also note that ID-based item information is not explicitly learned during LLM pretraining, raising questions about whether larger model sizes actually benefit sequential recommendation.

In this work, the authors make an initial attempt to reassess the necessity of LLMs for sequential recommendation. Through experiments on large-scale industry datasets, they find that performance gains from increasing model parameters are inconsistent and that many intermediate layers in LLMs are redundant for this task, similar to findings in NLP. Motivated by these insights, they propose SLMREC, a framework that empowers small language models (under 1 billion parameters) using vanilla knowledge distillation to align representation knowledge and multiple supervision signals to steer the student model toward task-aware hidden representations. SLMREC requires no extra model design elements and works with existing quantization and pruning techniques. Extensive experiments show that SLMREC achieves performance competitive with 7B+ parameter baselines while delivering up to 6.6x training speedup and 8.0x inference speedup. The authors also provide theoretical justification for why smaller models can match larger ones in this setting, and they present results using online knowledge distillation as well.

Dataset

1. Dataset composition and sources:

   • The authors use the Amazon18 dataset, which is an industry-scale dataset.
   • They specifically utilize data from four categories: clothing, movies, music, and sports.

2. Key details for each subset:

   • The interaction data is sourced from user ratings and reviews.
   • Any rating above 3 is interpreted as positive feedback, indicating a user interaction with an item.
   • Timestamps are used to establish the chronological order of actions.
   • Users and items with fewer than 5 associated actions are removed to ensure data density.

3. How the paper uses the data:

   • Each user's historical interaction sequence is split into three segments for training, validation, and testing.
   • The most recent interaction is reserved for testing, the second most recent for validation, and all preceding interactions are used for training.
   • For evaluation, the authors randomly select 999 negative items and combine them with 1 positive item to form the ranking candidates.
   • Metrics used include hit rate (HR@{1, 5, 10}), normalized discounted cumulative gain (NDCG@{5,10}), and Mean Reciprocal Rank (MRR).

4. Other processing details:

   • No specific cropping strategy is mentioned.
   • The data is processed by filtering based on interaction frequency and using timestamps to create a chronological sequence of user actions.

Method

The proposed SLMREC framework aims to distill recommendation knowledge from a large language model (LLM) teacher into a shallower LLM student. Both models follow the general E-LLMRec architecture: a pre-trained ID embedding layer (from models like BERT4Rec, SASRec, or GRU4Rec) maps a fixed-length action sequence $S \in \mathbb{R}^{T}$S∈RT into a low-dimensional sequential representation $\mathbf{S} \in \mathbb{R}^{T \times d_0}$S∈RT×d0​. A linear projection then upscales it to the LLM hidden dimension $d_1$d1​. The LLM decoder processes the combined embeddings of the ID sequence and the prompt text, and the final temporal hidden state $\mathbf{h}_M$hM​ is used as the user representation. A linear layer reduces its dimension back to $d_0$d0​, and the predicted interaction probability is obtained via a dot product with item representations, trained with cross-entropy loss. Unlike typical logits-based knowledge distillation, SLMREC focuses on aligning hidden representations between teacher and student across multiple layers.

The teacher model consists of $M$M stacked decoder layers, while the student has $N$N layers, both sharing the same hidden dimension $d$d. To enable layer-wise feature alignment, the authors partition the layers of both models into blocks: every $m$m teacher layers and every $n$n student layers are grouped together, producing $B = \lfloor M/m \rfloor = \lfloor N/n \rfloor$B=⌊M/m⌋=⌊N/n⌋ blocks. For each block $k \in \{1, \dots, B\}$k∈{1,…,B}, the final temporal hidden representation is extracted from the $k m$km-th teacher layer and the $k n$kn-th student layer, denoted as $\mathbf{h}_t^{(k m)}$ht(km)​ and $\mathbf{h}_s^{(k n)}$hs(kn)​, respectively, with all vectors in $\mathbb{R}^{d}$Rd.

To transfer the feature encoding capability, three complementary regularization terms are introduced. First, a cosine similarity loss aligns the direction of the hidden vectors:

Second, a feature norm regularization minimizes the L2 distance between the corresponding representations:

These two losses guide the student to mimic both the orientation and magnitude of the teacher’s internal features.

In addition, the authors employ a multiple supervision strategy to encourage the student to assimilate recommendation-specific knowledge at intermediate layers. For each representation except the final one, they attach a lightweight adapter $W_a$Wa​ that maps the hidden vector down to the item embedding dimension, and then compute a cross-entropy loss against the ground-truth interaction labels:

where $\hat{p}_t^{(k m)}$p^​t(km)​ is the prediction obtained from the adapter applied to the $k m$km-th teacher layer representation. This auxiliary supervision provides direct learning signals at multiple depths, rather than only at the final layer.

The total training objective for the student integrates the primary cross-entropy loss on the final prediction with the three distillation terms:

where $\lambda_1, \lambda_2, \lambda_3$λ1​,λ2​,λ3​ are hyperparameters that balance the contributions of each term. This composite loss is optimized in an offline distillation setting: the teacher is pre-trained and fixed, while the student and adapters are trained to minimize the objective.

The design is underpinned by a theoretical justification. The authors analyze each transformer layer as a gradient descent step on a denoising objective, and prove that any $K$K-layer attention model can be equivalently represented by a single-layer model that performs one gradient step from the initial embeddings. This implies that a shallower model, when properly guided, can achieve the same feature extraction capability as a deeper model. The feature distillation losses provide such guidance, enabling the student to compress the multi-layer inference process into fewer layers without sacrificing representation quality.

Experiment

The experiments first motivated the design of SLMREC by showing that LLM-based recommenders improve with more layers, yet even a shallow 8-layer model can nearly match a 24-layer one, suggesting redundant capacity and inspiring knowledge distillation. SLMREC, a small model trained via hidden-layer distillation from a larger teacher, outperforms that teacher and rivals a full 32-layer model, while cutting training and inference time to roughly 13% to 14% of the teacher's parameters and running 6.6 and 8.0 times faster respectively. Ablations confirm the contribution of the three knowledge regularizers, and the method also works with online distillation, where teacher and student train simultaneously. The proposed block number of 4 yields the best performance among tested configurations.

The Amazon datasets vary widely in scale, with the clothing category being the largest by users, items, and ratings, while the sports category is the smallest in item count and overall interactions. Data density is very low across all categories, ranging from 0.002% to 0.019%, indicating sparse user-item interactions. The datasets are preprocessed by filtering users and items with fewer than 5 actions and splitting each user's history into training, validation, and test segments. Clothing has the most users, items, and ratings, while sports has the fewest items and ratings. All datasets are extremely sparse, with density under 0.02%. The music and movie categories show similar density, both higher than clothing and sports.

The table compares recommendation accuracy across multiple models on Cloth and Movie datasets. The analysis shows that LLM-based methods outperform traditional baselines, and the proposed SLMREC, despite having fewer layers, achieves better performance than its larger teacher through knowledge distillation. SLMREC surpasses the teacher model E4SRec by roughly 8% through hidden-layer knowledge distillation. Smaller student models with effective training can rival or exceed larger language models in sequential recommendation. LLM-based methods show substantial gains over traditional sequential recommendation methods.

在Music和Sport数据集上，不同传统序列推荐模型的性能存在明显差异，但整体指标偏低，表明推荐任务具有挑战性。各模型在HR@5和NDCG@5上的表现远优于HR@1，说明排序能力有所提升。 在Music上，BERT4Rec取得最高的HR@1，而Caser表现最差。 在Sport上，GRU4Rec和LightSANs的HR@1相对较高，Caser仍然最低。 所有模型在HR@5和NDCG@5上的数值显著高于HR@1，体现出更好的排序效果。

The ablation study shows that adding knowledge regularizers to SLMREC consistently improves performance across both Cloth and Movie domains. The full combination of cosine, norm, and masked language modeling regularizers yields the best overall metrics, especially in ranking quality. Each regularizer contributes by aligning intermediate representations and incorporating domain knowledge. Combining all three regularizers achieves the highest NDCG@5 and MRR on both Cloth and Movie domains. Adding norm alignment on top of cosine regularization improves HR@5 and NDCG@5, while the masked language modeling regularizer boosts HR@1 in the Cloth domain.

The efficiency comparison shows that SLMREC is markedly more efficient than the other LLM-based methods, with substantially shorter training and inference times and far fewer parameters. Open-P5 has a reasonable training time but extremely long inference due to its generative approach, while E4SRec is intermediate in time but still much heavier than SLMREC. Overall, SLMREC achieves a major efficiency gain over E4SRec while maintaining competitive performance. SLMREC trains about 6.6 times faster and infers about 8 times faster than E4SRec. SLMREC uses only about 13% of E4SRec's training parameters and 14% of its inference parameters. Open-P5's inference time is extremely high, taking thousands of hours, because it generates a large candidate set during ranking.

The evaluation on sparse Amazon datasets shows that LLM-based methods consistently outperform traditional sequential recommendation baselines, with the proposed SLMREC, a lightweight student model, surpassing its larger teacher E4SRec through knowledge distillation. Ablation studies confirm that combining cosine, norm, and masked language modeling regularizers yields the best ranking quality across domains. Furthermore, SLMREC is markedly more efficient than other LLM-based approaches, requiring far fewer parameters and significantly shorter training and inference times while maintaining competitive accuracy. Overall, the findings highlight that compact student models with effective distillation and regularization can achieve state-of-the-art performance in sparse, real-world recommendation settings.