One-sentence Summary

ByteDance Research proposes G-DIG, a gradient-based instruction data selection method for machine translation that uses Influence Functions with a small seed dataset to identify high-quality examples and clusters on gradients with resampling to maximize data diversity, demonstrating superiority on WMT22 and FLORES benchmarks.

Key Contributions

• G-DIG is a gradient-based data selection framework for LLM instruction finetuning in machine translation, selecting high-quality and diverse examples without relying on external scoring models.
• The method identifies examples with beneficial influence by combining influence functions with a small high-quality seed dataset, and increases diversity by clustering candidate gradients and resampling from clusters.
• Experiments on WMT22 and FLORES show improved translation performance and better LLM alignment in human evaluation, and analysis supports the effectiveness and generalization of the method.

Introduction

Large language models rely on instruction finetuning to improve alignment and task performance, especially in machine translation. Prior data selection methods often prioritize scale or depend on external scoring models, limiting how well they capture the target model’s own learning behavior and often balancing data quality and diversity only indirectly. The authors propose G-DIG, a gradient-based selection method that uses influence functions to identify training examples that positively affect high-quality seed data, and then increases diversity by clustering and resampling data based on their gradient-level influence on the model.

Dataset

• The authors construct the translation finetuning data as instruction response pairs from parallel texts. Each example uses the prompt template `Translate the following text into {trg_lang}.

Text: “{src_text}”, and the response is the corresponding target language text. The main constructed field is the target language label, trg_lang`. The paper does not describe a cropping strategy, so data selection appears to operate on full source and target parallel segments.

• The data has two main subsets. First, a manually curated seed set contains 256 parallel texts drawn randomly from a validation set and revised by human translators. This seed set serves as the quality benchmark for selecting useful training examples. Second, the candidate pool consists of publicly available WMT22 parallel texts. The paper does not state the full size of the WMT22 candidate pool.

• The authors use the seed set with influence based filtering to identify high quality examples from the candidate pool. A candidate example is retained if it has beneficial influence on every seed example, meaning it reduces the model’s loss on all 256 human revised validation translations.

• After quality filtering, the authors further process the selected data for diversity. They use gradients related to the response loss, average them over tokens, reduce their dimension with Random Projection to 400 dimensions, cluster the examples into 512 clusters using Kmeans, and sample uniformly from the clusters to build a diverse training set.

Method

The authors design a two-stage data selection pipeline for instruction finetuning. The first stage selects examples that improve performance on a small high-quality seed set, and the second stage preserves coverage across diverse translation patterns. Refer to the framework diagram.

The framework begins with a candidate pool $\mathcal{D}_{raw}$Draw​ and produces final supervised finetuning data $\mathcal{D}_{sft}$Dsft​. The first module, high-quality data selection, uses influence functions to estimate whether a candidate example improves the model on seed examples. The second module, diversity enhancement, represents each selected example by its gradient pattern, clusters these representations, and samples uniformly from the clusters to avoid overrepresenting one type of translation behavior.

For high-quality selection, the authors first assume a finetuning objective of the form

where $z_i = (x_i, y_i)$zi​=(xi​,yi​) is a prompt-response pair and $\boldsymbol{\theta}$θ denotes the model parameters. The loss is the language modeling loss on the response only:

To measure how one training example affects the model on another example, the method computes the influence score

where $\mathbf{H}_{\boldsymbol{\theta}^{*}}$Hθ∗​ is the Hessian of the training objective at the optimal parameters. A negative influence score indicates that training on $z_m$zm​ reduces the loss on $z_t$zt​, so $z_m$zm​ is treated as beneficial for producing high-quality outputs.

The practical implementation approximates this computation to make it scalable. The authors use gradients from selected multilayer perceptron layers, specifically layers 3, 6, 9, 12, 15, 18, 21, 24, 27, and 30, and average token-level gradients within each example to form a fixed-size representation. They also use Kronecker-Factored Approximate Curvature, or KFAC, to approximate the Hessian and reduce memory cost.

The high-quality selection step then chooses candidates that improve all seed examples. The seed set $\mathcal{D}_{seed}$Dseed​ is a small manually curated collection of high-quality translation pairs. A candidate $z_m$zm​ is retained only if it has a beneficial influence on every seed example:

This produces a reduced pool $\mathcal{D}_{Q}$DQ​ of examples that are both relevant and positively associated with high-quality generation behavior.

After this filtering step, the method further controls diversity. For each remaining example, the authors compute the gradient of the response loss with respect to the final multilayer perceptron layer and average the token-level gradients. Euclidean distance is then used to compare these gradient representations. Examples with similar gradients are interpreted as having similar training effects, while distant gradient vectors correspond to more distinct influence patterns.

To make clustering tractable, the gradient vectors are reduced to 400 dimensions with random projection. The projected representations are clustered with K-means, with 512 clusters used in the implementation. Finally, examples are sampled uniformly from each cluster and collected as the final instruction finetuning set. This process preserves the quality filter while ensuring that the resulting data covers multiple translation patterns rather than overfitting to a narrow set of training effects.

Experiment

Experiments on Zh-En and De-En machine translation use 7B LLMs, large candidate pools, WMT22/Flores-101 evaluation, BLEU/BLEURT/COMET, and baselines including random selection, reward-model selection, TIM, and larger open models. Main results validate that G-DIG improves robustly across data budgets, surpasses baselines, reaches competitive SOTA-like quality, and yields better human-rated translations than random selection. Ablations further validate that diversity enhancement is especially important under small budgets and outperforms embedding-based selection, while high-quality selection beats reward-model filtering and hyperparameter settings favor moderate seed size and many clusters.

The comparison covers Chinese to English and German to English translation results across several automatic metrics. G-DIG generally outperforms the baseline methods and is competitive with or stronger than the cited state of the art systems, including comparisons against both comparable and much larger reference models. The variant without diversity is not uniformly worse, as it still shows a clear BLEU advantage over a reward model baseline for German to English. G-DIG surpasses the reported baselines in nearly all cases across the compared benchmarks. Its 7B results are comparable to TIM-7B and better than the listed Bayling, BigTranslate, and NLLB reference systems.

Human evaluation compares translations from a random subset model with those from the proposed G-DIG subset model on sampled Zh-En and De-En sentences. Judges assign higher ratings to the G-DIG translations and prefer them more often than they prefer the random-subset translations. These results support the claim that the selected training data produces translations that are better aligned with human preferences. G-DIG translations receive higher average human ratings than random-subset translations in both Zh-En and De-En. Human judges select G-DIG translations as better more often than they select random translations as better. A substantial share of evaluations are ties, but the overall preference still favors the proposed method over random selection.

The examples compare selected translations with randomly selected training texts. The selected Chinese to English and German to English pairs are faithful, natural, and well formed, while the random pairs show omissions, extra details, or incomplete target statements. This supports the claim that the method chooses more reliable parallel training data. Selected pairs preserve source meaning and produce readable target sentences Random pairs can omit important details or include content not present in the source Quality differences are visible across both Chinese English and German English examples

The experiments evaluate the proposed method for Chinese-English and German-English translation using automatic metrics, human ratings, and qualitative examples. The method generally outperforms baseline approaches, remains competitive with strong reference systems, and shows that the diversity component provides additional gains in most settings. Human evaluation further indicates that translations trained on selected data are preferred over those trained on random data, while qualitative examples show that selected pairs are more faithful and natural than random pairs.