Last Translation Benchmark is a benchmark dataset for evaluating machine translation models, released by Vilém Zouhar et al. in 2026. The associated paper is titled "Last Translation Benchmark", aiming to evaluate the capabilities of current machine translation models on complex translation tasks and to identify specific failure cases in real-world scenarios.

The dataset contains 3,456 high-difficulty translation samples, covering multiple languages and language pairs, with data sourced from human-written and peer-reviewed translation examples. The samples include multimodal inputs such as text, images, audio, and video, and provide human-defined verification rules for each sample.

Dataset Composition

• source_text: Source text;
• source_lang: Source language;
• target_lang: Target language;
• source_instructions: Specific requirements for the translation task;
• source_media: Optional image, audio, or video data;
• translations: Translation results generated by different models or humans;
• verification_rules: Verification rules used to determine translation correctness;
• linguistics: Linguistic annotations describing translation challenges;
• attribution: Data source information;
• tags: Data subset to which the data belongs.

Citation

@misc{zouhar2026translationbenchmark,
  title={Last Translation Benchmark},
  author={LTB},
  year={2026},
  eprint={2609.04173},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2609.04173}
}