TMMLU+ Traditional Chinese Large-Scale Multi-Task Language Understanding Dataset

TMMLU+ stands for Large scale traditional Chinese massive multitask language understanding, which is a Chinese large-scale multi-task language understanding dataset.It is a multiple-choice question-answering dataset covering 66 topics, ranging from elementary to professional levels.

The TMMLU+ dataset is significantly larger than its predecessor TMMLU 6 times larger, with a more balanced theme.TMMLU+ includes benchmark results from closed-source models and 20 open-source Chinese large language models.The parameters of these models range from 1.8 B to 72 B. The benchmark results show that the Traditional Chinese variants still lag behind the models trained on the main Simplified Chinese model.