HellaSwag Large Model Common Sense Reasoning Dataset

*This dataset supports online use.Click here to jump.

The HellaSwag dataset is a new challenge dataset for testing commonsense natural language inference (commonsense NLI). The dataset was launched by the University of Washington and Allen AI in 2019. It aims to explore the performance of deep pre-trained models in commonsense reasoning by building a dataset that is challenging to existing state-of-the-art models.HellaSwag: Can a Machine Really Finish Your Sentence?" has been accepted by ACL 2019.

The HellaSwag dataset contains 70,000 questions that, despite being very simple for humans (over 95% accuracy), even the most advanced models have difficulty achieving near-human performance (about 48% accuracy). The dataset was constructed using an adversarial filtering (AF) approach that iteratively selects machine-generated incorrect answers using a series of discriminators to increase the difficulty of the dataset. The creation of HellaSwag reveals the inner workings of deep pre-trained models and provides a new direction for NLP research, where benchmarks co-evolve with evolving state-of-the-art models in an adversarial manner to provide more challenging tasks.