OpenWebMath Open Web Mathematics Training Dataset

OpenWebMath is a dataset containing high-quality mathematical text from most of the Internet. It is filtered and extracted from more than 200B HTML files on Common Crawl, resulting in a set of 6.3 million documents containing a total of 14.7B tokens. OpenWebMath is intended for pre-training andFine-tuningLarge language models.