Language Modelling On The Pile

Metriken

Bits per byte

Ergebnisse

Leistungsergebnisse verschiedener Modelle zu diesem Benchmark

Modellname	Bits per byte	Paper Title	Repository
Test-Time Fine-Tuning with SIFT + Llama-3.2 (3B)	0.557	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs	-
Transformer 125M	-	Hungry Hungry Hippos: Towards Language Modeling with State Space Models
Jurassic-1	0.65	GLM-130B: An Open Bilingual Pre-trained Model
Llama-3.2 3B	0.640	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs	-
Phi-3 3.8B	0.679	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs	-
GPT-3 Davinci 175B (pre-trained)	0.7177	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Gemma-2 9B	0.670	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs	-
Gemma-2 27B	0.629	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs	-
OPT 1.3B	-	Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Gemma-2 2B	0.721	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs	-
Test-Time Fine-Tuning with SIFT + GPT-2 (124M)	0.862	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs	-
Phi-3 7B	0.678	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs	-
GPT-Neo 2.7B	-	Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Hybrid H3 125M	-	Hungry Hungry Hippos: Towards Language Modeling with State Space Models
Larger Transformer 771M (fine-tuned)	-	Need a Small Specialized Language Model? Plan Early!	-
GPT-Neo 1.3B	-	Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Larger Transformer 771M (pre-trained)	-	Need a Small Specialized Language Model? Plan Early!	-
Test-Time Fine-Tuning with SIFT + GPT-2 (774M)	0.762	Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs	-
GPT-2 Medium 355M (pre-trained)	1.0928	The Pile: An 800GB Dataset of Diverse Text for Language Modeling
GPT-2 XL 1.5B (pre-trained)	1.0468	The Pile: An 800GB Dataset of Diverse Text for Language Modeling

0 of 39 row(s) selected.