Reading Comprehension On Race

Métriques

Accuracy

Accuracy (High)

Accuracy (Middle)

Résultats

Résultats de performance de divers modèles sur ce benchmark

Nom du modèle	Accuracy	Accuracy (High)	Accuracy (Middle)	Paper Title	Repository
B10-10-10	85.7	84.4	88.8	Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing	-
Megatron-BERT	89.5	88.6	91.8	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism	-
GPT-3 175B (zero-shot)	-	45.5	-	Language Models are Few-Shot Learners	-
LLaMA 33B (zero-shot)	-	48.3	64.1	LLaMA: Open and Efficient Foundation Language Models	-
LLaMA 65B (zero-shot)	-	51.6	67.9	LLaMA: Open and Efficient Foundation Language Models	-
RoBERTa	83.2	81.3	86.5	RoBERTa: A Robustly Optimized BERT Pretraining Approach	-
Megatron-BERT (ensemble)	90.9	90.0	93.1	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism	-
DeBERTalarge	86.8	-	-	DeBERTa: Decoding-enhanced BERT with Disentangled Attention	-
GPT-3 175B (0-shot)	-	-	58.4	Language Models are Few-Shot Learners	-
ALBERT (Ensemble)	91.4	-	-	Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning	-
BLOOM 176B (one-shot)	-	39.14	52.3	BloombergGPT: A Large Language Model for Finance	-
GPT-NeoX (one-shot)	-	34.33	41.23	BloombergGPT: A Large Language Model for Finance	-
OPT 66B (one-shot)	-	37.02	47.42	BloombergGPT: A Large Language Model for Finance	-
Orca 2-7B	80.79	-	-	Orca 2: Teaching Small Language Models How to Reason	-
PaLM 8B (zero-shot)	-	42.3	57.9	PaLM: Scaling Language Modeling with Pathways	-
XLNet	-	84.0	88.6	XLNet: Generalized Autoregressive Pretraining for Language Understanding	-
Bloomberg GPT (one-shot)	-	41.74	54.32	BloombergGPT: A Large Language Model for Finance	-
PaLM 540B (zero-shot)	-	49.1	68.1	PaLM: Scaling Language Modeling with Pathways	-
LLaMA 7B (zero-shot)	-	46.9	61.1	LLaMA: Open and Efficient Foundation Language Models	-
Orca 2-13B	82.87	-	-	Orca 2: Teaching Small Language Models How to Reason	-

0 of 24 row(s) selected.