Reading Comprehension On Race

Métriques

Accuracy

Accuracy (High)

Accuracy (Middle)

Résultats

Résultats de performance de divers modèles sur ce benchmark

Nom du modèle	Accuracy	Accuracy (High)	Accuracy (Middle)	Paper Title	Repository
B10-10-10	85.7	84.4	88.8	Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
Megatron-BERT	89.5	88.6	91.8	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
GPT-3 175B (zero-shot)	-	45.5	-	Language Models are Few-Shot Learners
LLaMA 33B (zero-shot)	-	48.3	64.1	LLaMA: Open and Efficient Foundation Language Models
LLaMA 65B (zero-shot)	-	51.6	67.9	LLaMA: Open and Efficient Foundation Language Models
RoBERTa	83.2	81.3	86.5	RoBERTa: A Robustly Optimized BERT Pretraining Approach
Megatron-BERT (ensemble)	90.9	90.0	93.1	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
DeBERTalarge	86.8	-	-	DeBERTa: Decoding-enhanced BERT with Disentangled Attention
GPT-3 175B (0-shot)	-	-	58.4	Language Models are Few-Shot Learners
ALBERT (Ensemble)	91.4	-	-	Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning	-
BLOOM 176B (one-shot)	-	39.14	52.3	BloombergGPT: A Large Language Model for Finance	-
GPT-NeoX (one-shot)	-	34.33	41.23	BloombergGPT: A Large Language Model for Finance	-
OPT 66B (one-shot)	-	37.02	47.42	BloombergGPT: A Large Language Model for Finance	-
Orca 2-7B	80.79	-	-	Orca 2: Teaching Small Language Models How to Reason	-
PaLM 8B (zero-shot)	-	42.3	57.9	PaLM: Scaling Language Modeling with Pathways
XLNet	-	84.0	88.6	XLNet: Generalized Autoregressive Pretraining for Language Understanding
Bloomberg GPT (one-shot)	-	41.74	54.32	BloombergGPT: A Large Language Model for Finance	-
PaLM 540B (zero-shot)	-	49.1	68.1	PaLM: Scaling Language Modeling with Pathways
LLaMA 7B (zero-shot)	-	46.9	61.1	LLaMA: Open and Efficient Foundation Language Models
Orca 2-13B	82.87	-	-	Orca 2: Teaching Small Language Models How to Reason	-

0 of 24 row(s) selected.