Reading Comprehension On Race

评估指标

Accuracy

Accuracy (High)

Accuracy (Middle)

评测结果

各个模型在此基准测试上的表现结果

模型名称	Accuracy	Accuracy (High)	Accuracy (Middle)	Paper Title	Repository
B10-10-10	85.7	84.4	88.8	Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
Megatron-BERT	89.5	88.6	91.8	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
GPT-3 175B (zero-shot)	-	45.5	-	Language Models are Few-Shot Learners
LLaMA 33B (zero-shot)	-	48.3	64.1	LLaMA: Open and Efficient Foundation Language Models
LLaMA 65B (zero-shot)	-	51.6	67.9	LLaMA: Open and Efficient Foundation Language Models
RoBERTa	83.2	81.3	86.5	RoBERTa: A Robustly Optimized BERT Pretraining Approach
Megatron-BERT (ensemble)	90.9	90.0	93.1	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
DeBERTalarge	86.8	-	-	DeBERTa: Decoding-enhanced BERT with Disentangled Attention
GPT-3 175B (0-shot)	-	-	58.4	Language Models are Few-Shot Learners
ALBERT (Ensemble)	91.4	-	-	Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning	-
BLOOM 176B (one-shot)	-	39.14	52.3	BloombergGPT: A Large Language Model for Finance
GPT-NeoX (one-shot)	-	34.33	41.23	BloombergGPT: A Large Language Model for Finance
OPT 66B (one-shot)	-	37.02	47.42	BloombergGPT: A Large Language Model for Finance
Orca 2-7B	80.79	-	-	Orca 2: Teaching Small Language Models How to Reason	-
PaLM 8B (zero-shot)	-	42.3	57.9	PaLM: Scaling Language Modeling with Pathways
XLNet	-	84.0	88.6	XLNet: Generalized Autoregressive Pretraining for Language Understanding
Bloomberg GPT (one-shot)	-	41.74	54.32	BloombergGPT: A Large Language Model for Finance
PaLM 540B (zero-shot)	-	49.1	68.1	PaLM: Scaling Language Modeling with Pathways
LLaMA 7B (zero-shot)	-	46.9	61.1	LLaMA: Open and Efficient Foundation Language Models
Orca 2-13B	82.87	-	-	Orca 2: Teaching Small Language Models How to Reason	-

0 of 24 row(s) selected.