Question Answering On Drop Test

評価結果

このベンチマークにおける各モデルのパフォーマンス結果

モデル名	F1	Paper Title	Repository
QDGAT (ensemble)	88.38	Question Directed Graph Attention Network for Numerical Reasoning over Text	-
PaLM 2 (few-shot)	85.0	PaLM 2 Technical Report
GPT-3 175B (few-shot, k=32)	36.5	Language Models are Few-Shot Learners
GPT-4 (few-shot, k=3)	80.9	GPT-4 Technical Report
NeRd	81.71	Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension	-
NumNet	67.97	NumNet: Machine Reading Comprehension with Numerical Reasoning
Orca 2-7B	60.26	Orca 2: Teaching Small Language Models How to Reason	-
GPT 3.5 (few-shot, k=3)	64.1	GPT-4 Technical Report
Orca 2-13B	57.97	Orca 2: Teaching Small Language Models How to Reason	-
POET	87.6	Reasoning Like Program Executors
BERT	32.7	DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
BERT+Calculator (ensemble)	81.78	Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension	-
NAQA Net	47.01	DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
GenBERT (+ND+TD)	72.4	Injecting Numerical Reasoning Skills into Language Models
MTMSN Large	79.88	A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning
TASE-BERT	80.7	A Simple and Effective Model for Answering Multi-span Questions

0 of 16 row(s) selected.