Question Answering On Natural Questions

評価指標

評価結果

このベンチマークにおける各モデルのパフォーマンス結果

モデル名	EM	Paper Title	Repository
RankRAG-llama3-70b (Zero-Shot, DPR)	50.0	RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs	-
PaLM 2-S (one-shot)	25.3	PaLM 2 Technical Report
LLaMA 7B (Contriever)	26.07	-	-
Search-o1	34	Search-o1: Agentic Search-Enhanced Large Reasoning Models
Atlas (few-shot, k=64, Wiki-dec-2021+CC index)	42.4	Atlas: Few-shot Learning with Retrieval Augmented Language Models
LLaMA 2 70B (one-shot)	33.0	Llama 2: Open Foundation and Fine-Tuned Chat Models
Mistral 7B (5-shot)	28.8	Mistral 7B
ChatQA-1.5-llama3-70b (Zero-Shot, KILT)	47.0	ChatQA: Surpassing GPT-4 on Conversational QA and RAG	-
LLaMA 65B (few-shot, k=5)	35.0	LLaMA: Open and Efficient Foundation Language Models
GLaM 62B/64E (One-Shot)	26.3	GLaM: Efficient Scaling of Language Models with Mixture-of-Experts	-
Atlas (full, Wiki-dec-2018 index)	64.0	Atlas: Few-shot Learning with Retrieval Augmented Language Models
code-davinci-002 175B + REPLUG (few-shot)	44.7	REPLUG: Retrieval-Augmented Black-Box Language Models
RankRAG-llama3-8b (Zero-Shot, DPR)	46.1	RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs	-
ReAtt	54.7	Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer
LLaMA 65B (few-shot, k=64)	39.9	LLaMA: Open and Efficient Foundation Language Models
RankRAG-llama3-70b (Zero-Shot, KILT)	54.2	RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs	-
DPR	41.5	Dense Passage Retrieval for Open-Domain Question Answering
FiD-KD (full)	54.7	Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Gopher (few-shot, k=64)	28.2	Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Atlas (full, Wiki-dec-2021+CC index)	60.4	Atlas: Few-shot Learning with Retrieval Augmented Language Models

0 of 47 row(s) selected.