Slot Filling On Kilt Zero Shot Re

Accuracy

KILT-AC

KILT-F1

R-Prec

Recall@5

評価結果

このベンチマークにおける各モデルのパフォーマンス結果

							Paper Title
single ngram	74.63	79.66	73.2	78.12	97.99	99.34	-
Wikipedia	73.96	78.43	67.2	70.99	89.63	97.87	-
KGI_1	72.55	77.05	72.31	76.69	98.49	99.23	-
MetaRAG	71.61	76.6	71.1	75.86	95.81	96.64	-
KGI_0 (reupload)	68.97	74.47	68.32	73.45	94.18	95.19	-
Multitask DPR + BART	57.95	63.75	50.64	55.44	80.91	93.05	-
DensePhrases	47.42	54.75	41.34	46.79	57.43	60.47	Learning Dense Representations of Phrases at Scale
10k	47.42	54.75	41.34	46.79	57.43	60.47	-
RAG	44.74	49.95	36.83	39.91	53.73	59.52	-
Sphere	36.55	44.94	0.0	0.0	0.0	0.0	-
Coop. Distil Bert	36.23	40.34	34.13	37.22	61.34	63.85	-
BART + DPR	30.43	34.47	18.91	20.32	28.9	39.21	-
bart-base+ssm	11.22	16.79	0.0	0.0	0.0	0.0	-
BART	9.14	12.21	0.0	0.0	0.0	0.0	-
T5-base	9.02	13.52	0.0	0.0	0.0	0.0	KILT: a Benchmark for Knowledge Intensive Language Tasks
BERT + DPR	6.93	37.28	4.47	27.09	40.11	40.11	-
multi-task small	3.38	10.44	0.0	0.0	0.0	0.0	-
GENRE	0.02	2.1	0.0	1.85	95.81	97.83	-
chriskuei	0.0	0.0	0.0	0.0	98.27	98.89	-
TABi	0.0	0.0	0.0	0.0	96.15	98.71	-

0 of 21 row(s) selected.