Open Domain Dialog On Kilt Wizard Of
評価指標
F1
KILT-F1
KILT-RL
R-Prec
ROUGE-L
Recall@5
評価結果
このベンチマークにおける各モデルのパフォーマンス結果
モデル名 | F1 | KILT-F1 | KILT-RL | R-Prec | ROUGE-L | Recall@5 | Paper Title | Repository |
|---|---|---|---|---|---|---|---|---|
| chriskuei | 0.0 | 0.0 | 0.0 | 64.79 | 0.0 | 82.15 | - | - |
| bart-base | 14.82 | 0.0 | 0.0 | 0.0 | 13.35 | 0.0 | - | - |
| aa_evalai | 17.3 | 0.0 | 0.0 | 0.0 | 15.93 | 0.0 | - | - |
| GENRE | 0.0 | 0.0 | 0.0 | 62.88 | 0.0 | 77.74 | - | - |
| multitask | 3.09 | 2.18 | 2.04 | 55.71 | 2.92 | 75.59 | - | - |
| KGI | 18.57 | 11.79 | 10.36 | 55.37 | 16.36 | 78.45 | - | - |
| Sphere | 17.28 | 0.0 | 0.0 | 0.0 | 15.71 | 0.0 | - | - |
| intersect | 18.34 | 11.63 | 10.45 | 57.55 | 16.65 | 78.96 | - | - |
| TABi | 0.0 | 0.0 | 0.0 | 59.11 | 0.0 | 69.1 | - | - |
| Hindsight | 19.19 | 13.39 | 11.92 | 56.08 | 17.06 | 74.27 | - | - |
| Wikipedia | 15.66 | 7.57 | 6.55 | 41.54 | 13.94 | 68.25 | - | - |
| BART | 12.86 | 0.0 | 0.0 | 0.0 | 11.77 | 0.0 | - | - |
| T5-base | 13.53 | 0.0 | 0.0 | 0.0 | 12.4 | 0.0 | KILT: a Benchmark for Knowledge Intensive Language Tasks | |
| Multitask DPR + BART | 15.12 | 6.96 | 5.91 | 41.06 | 13.27 | 67.13 | - | - |
| Routing Transformer, c-REALM | 12.15 | 4.8 | 4.41 | 39.06 | 11.42 | 51.63 | - | - |
| Re2G | 18.9 | 12.98 | 11.39 | 60.1 | 16.76 | 79.98 | Re2G: Retrieve, Rerank, Generate | |
| multi-task small | 13.75 | 0.0 | 0.0 | 0.0 | 12.81 | 0.0 | - | - |
| TransMemNet | 11.85 | 2.2 | 1.85 | 18.35 | 10.11 | 18.35 | - | - |
| RAG | 13.11 | 8.75 | 7.59 | 57.75 | 11.57 | 74.61 | - | - |
| BART + DPR | 15.19 | 4.37 | 3.71 | 25.46 | 13.23 | 51.19 | - | - |
0 of 21 row(s) selected.