Long Range Modeling On Scrolls
Metriken
Avg.
CNLI
GovRep
Nrtv
QALT EM-T/H
QMSum
Qspr
SumScr
Ergebnisse
Leistungsergebnisse verschiedener Modelle zu diesem Benchmark
Modellname | Avg. | CNLI | GovRep | Nrtv | QALT EM-T/H | QMSum | Qspr | SumScr | Paper Title | Repository |
|---|---|---|---|---|---|---|---|---|---|---|
| LongT5 Base | 38.6 | 85.6 | 57.7 / 30.0 / 31.4 | 23.0 | 37.9 / 36.6 | 33.9 / 11.0 / 22.8 | 46.6 | 34.8 / 9.6 / 21.1 | LongT5: Efficient Text-To-Text Transformer for Long Sequences | |
| BART-LS | 39.76 | 87.1 | 59.4 / 29.8 / 30.8 | 26.2 | 37.8 / 34.0 | 35.1 / 11.0 / 22.0 | 48.7 | 37.7 / 10.2 / 21.5 | Adapting Pretrained Text-to-Text Models for Long Text Sequences | |
| BART-large SLED | 37.99 | 87.3 | 57.5 / 26.3 / 27.4 | 24.1 | 34.8 / 34.8 | 34.2 / 11.0 / 22.0 | 46.9 | 35.2 / 8.7 / 19.4 | Efficient Long-Text Understanding with Short-Text Models | |
| LongT5 XL | 42.53 | 88.2 | 61.1 / 32.3 / 33.7 | 29.3 | 46.0 / 42.1 | 34.9 / 11.8 / 23.5 | 53.1 | 35.8 / 9.6 / 21.1 | LongT5: Efficient Text-To-Text Transformer for Long Sequences | |
| LongT5 Large | 41.03 | 87.3 | 61.3/32.2/33.8 | 27.2 | 40.6 / 38.6 | 35.1 / 12.0 / 23.3 | 52.3 | 60.3 / 31.1 / 32.8 | LongT5: Efficient Text-To-Text Transformer for Long Sequences | |
| Naive | 19.35 | 66 | 45.3 / 17.9 / 20.8 | 1.5 | 25.2 / 26.1 | 14.2 / 2.0 / 9.3 | 3.4 | 19.6 / 1.8 / 11.0 | SCROLLS: Standardized CompaRison Over Long Language Sequences | |
| UL2 20B | - | 88.7 | - | - | - | - | - | - | UL2: Unifying Language Learning Paradigms | |
| BART Base | 29.01 | 77.4 | 47.9 / 18.6 / 22.7 | 15.4 | 26.0 / 25.9 | 30.2 / 8.7 / 20.7 | 26.3 | 27.2 / 4.9 / 16.7 | SCROLLS: Standardized CompaRison Over Long Language Sequences | |
| PEGASUS-X-Base | - | - | 59.3 / 29.3 / 30.9 | - | - | 32.9 / 9.8 / 21.4 | - | 35.0 / 8.9 / 20.4 | Investigating Efficiently Extending Transformers for Long Input Summarization | |
| LED Base | - | - | - | - | - | - | - | - | SCROLLS: Standardized CompaRison Over Long Language Sequences | |
| CoLT5 XL | 43.51 | 88.4 | 61.3/32.2/33.8 | 31.1 | 48.1/43.8 | 36.2/12.9/24.3 | 53.9 | 36.4/10.2/21.7 | CoLT5: Faster Long-Range Transformers with Conditional Computation | - |
| UL2 | 37.87 | - | 53.6 / 26.1 / 28.8 | 24.2 | 45.8 / 40.7 | 31.1 / 8.5 / 20.4 | 37.6 | 32.9 / 7.8 / 19.4 | UL2: Unifying Language Learning Paradigms | |
| PEGASUS-X | - | - | 60.3 / 30.0 / 31.5 | - | - | 33.2 / 9.6 / 21.6 | - | 35.7 / 9.1 / 20.6 | Investigating Efficiently Extending Transformers for Long Input Summarization |
0 of 13 row(s) selected.