Long Range Modeling On Lra
Metrics
Avg
Image
ListOps
Pathfinder
Retrieval
Text
Results
Performance results of various models on this benchmark
Model Name | Avg | Image | ListOps | Pathfinder | Retrieval | Text | Paper Title | Repository |
|---|---|---|---|---|---|---|---|---|
| Linear Trans. | 50.55 | 42.34 | 16.13 | 75.3 | 53.09 | 65.9 | Long Range Arena: A Benchmark for Efficient Transformers | |
| Sparse Trans. | 51.24 | 44.24 | 17.07 | 71.71 | 59.59 | 63.58 | Long Range Arena: A Benchmark for Efficient Transformers | |
| Transformer | 54.39 | 42.44 | 36.37 | 71.4 | 57.46 | 64.27 | Long Range Arena: A Benchmark for Efficient Transformers | |
| Converter | 75.94 | 61.02 | 60.38 | 88.43 | 83.41 | 86.44 | Converting Transformers into DGNNs Form | |
| Performer | 51.41 | 42.77 | 18.01 | 77.05 | 53.82 | 65.4 | Long Range Arena: A Benchmark for Efficient Transformers | |
| S4 | 86.09 | 88.65 | 59.60 | 94.20 | 90.90 | 86.82 | How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections | |
| S5 | 87.46 | 88 | 62.15 | 95.33 | 91.4 | 89.31 | Simplified State Space Layers for Sequence Modeling |
0 of 7 row(s) selected.