| W2V2-L-LL60K (+ in-domain LM) | 11.8 | 13.8 | 12.0 | 12.5 | SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech | - |
| W2V2-L-LL60K (+ TED-LIUM 3 LM) | 9.1 | 10.8 | 9.1 | 9.3 | SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech | - |
| W2V2-B-LS960 (+ in-domain LM) | 15.2 | 18.2 | 14.6 | 15.2 | SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech | - |
| W2V2-B-LS960 (+ TED-LIUM 3 LM) | 13.2 | 15.8 | 12.0 | 12.2 | SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech | - |