Language Modelling On Salmon
評価指標
Background (Domain) Consistency
Background (Random) Consistency
Background Alignment
Gender Consistency
Room Consistency
Sentiment Alignment
Sentiment Consistency
Speaker Consistency
評価結果
このベンチマークにおける各モデルのパフォーマンス結果
比較表
| モデル名 | Background (Domain) Consistency | Background (Random) Consistency | Background Alignment | Gender Consistency | Room Consistency | Sentiment Alignment | Sentiment Consistency | Speaker Consistency |
|---|---|---|---|---|---|---|---|---|
| spirit-lm-interleaved-spoken-and-written | 55.0 | 64.0 | 59.5 | 85.0 | 54.5 | 52.0 | 73.5 | 81.0 |
| last-language-model-aware-speech-tokenization | 55.5 | 60.5 | 54.5 | 70.5 | 61.0 | 51.5 | 64.0 | 63.0 |
| textually-pretrained-speech-language-models | 55.5 | 60.5 | 56.5 | 69.5 | 59.0 | 53.0 | 61.5 | 69.0 |
| textually-pretrained-speech-language-models | 54.0 | 61.5 | 56.5 | 68.0 | 59.0 | 51.5 | 59.0 | 69.5 |
| text-free-prosody-aware-generative-spoken | 57.0 | 66.0 | 53.5 | 88.5 | 53.5 | 55.5 | 40.5 | 83.0 |
| last-language-model-aware-speech-tokenization | 56.0 | 61.0 | 53.0 | 68.5 | 62.5 | 53.5 | 65.0 | 64.5 |
| textually-pretrained-speech-language-models | 55.0 | 60.5 | 54.5 | 70.0 | 62.0 | 51.5 | 61.5 | 71.0 |
| spirit-lm-interleaved-spoken-and-written | 53.5 | 55.5 | 51.5 | 67.0 | 54.5 | 48.0 | 54.5 | 69.5 |