Task 1 Grouping On Ocw
Métriques
Wasserstein Distance (WD)
# Correct Groups
# Solved Walls
Adjusted Mutual Information (AMI)
Adjusted Rand Index (ARI)
Fowlkes Mallows Score (FMS)
Résultats
Résultats de performance de divers modèles sur ce benchmark
Tableau comparatif
| Nom du modèle | Wasserstein Distance (WD) | # Correct Groups | # Solved Walls | Adjusted Mutual Information (AMI) | Adjusted Rand Index (ARI) | Fowlkes Mallows Score (FMS) |
|---|---|---|---|---|---|---|
| pre-training-of-deep-bidirectional-protein | 89.5 ± .4 | 22 ± 2 | 0 ± 0 | 8.1 ± .4 | 6.4 ± .3 | 25.1 ± .2 |
| gpt-4-technical-report-1 | 82.5 | 114 | 0 | 21.6 | 18.4 | 34.0 |
| gpt-4-technical-report-1 | 82.3 | 123 | 0 | 21.2 | 18.2 | 34.4 |
| gpt-4-technical-report-1 | 73.4 | 262 | 4 | 33.5 | 29.7 | 43.7 |
| gpt-4-technical-report-1 | 81.2 | 137 | 2 | 24.0 | 20.4 | 36.1 |
| text-embeddings-by-weakly-supervised | 84.4 ± .7 | 76 ± 5 | 0 ± 0 | 18.5 ± .6 | 15.4 ± .5 | 32.3 ± .4 |
| learning-word-vectors-for-157-languages | 85.5 ± .5 | 62 ± 3 | 0 ± 0 | 15.8 ± .3 | 13.0 ± .2 | 30.4 ± .2 |
| learning-word-vectors-for-157-languages | 84.2 ± .5 | 80 ± 4 | 0 ± 0 | 18.4 ± .4 | 15.2 ± .3 | 32.1 ± .3 |
| text-embeddings-by-weakly-supervised | 83.8 ± .6 | 89 ± 6 | 1 ± 0 | 19.5 ± .4 | 16.3 ± .4 | 33.1 ± .3 |
| pre-training-of-deep-bidirectional-protein | 88.3 ± .5 | 33 ± 2 | 0 ± 0 | 10.3 ± .3 | 8.2 ± .3 | 26.5 ± .2 |
| gpt-4-technical-report-1 | 72.9 | 269 | 7 | 32.8 | 29.1 | 43.4 |
| large-language-models-are-fixated-by-red-1 | - | 1405 | 285 | - | - | - |
| gpt-4-technical-report-1 | 80.6 | 149 | 2 | 25.4 | 22.0 | 37.3 |
| glove-global-vectors-for-word-representation | 84.9 ± .4 | 68 ± 4 | 0 ± 0 | 17.6 ± .4 | 14.4 ± .3 | 31.5 ± .3 |
| gpt-4-technical-report-1 | 75.8 | 239 | 6 | 30.7 | 27.2 | 41.5 |
| deep-contextualized-word-representations | - | 55 ± 4 | 0 ± 0 | 14.5 ± .4 | 11.8 ± .4 | 29.5 ± .3 |
| distilbert-a-distilled-version-of-bert | - | 49 ± 4 | 0 ± 0 | 14.0 ± .3 | 11.3 ± .3 | 29.1 ± .2 |
| gpt-4-technical-report-1 | 80.9 | 140 | 0 | 24.7 | 21.3 | 36.8 |
| gpt-4-technical-report-1 | 73.6 | 249 | 3 | 32.3 | 28.5 | 42.8 |
| roberta-a-robustly-optimized-bert-pretraining | - | 29 ± 3 | 0 ± 0 | 9.4 ± .4 | 8.4 ± .3 | 26.7 ± .2 |
| gpt-4-technical-report-1 | 73.7 | 272 | 5 | 33.6 | 29.9 | 43.9 |
| mpnet-masked-and-permuted-pre-training-for | 86.3 ± .4 | 50 ± 4 | 0 ± 0 | 14.3 ± .5 | 11.7 ± .4 | 29.4 ± .3 |