Word Sense Disambiguation On Wic Tsv
評価指標
Task 1 Accuracy: all
Task 1 Accuracy: domain specific
Task 1 Accuracy: general purpose
Task 2 Accuracy: all
Task 2 Accuracy: domain specific
Task 2 Accuracy: general purpose
Task 3 Accuracy: all
Task 3 Accuracy: domain specific
Task 3 Accuracy: general purpose
評価結果
このベンチマークにおける各モデルのパフォーマンス結果
モデル名 | Task 1 Accuracy: all | Task 1 Accuracy: domain specific | Task 1 Accuracy: general purpose | Task 2 Accuracy: all | Task 2 Accuracy: domain specific | Task 2 Accuracy: general purpose | Task 3 Accuracy: all | Task 3 Accuracy: domain specific | Task 3 Accuracy: general purpose | Paper Title | Repository |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bert-base | 75.3 | 77.9 | 73.3 | 71.7 | 74.7 | 68.6 | 76.6 | 80.4 | 73.5 | WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context | |
| Human | - | - | - | - | - | - | 85.3 | 89.2 | 82.1 | WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context | |
| transformers | 77.8 | 81.0 | 75.2 | - | - | - | 71.9 | 65.7 | 77.0 | - | - |
| FastText | 53.7 | 50.6 | 56.2 | 52.7 | 47.7 | 56.8 | 53.4 | 49.0 | 57.1 | WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context | |
| GlossBert-ws | 75.9 | 76.7 | 75.2 | - | - | - | - | - | - | GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge | |
| CTLR | 76.8 | 79.6 | 74.5 | 72.7 | 81.5 | 65.6 | 78.3 | 85.7 | 72.1 | - | - |
| All true | 50.8 | 47.0 | 53.8 | 50.8 | 47.0 | 53.8 | 50.8 | 47.0 | 53.8 | WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context | |
| Unsupervised Bert | 54.4 | 60.6 | 49.2 | 62.8 | 69.1 | 57.6 | 60.5 | 67.9 | 54.4 | WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context |
0 of 8 row(s) selected.