Word Sense Disambiguation On Words In Context

Metrics

Accuracy

Results

Performance results of various models on this benchmark

Model Name	Accuracy	Paper Title	Repository
GPT-2-XL 1.5B	49.8	LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
ST-MoE-L 4.1B (fine-tuned)	74	ST-MoE: Designing Stable and Transferable Sparse Expert Models	-
Hybrid H3 125M (0-shot, rank classification)	51.4	Hungry Hungry Hippos: Towards Language Modeling with State Space Models
ST-MoE-32B 269B (fine-tuned)	77.7	ST-MoE: Designing Stable and Transferable Sparse Expert Models	-
PaLM 540B (finetuned)	78.8	PaLM: Scaling Language Modeling with Pathways
LaMini-GPT 1.5B	52.4	LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
RoE-3B	52.97	Exploring the Benefits of Training Expert Language Models over Instruction Tuning
Sentence LSTM	53.1	WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations	-
PaLM 2-L (one-shot)	66.8	PaLM 2 Technical Report
Hybrid H3 125M (3-shot, logit scoring)	49.1	Hungry Hungry Hippos: Towards Language Modeling with State Space Models
DeConf	58.7	WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations	-
GPT-3 175B (few-shot, k=32)	49.4	Language Models are Few-Shot Learners
Vega v2 6B (fine-tuned)	77.4	Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE	-
FLAN-T5-Large 783M	64.7	LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
KiC-770M	52.40	Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models	-
ElMo	57.7	WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations	-
Hybrid H3 125M (0-shot, logit scoring)	51.4	Hungry Hungry Hippos: Towards Language Modeling with State Space Models
T5-XXL 11B	76.9	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
UL2 20B (fine-tuned)	77.3	UL2: Unifying Language Learning Paradigms
N-Grammer 343M	56.1	N-Grammer: Augmenting Transformers with latent n-grams

0 of 37 row(s) selected.