Question Answering On Social Iqa

Métriques

Accuracy

Résultats

Résultats de performance de divers modèles sur ce benchmark

Nom du modèle	Accuracy	Paper Title
LLaMA 13B (zero-shot)	50.4	LLaMA: Open and Efficient Foundation Language Models
LLaMA 7B (zero-shot)	48.9	LLaMA: Open and Efficient Foundation Language Models
UnifiedQA 3B	79.8	UnifiedQA: Crossing Format Boundaries With a Single QA System
Chinchilla (zero-shot)	51.3	Training Compute-Optimal Large Language Models
LLaMA 65B (zero-shot)	52.3	LLaMA: Open and Efficient Foundation Language Models
DeBERTa-Large 304M	80.2	Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering
RoBERTa-Large 355M (fine-tuned)	76.7	RoBERTa: A Robustly Optimized BERT Pretraining Approach
DeBERTa-Large 304M (classification-based)	79.9	Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering
LLaMA-3 8B + MixLoRA	78.8	MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
CompassMTL 567M with Tailor	82.2	Task Compass: Scaling Multi-task Pre-training with Task Prefix
LLaMA-3 8B+MoSLoRA (fine-tuned)	81.0	Mixture-of-Subspaces in Low-Rank Adaptation
Random chance baseline	33.3	SocialIQA: Commonsense Reasoning about Social Interactions
Gopher (zero-shot)	50.6	Scaling Language Models: Methods, Analysis & Insights from Training Gopher
BERT-base 110M (fine-tuned)	63.1	SocialIQA: Commonsense Reasoning about Social Interactions
ExDeBERTa 567M	79.6	Task Compass: Scaling Multi-task Pre-training with Task Prefix
BERT-large 340M (fine-tuned)	64.5	SocialIQA: Commonsense Reasoning about Social Interactions
GPT-1 117M (fine-tuned)	63	SocialIQA: Commonsense Reasoning about Social Interactions
LLaMA-2 13B + MixLoRA	82.5	MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
LLaMA 33B (zero-shot)	50.4	LLaMA: Open and Efficient Foundation Language Models
CompassMTL 567M	81.7	Task Compass: Scaling Multi-task Pre-training with Task Prefix

0 of 24 row(s) selected.