다중 선택 질문 응답
다중 선택 질문 응답(MCQA)은 자연어 처리의 하위 작업으로, 모델이 제공된 후보 옵션과 지원 문맥을 기반으로 주어진 질문에 대한 최적의 답변을 예측하도록 요구합니다. 이 작업은 모델의 이해력과 추론 능력을 평가하는 것을 목표로 하며, 지능형 교육, 온라인 평가, 지식 검색 등 다양한 응용 가치를 가지고 있습니다.
MedMCQA
Meditron-70B (CoT + SC)
BIG-bench (Hyperbaton)
BIG-bench (Movie Recommendation)
BIG-bench (Navigate)
BIG-bench (Ruin Names)
MMLU (College Biology)
Chinchilla (few-shot, k=5)
MMLU (Medical Genetics)
MMLU (Professional medicine)
MMLU (Elementary Mathematics)
Chinchilla (few-shot, k=5)
MMLU (High School Biology)
Chinchilla (few-shot, k=5)
MMLU (College Chemistry)
Chinchilla (few-shot, k=5)
MMLU (High School Mathematics)
GAL 120B (zero-shot)
MMLU (Electrical Engineer)
GAL 120B (zero-shot)
MMLU (College Physics)
MMLU (Formal Logic)
Gopher (few-shot, k=5)
MMLU (High School Statistics)
MMLU (Abstract Algebra)
GAL 30B (zero-shot)
MMLU (Econometrics)
Gopher (few-shot, k=5)
MMLU (High School Computer Science)
GAL 120B (zero-shot)
MMLU (College Mathematics)
GAL 120B (zero-shot)
MMLU (Astronomy)
Chinchilla (few-shot, k=5)
MMLU (High School Chemistry)
Chinchilla (few-shot, k=5)
MMLU (College Computer Science)
Chinchilla (few-shot, k=5)
MMLU (High School Physics)
BIG-bench (Novel Concepts)
MMLU (Machine Learning)
Chinchilla (few-shot, k=5)
IndicGLUE WSTP Pa
MMLU (Clinical Knowledge)
MMLU (Anatomy)
Med-PaLM 2 (ER)
MMLU (College Medicine)
FrenchMedMCQA
CamemBERT