HyperAI超神経

Natural Language Inference On Terra

評価指標

Accuracy

評価結果

このベンチマークにおける各モデルのパフォーマンス結果

モデル名
Accuracy
Paper TitleRepository
YaLM 1.0B few-shot0.605--
RuGPT3Small0.488--
Multilingual Bert0.617--
ruBert-base finetune0.703--
SBERT_Large_mt_ru_finetuning0.637--
RuBERT plain0.642--
MT5 Large0.561mT5: A massively multilingual pre-trained text-to-text transformer
RuGPT3XL few-shot0.573--
Random weighted0.483Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks-
ruBert-large finetune0.704--
SBERT_Large0.637--
Golden Transformer0.871--
RuBERT conversational0.64--
ruRoberta-large finetune0.801--
ruT5-large-finetune0.747--
heuristic majority0.549Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks-
ruT5-base-finetune0.692--
Human Benchmark0.92RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark
majority_class0.513Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks-
RuGPT3Large0.654--
0 of 22 row(s) selected.