HyperAI超神经

首页资讯最新论文教程数据集百科 SOTA LLM 模型天梯 GPU 天梯顶会

中文

HyperAI超神经

Code Generation On Conala

评估指标

BLEU

Exact Match Accuracy

评测结果

各个模型在此基准测试上的表现结果

模型名称	BLEU	Exact Match Accuracy	Paper Title	Repository
Reranker	30.11	2.8	Reranking for Neural Semantic Parsing	-
ELECTRAMarian	30.18	10.0	Leveraging pre-trained language models for code generation
External Knowledge With API + Reranking	32.26	-	Incorporating External Knowledge through Pre-training for Natural Language to Code Generation	-
TranX	24.30	-	TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation	-
PanGu-Coder-FT-I	44.32	-	Fine-Tuning Large Language Models for Answering Programming Questions with Code Snippets	-
RoBERTaMarian	35.74	13.8	Leveraging pre-trained language models for code generation
TranX + BERT w/mined	34.2	5.8	The impact of lexical and grammatical processing on generating code from natural language	-
BART W/ Mined	30.55	-	Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation	-
BERTMarian	32.46	12.40	Leveraging pre-trained language models for code generation
BART Base	26.24	-	Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation	-
BERT + TAE	33.41	-	Code Generation from Natural Language with Less Prior and More Monolingual Data	-
LUKEMarian	29.83	7.6	Leveraging pre-trained language models for code generation
MarianCG	34.43	10.2	MarianCG: a code generation transformer model inspired by machine translation
External Knowledge With API	30.69	-	Incorporating External Knowledge through Pre-training for Natural Language to Code Generation	-

0 of 14 row(s) selected.

Code Generation On Conala | SOTA | HyperAI超神经