Code Generation On Conala Ext
평가 지표
BLEU
평가 결과
이 벤치마크에서 각 모델의 성능 결과
모델 이름 | BLEU | Paper Title | Repository |
|---|---|---|---|
| External Knowledge With API | 20.37 | Incorporating External Knowledge through Pre-training for Natural Language to Code Generation | |
| BART Base | 34.35 | Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation | |
| External Knowledge With API + Reranking | 20.54 | Incorporating External Knowledge through Pre-training for Natural Language to Code Generation | |
| TranX | 18.85 | TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation | |
| BART W/ Mined | 35.32 | Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation | |
| Reranker | 19.85 | Reranking for Neural Semantic Parsing | - |
0 of 6 row(s) selected.