Text To Sql On Spider

평가 지표

Exact Match Accuracy (Dev)

Exact Match Accuracy (Test)

Execution Accuracy (Test)

평가 결과

이 벤치마크에서 각 모델의 성능 결과

모델 이름	Exact Match Accuracy (Dev)	Exact Match Accuracy (Test)	Execution Accuracy (Test)	Paper Title	Repository
T5-3B + PICARD	75.5	71.9	75.1	PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models
MSc-SQL	-	-	84.7	MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation
T5-3B+NatSQL+Token Preprocessing	69.4	-	78	Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques
datagpt-sql-7B + InvalidSQL-Feedback	81.6	-	-	DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL	-
SADGA + GAP	73.1	70.1	-	SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQL
MAPO + TABERTLarge (K = 3)	64.5	-	-	TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
T5-SR	77.2	72.4	75.2	T5-SR: A Unified Seq-to-Seq Decoding Strategy for Semantic Parsing
DAIL-SQL + GPT-4 + Self-Consistency	74.4	-	86.6	Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation
code-davinci-002 175B (LEVER)	-	-	-	LEVER: Learning to Verify Language-to-Code Generation with Execution
PET-SQL	-	66.6	87.6	PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency
RESDSQL-3B + NatSQL	80.5	72.0	79.9	RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL
MARLO + Claude 2.1	-	-	84.0	Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection	-
DIN-SQL + GPT-4	-	60	85.3	DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction
RYANSQL (BERT)	66.6	58.2	-	RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases
Graphix-3B+PICARD	77.1	-	77.6	Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing
datagpt-sql-7B	80.3	-	-	DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL	-
C3 + ChatGPT + Zero-Shot	-	-	82.3	C3: Zero-shot Text-to-SQL with ChatGPT
RATSQL + GAP	71.8	-	-	Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training
XiYan-SQL	-	-	89.65	A Preview of XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL
DELLM + GPT-4	-	-	-	Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLM	-

0 of 20 row(s) selected.