Text To Sql On Spider
Metrics
Exact Match Accuracy (Dev)
Exact Match Accuracy (Test)
Execution Accuracy (Test)
Results
Performance results of various models on this benchmark
Comparison Table
| Model Name | Exact Match Accuracy (Dev) | Exact Match Accuracy (Test) | Execution Accuracy (Test) |
|---|---|---|---|
| picard-parsing-incrementally-for-constrained | 75.5 | 71.9 | 75.1 |
| msc-sql-multi-sample-critiquing-small | - | - | 84.7 |
| improving-generalization-in-language-model | 69.4 | - | 78 |
| datagpt-sql-7b-an-open-source-language-model-1 | 81.6 | - | - |
| sadga-structure-aware-dual-graph-aggregation | 73.1 | 70.1 | - |
| tabert-pretraining-for-joint-understanding-of | 64.5 | - | - |
| t5-sr-a-unified-seq-to-seq-decoding-strategy | 77.2 | 72.4 | 75.2 |
| text-to-sql-empowered-by-large-language | 74.4 | - | 86.6 |
| lever-learning-to-verify-language-to-code | - | - | - |
| pet-sql-a-prompt-enhanced-two-stage-text-to | - | 66.6 | 87.6 |
| decoupling-the-skeleton-parsing-and-schema | 80.5 | 72.0 | 79.9 |
| learning-metadata-agnostic-representations | - | - | 84.0 |
| din-sql-decomposed-in-context-learning-of-1 | - | 60 | 85.3 |
| ryansql-recursively-applying-sketch-based | 66.6 | 58.2 | - |
| graphix-t5-mixing-pre-trained-transformers | 77.1 | - | 77.6 |
| datagpt-sql-7b-an-open-source-language-model-1 | 80.3 | - | - |
| c3-zero-shot-text-to-sql-with-chatgpt | - | - | 82.3 |
| learning-contextual-representations-for | 71.8 | - | - |
| xiyan-sql-a-multi-generator-ensemble | - | - | 89.65 |
| knowledge-to-sql-enhancing-sql-generation | - | - | - |