Text To Sql On Bird Big Bench For Large Scale
평가 지표
Execution Accuracy % (Dev)
Execution Accuracy % (Test)
평가 결과
이 벤치마크에서 각 모델의 성능 결과
비교 표
| 모델 이름 | Execution Accuracy % (Dev) | Execution Accuracy % (Test) |
|---|---|---|
| 모델 1 | 68.12 | 70.21 |
| msc-sql-multi-sample-critiquing-small | 65.6 | - |
| 모델 3 | 59.71 | 60.71 |
| 모델 4 | 58.47 | 60.37 |
| 모델 5 | 62.97 | 64.51 |
| text-to-sql-empowered-by-large-language | 54.76 | 57.41 |
| can-llm-already-serve-as-a-database-interface | 37.22 | 39.30 |
| 모델 8 | 55.48 | 63.39 |
| 모델 9 | 72.43 | 73.17 |
| can-llm-already-serve-as-a-database-interface | - | - |
| 모델 11 | 55.48 | 63.39 |
| can-llms-effectively-leverage-structural | 42.70 | 49.02 |
| 모델 13 | 64.73 | 65.23 |
| 모델 14 | 67.99 | 66.21 |
| 모델 15 | 65.45 | 68.87 |
| 모델 16 | 63.36 | 65.45 |
| can-llm-already-serve-as-a-database-interface | 34.35 | 36.47 |
| chase-sql-multi-path-reasoning-and-preference | 73.14 | 74.06 |
| xiyan-sql-a-multi-generator-ensemble | 73.34 | 75.63 |
| 모델 20 | 69.3 | 72.28 |
| chess-contextual-harnessing-for-efficient-sql | 65 | 66.69 |
| can-llms-effectively-leverage-structural | 46.35 | 54.89 |
| 모델 23 | 60.5 | 64.84 |
| 모델 24 | 62.58 | 63.22 |
| 모델 25 | 57.17 | 59.25 |
| 모델 26 | 58.5 | 62.66 |
| mac-sql-multi-agent-collaboration-for-text-to | 57.56 | 59.59 |
| 모델 28 | 66.82 | 64.00 |
| 모델 29 | 65.38 | 67.86 |
| 모델 30 | 64.62 | - |
| knowledge-to-sql-enhancing-sql-generation | 48.92 | - |
| the-death-of-schema-linking-text-to-sql-in | 67.21 | 71.83 |
| can-llm-already-serve-as-a-database-interface | 36.64 | 40.08 |
| 모델 34 | 37.68 | 47.74 |
| din-sql-decomposed-in-context-learning-of-1 | 50.72 | 55.90 |
| 모델 36 | 66.95 | 69.03 |
| 모델 37 | 72.16 | 70.26 |
| can-llm-already-serve-as-a-database-interface | 27.38 | 33.04 |
| 모델 39 | 74.32 | 74.12 |
| 모델 40 | 61.34 | 64.95 |