Question Answering On Squad20
Metrics
EM
F1
Results
Performance results of various models on this benchmark
Comparison Table
| Model Name | EM | F1 |
|---|---|---|
| Model 1 | 80.208 | 83.149 |
| Model 2 | 79.971 | 83.184 |
| Model 3 | 80.038 | 82.796 |
| Model 4 | 80.117 | 83.189 |
| Model 5 | 82.803 | 85.863 |
| Model 6 | 74.769 | 77.706 |
| Model 7 | 89.021 | 91.765 |
| albert-a-lite-bert-for-self-supervised | 88.107 | 90.902 |
| Model 9 | 80.377 | 83.262 |
| Model 10 | 86.436 | 89.086 |
| Model 11 | 75.344 | 78.381 |
| Model 12 | 84.800 | 87.864 |
| Model 13 | 73.742 | 76.858 |
| Model 14 | 80.388 | 82.908 |
| Model 15 | 78.933 | 81.863 |
| Model 16 | 78.052 | 81.174 |
| Model 17 | 90.002 | 92.497 |
| Model 18 | 84.924 | 88.204 |
| ensemble-albert-on-squad-2-0 | - | 90.123 |
| Model 20 | 79.181 | 82.259 |
| Model 21 | 90.487 | 92.894 |
| Model 22 | 85.838 | 88.921 |
| Model 23 | 82.995 | 86.035 |
| Model 24 | 84.642 | 88.000 |
| Model 25 | 89.551 | 92.366 |
| Model 26 | 77.003 | 80.209 |
| Model 27 | 89.562 | 92.226 |
| Model 28 | 87.847 | 90.532 |
| Model 29 | 86.448 | 89.586 |
| Model 30 | 87.802 | 90.872 |
| deep-contextualized-word-representations | 63.372 | 66.251 |
| Model 32 | 84.642 | 88.000 |
| Model 33 | 90.600 | 92.899 |
| Model 34 | 79.948 | 83.023 |
| Model 35 | 78.357 | 81.500 |
| Model 36 | 90.679 | 92.948 |
| Model 37 | 90.194 | 92.594 |
| Model 38 | 80.411 | 83.457 |
| Model 39 | 88.107 | 90.902 |
| Model 40 | 82.126 | 84.820 |
| Model 41 | 88.614 | 91.303 |
| Model 42 | 87.429 | 90.163 |
| Model 43 | 90.284 | 92.691 |
| Model 44 | 88.050 | 90.645 |
| Model 45 | 82.024 | 84.854 |
| read-verify-machine-reading-comprehension | 71.767 | 74.295 |
| Model 47 | 89.461 | 92.134 |
| Model 48 | 87.949 | 90.818 |
| sg-net-syntax-guided-machine-reading | 88.174 | 90.702 |
| deberta-decoding-enhanced-bert-with | 88.0 | 90.7 |
| Model 51 | 58.508 | 62.045 |
| Model 52 | 59.332 | 62.305 |
| Model 53 | 82.713 | 85.584 |
| Model 54 | 88.637 | 91.230 |
| Model 55 | 79.779 | 82.912 |
| Model 56 | 76.055 | 79.329 |
| Model 57 | 90.420 | 92.799 |
| Model 58 | 83.142 | 85.873 |
| Model 59 | 81.979 | 84.846 |
| Model 60 | 72.884 | 76.217 |
| Model 61 | 85.003 | 87.833 |
| Model 62 | 88.998 | 91.635 |
| Model 63 | 81.178 | 84.251 |
| Model 64 | 78.594 | 81.445 |
| Model 65 | 63.372 | 66.251 |
| xlnet-generalized-autoregressive-pretraining | 87.926 | 90.689 |
| Model 67 | 69.262 | 72.642 |
| Model 68 | 65.651 | 68.866 |
| Model 69 | 71.666 | 75.457 |
| Model 70 | 88.197 | 90.830 |
| Model 71 | 84.721 | 87.117 |
| Model 72 | 90.115 | 92.580 |
| Model 73 | 72.884 | 76.217 |
| stochastic-answer-networks-for-machine | 68.653 | 71.439 |
| Model 75 | 76.055 | 79.329 |
| semantics-aware-bert-for-language | 84.800 | 87.864 |
| Model 77 | 87.847 | 91.265 |
| Model 78 | 85.229 | 87.926 |
| Model 79 | 88.186 | 90.939 |
| Model 80 | 88.050 | 91.036 |
| Model 81 | 86.346 | 89.133 |
| Model 82 | 85.240 | 87.901 |
| Model 83 | 79.632 | 82.852 |
| Model 84 | 80.715 | 83.827 |
| Model 85 | 88.592 | 90.859 |
| Model 86 | 80.456 | 83.509 |
| semantics-aware-bert-for-language | 86.166 | 88.886 |
| Model 88 | 84.123 | 87.013 |
| Model 89 | 79.971 | 83.266 |
| Model 90 | 69.476 | 72.857 |
| Model 91 | 77.262 | 80.258 |
| Model 92 | 84.620 | 87.625 |
| Model 93 | 84.721 | 87.117 |
| Model 94 | 86.820 | 89.795 |
| Model 95 | 89.325 | 91.994 |
| Model 96 | 80.343 | 83.243 |
| Model 97 | 84.202 | 86.767 |
| Model 98 | 0.068 | 3.971 |
| Model 99 | 84.292 | 86.967 |
| Model 100 | 78.481 | 81.531 |
| Model 101 | 86.651 | 89.595 |
| Model 102 | 85.173 | 88.425 |
| Model 103 | 80.354 | 83.329 |
| Model 104 | 86.166 | 88.886 |
| Model 105 | 89.348 | 91.985 |
| Model 106 | 89.224 | 91.853 |
| Model 107 | 68.213 | 70.878 |
| semantics-aware-bert-for-language | 86.166 | 88.886 |
| Model 109 | 83.819 | 86.669 |
| Model 110 | 84.123 | 87.013 |
| Model 111 | 40.397 | 43.213 |
| Model 112 | 84.834 | 87.644 |
| Model 113 | 80.140 | 82.962 |
| Model 114 | 56.545 | 59.546 |
| Model 115 | 88.716 | 91.365 |
| Model 116 | 90.442 | 92.877 |
| Model 117 | 83.457 | 86.122 |
| Model 118 | 85.884 | 88.621 |
| Model 119 | 89.449 | 92.118 |
| Model 120 | 48.883 | 48.883 |
| Model 121 | 75.073 | 77.805 |
| Model 122 | 90.724 | 93.011 |
| Model 123 | 85.872 | 88.793 |
| Model 124 | 88.524 | 91.256 |
| Model 125 | 87.147 | 89.474 |
| Model 126 | 78.357 | 81.500 |
| Model 127 | 67.897 | 70.884 |
| Model 128 | 63.327 | 66.633 |
| Model 129 | 88.107 | 90.902 |
| Model 130 | 83.751 | 86.594 |
| Model 131 | 90.386 | 92.777 |
| sg-net-syntax-guided-machine-reading | 87.238 | 90.071 |
| Model 133 | 83.051 | 85.737 |
| Model 134 | 74.272 | 77.052 |
| Model 135 | 90.871 | 93.183 |
| Model 136 | 76.563 | 79.776 |
| Model 137 | 88.231 | 90.713 |
| Model 138 | 78.650 | 81.497 |
| Model 139 | 68.213 | 70.878 |
| Model 140 | 86.730 | 89.286 |
| Model 141 | 44.945 | 47.994 |
| Model 142 | 75.457 | 78.232 |
| Model 143 | 83.536 | 86.096 |
| Model 144 | 89.449 | 92.118 |
| Model 145 | 90.454 | 92.748 |
| Model 146 | 84.642 | 88.000 |
| Model 147 | 85.827 | 89.778 |
| Model 148 | 80.241 | 83.175 |
| Model 149 | 74.791 | 77.988 |
| Model 150 | 4.830 | 5.920 |
| Model 151 | 82.374 | 85.310 |
| u-net-machine-reading-comprehension-with | 71.417 | 74.869 |
| Model 153 | 80.354 | 83.329 |
| Model 154 | 78.876 | 82.524 |
| Model 155 | 74.329 | 77.396 |
| Model 156 | 85.748 | 88.709 |
| pay-attention-to-mlps | - | 78.3 |
| Model 158 | 81.731 | 84.862 |
| spanbert-improving-pre-training-by | 85.7 | 88.7 |
| Model 160 | 88.298 | 91.078 |
| Model 161 | 88.761 | 91.745 |
| Model 162 | 79.745 | 83.020 |
| Model 163 | 48.804 | 48.815 |
| Model 164 | 76.710 | 79.659 |
| Model 165 | 74.656 | 77.404 |
| luke-deep-contextualized-entity | - | 90.2 |
| Model 167 | 85.150 | 87.715 |
| stochastic-answer-networks-for-machine | 71.316 | 73.704 |
| Model 169 | 79.779 | 83.099 |
| Model 170 | 85.827 | 88.699 |
| Model 171 | 90.939 | 93.214 |
| Model 172 | 78.052 | 81.174 |
| Model 173 | 88.569 | 91.287 |
| Model 174 | 74.577 | 77.464 |
| Model 175 | 59.174 | 62.093 |
| Model 176 | 88.434 | 90.918 |
| Model 177 | 63.338 | 67.422 |
| Model 178 | 84.123 | 87.013 |
| Model 179 | 87.700 | 90.588 |
| Model 180 | 73.099 | 76.236 |
| Model 181 | 87.994 | 90.944 |
| Model 182 | 88.107 | 90.902 |
| Model 183 | 86.933 | 90.037 |
| fusionnet-fusing-via-fully-aware-attention | 70.300 | 72.484 |
| Model 185 | 86.211 | 88.848 |
| Model 186 | 87.046 | 89.899 |
| Model 187 | 78.650 | 81.474 |
| Model 188 | 89.743 | 92.180 |
| Model 189 | 68.766 | 71.662 |
| Model 190 | 90.081 | 92.457 |
| Model 191 | 82.431 | 85.178 |
| Model 192 | 78.933 | 81.863 |
| Model 193 | 89.923 | 92.425 |
| sg-net-syntax-guided-machine-reading | 86.211 | 88.848 |
| Model 195 | 87.193 | 89.934 |
| Model 196 | 88.851 | 91.486 |
| Model 197 | 88.603 | 91.299 |
| Model 198 | 89.528 | 92.059 |
| Model 199 | 84.202 | 86.767 |
| Model 200 | 86.673 | 89.147 |
| Model 201 | 83.040 | 85.892 |
| Model 202 | 80.896 | 83.604 |
| Model 203 | 88.107 | 91.419 |
| Model 204 | 72.670 | 75.507 |
| Model 205 | 86.594 | 89.082 |
| Model 206 | 90.521 | 92.824 |
| Model 207 | 78.933 | 81.863 |
| Model 208 | 83.142 | 85.873 |
| Model 209 | 79.993 | 83.039 |
| Model 210 | 82.882 | 86.002 |
| Model 211 | 88.073 | 91.179 |
| Model 212 | 77.262 | 80.258 |
| Model 213 | 82.577 | 85.603 |
| retrospective-reader-for-machine-reading | 90.578 | 92.978 |
| roberta-a-robustly-optimized-bert-pretraining | 86.820 | 89.795 |
| Model 216 | 88.874 | 91.546 |
| Model 217 | 49.695 | 49.701 |
| Model 218 | 87.994 | 90.944 |
| Model 219 | 90.059 | 92.517 |
| Model 220 | 84.823 | 87.489 |
| Model 221 | 89.133 | 91.666 |
| Model 222 | 90.420 | 92.807 |
| Model 223 | 77.262 | 80.258 |
| Model 224 | 82.126 | 84.624 |
| Model 225 | 88.592 | 91.286 |
| Model 226 | 90.126 | 92.535 |
| Model 227 | 85.703 | 88.400 |
| Model 228 | 57.707 | 62.341 |
| Model 229 | 83.469 | 86.043 |
| albert-a-lite-bert-for-self-supervised | 89.731 | 92.215 |
| Model 231 | 83.525 | 86.222 |
| Model 232 | 83.119 | 85.510 |
| Model 233 | 86.572 | 89.063 |
| Model 234 | 86.572 | 89.063 |
| Model 235 | 74.385 | 77.308 |
| Model 236 | 90.002 | 92.425 |
| Model 237 | 69.476 | 72.857 |
| Model 238 | 27.217 | 29.597 |
| Model 239 | 72.072 | 75.513 |
| Model 240 | 74.791 | 77.988 |
| Model 241 | 90.860 | 93.100 |
| Model 242 | 88.355 | 91.019 |
| Model 243 | 85.850 | 88.449 |
| Model 244 | 86.098 | 89.634 |
| luke-deep-contextualized-entity | 87.429 | 90.163 |
| Model 246 | 82.724 | 85.491 |
| Model 247 | 70.763 | 74.449 |
| Model 248 | 86.166 | 88.886 |
| Model 249 | 80.749 | 83.851 |
| Model 250 | 65.256 | 69.206 |
| Model 251 | 81.110 | 84.386 |
| Model 252 | 71.462 | 74.434 |
| Model 253 | 82.972 | 85.810 |
| Model 254 | 66.610 | 70.303 |
| sg-net-syntax-guided-machine-reading | 85.229 | 87.926 |
| Model 256 | 68.021 | 71.583 |
| Model 257 | 89.235 | 91.900 |
| Model 258 | 87.870 | 90.823 |
| Model 259 | 73.505 | 76.424 |
| Model 260 | 85.838 | 88.921 |
| retrospective-reader-for-machine-reading | 89.562 | 92.052 |
| Model 262 | 89.404 | 91.964 |
| Model 263 | 82.306 | 85.670 |
| Model 264 | 74.746 | 78.227 |
| retrospective-reader-for-machine-reading | 90.115 | 92.580 |
| Model 266 | 89.235 | 91.739 |
| Model 267 | 85.838 | 88.921 |
| Model 268 | 56.545 | 59.546 |
| Model 269 | 71.699 | 74.430 |
| Model 270 | 72.072 | 75.513 |
| Model 271 | 85.082 | 87.615 |
| Model 272 | 81.573 | 84.535 |
| Model 273 | 77.319 | 80.310 |
| Model 274 | 89.325 | 91.939 |
| Model 275 | 71.293 | 74.578 |
| Model 276 | 80.591 | 83.391 |
| Model 277 | 82.961 | 86.075 |
| Model 278 | 80.422 | 83.118 |
| Model 279 | 88.998 | 91.635 |
| Model 280 | 86.403 | 89.148 |
| Model 281 | 85.872 | 88.989 |
| Model 282 | 65.719 | 69.381 |
| Model 283 | 89.777 | 92.312 |
| Model 284 | 80.005 | 83.208 |
| Model 285 | 73.302 | 76.284 |
| retrospective-reader-for-machine-reading | 88.107 | 91.419 |