Question Answering On Squad11
Metrics
EM
F1
Results
Performance results of various models on this benchmark
Comparison Table
| Model Name | EM | F1 |
|---|---|---|
| stochastic-answer-networks-for-machine | 79.608 | 86.496 |
| Model 2 | 71.908 | 81.023 |
| harvesting-and-refining-question-answer-pairs | 55.827 | 65.467 |
| Model 4 | 68.331 | 77.783 |
| information-theoretic-representation | 77.7 | 85.8 |
| adaptation-of-deep-bidirectional-multilingual | - | 84.6 |
| Model 7 | 84.926 | 91.932 |
| Model 8 | 90.622 | 95.719 |
| Model 9 | 85.314 | 91.756 |
| phase-conductor-on-multi-layered-attentions | 74.405 | 82.742 |
| Model 11 | 78.664 | 85.780 |
| a-large-batch-optimizer-reality-check | - | 91.58 |
| fusionnet-fusing-via-fully-aware-attention | 75.968 | 83.900 |
| Model 14 | 84.402 | 90.561 |
| dyrex-dynamic-query-representation-for | - | 91.01 |
| Model 16 | 0.000 | 0.000 |
| Model 17 | 74.121 | 82.342 |
| contextualized-word-representations-for | 75.789 | 83.261 |
| harvesting-and-refining-question-answer-pairs | 61.145 | 71.389 |
| memen-multi-layer-embedding-with-memory | 78.234 | 85.344 |
| Model 21 | 84.328 | 91.281 |
| end-to-end-answer-chunk-extraction-and | 62.499 | 70.956 |
| Model 23 | 81.003 | 87.432 |
| multi-perspective-context-matching-for | 73.765 | 81.257 |
| Model 25 | 55.827 | 65.467 |
| Model 26 | 86.458 | 92.645 |
| exploring-machine-reading-comprehension-with | 76.125 | 83.538 |
| Model 28 | 79.996 | 86.711 |
| stochastic-answer-networks-for-machine | 76.828 | 84.396 |
| Model 30 | 70.985 | 79.939 |
| Model 31 | 75.034 | 83.405 |
| Model 32 | 66.527 | 75.787 |
| bert-pre-training-of-deep-bidirectional | 87.4 | 93.2 |
| Model 34 | 75.265 | 82.769 |
| Model 35 | 71.016 | 79.835 |
| information-theoretic-representation | 81.5 | 88.5 |
| making-neural-qa-as-simple-as-possible-but | 70.849 | 78.857 |
| Model 38 | 69.600 | 78.236 |
| Model 39 | 59.058 | 69.436 |
| Model 40 | 76.859 | 84.739 |
| Model 41 | 72.485 | 80.550 |
| multi-perspective-context-matching-for | 70.387 | 78.784 |
| Model 43 | 85.356 | 91.202 |
| bert-pre-training-of-deep-bidirectional | 87.433 | 93.160 |
| textbox-2-0-a-text-generation-library-with | - | 93.04 |
| learning-to-compute-word-embeddings-on-the | 62.897 | 72.016 |
| Model 47 | 85.125 | 91.623 |
| Model 48 | 73.639 | 81.931 |
| simple-and-effective-multi-paragraph-reading | 72.139 | 81.048 |
| Model 50 | 61.145 | 71.389 |
| dcn-mixed-objective-and-deep-residual | 74.866 | 82.806 |
| machine-comprehension-using-match-lstm-and | 67.901 | 77.022 |
| machine-comprehension-using-match-lstm-and | 64.744 | 73.743 |
| Model 54 | 80.615 | 87.311 |
| fusionnet-fusing-via-fully-aware-attention | 78.978 | 86.016 |
| Model 56 | 79.199 | 86.590 |
| Model 57 | 81.790 | 88.163 |
| words-or-characters-fine-grained-gating-for | 62.446 | 73.327 |
| Model 59 | 83.426 | 89.218 |
| efficientqa-a-roberta-based-phrase-indexed | 74.9 | 83.1 |
| Model 61 | 47.341 | 56.436 |
| Model 62 | 0.000 | 0.000 |
| Model 63 | 80.027 | 87.288 |
| Model 64 | 52.544 | 62.780 |
| learning-to-compute-word-embeddings-on-the | 62.604 | 71.968 |
| Model 66 | 79.692 | 86.727 |
| deep-contextualized-word-representations | 81.003 | 87.432 |
| spanbert-improving-pre-training-by | 88.8 | 94.6 |
| smarnet-teaching-machines-to-read-and | 71.415 | 80.160 |
| Model 70 | 75.821 | 83.843 |
| Model 71 | 89.709 | 94.859 |
| Model 72 | 82.681 | 89.379 |
| Model 73 | 79.083 | 86.288 |
| Model 74 | 71.698 | 80.462 |
| qanet-combining-local-convolution-with-global | 76.2 | 84.6 |
| making-neural-qa-as-simple-as-possible-but | 68.436 | 77.070 |
| reasonet-learning-to-stop-reading-in-machine | 70.555 | 79.364 |
| Model 78 | 69.443 | 78.358 |
| Model 79 | 0.000 | 0.000 |
| Model 80 | 71.898 | 79.989 |
| structural-embedding-of-syntactic-trees-for | 74.090 | 81.761 |
| Model 82 | 67.618 | 77.151 |
| Model 83 | 83.982 | 89.796 |
| Model 84 | 80.436 | 87.021 |
| Model 85 | 72.758 | 81.001 |
| Model 86 | 0.000 | 0.000 |
| Model 87 | 82.482 | 89.281 |
| Model 88 | 89.646 | 94.930 |
| memen-multi-layer-embedding-with-memory | 75.370 | 82.658 |
| Model 90 | 77.237 | 84.466 |
| Model 91 | 85.944 | 92.425 |
| Model 92 | 78.087 | 85.348 |
| Model 93 | 88.912 | 94.584 |
| bidirectional-attention-flow-for-machine | 67.974 | 77.323 |
| Model 95 | 83.804 | 90.429 |
| Model 96 | 81.580 | 88.948 |
| Model 97 | 68.132 | 77.569 |
| Model 98 | 74.489 | 82.815 |
| Model 99 | 89.856 | 94.903 |
| Model 100 | 79.901 | 86.536 |
| Model 101 | 64.439 | 73.921 |
| Model 102 | 86.521 | 92.617 |
| Model 103 | 77.646 | 84.905 |
| Model 104 | 78.664 | 85.780 |
| Model 105 | 75.989 | 83.475 |
| reading-wikipedia-to-answer-open-domain | 70.733 | 79.353 |
| structural-embedding-of-syntactic-trees-for | 68.478 | 77.971 |
| Model 108 | 78.328 | 85.682 |
| Model 109 | 64.932 | 74.594 |
| exploring-question-understanding-and | 73.010 | 81.517 |
| Model 111 | 88.912 | 94.584 |
| Model 112 | 78.171 | 85.543 |
| Model 113 | 79.083 | 86.288 |
| Model 114 | 63.306 | 73.463 |
| Model 115 | 0.000 | 6.907 |
| bidirectional-attention-flow-for-machine | 73.744 | 81.525 |
| Model 117 | 78.496 | 85.469 |
| exploring-question-understanding-and | 70.607 | 79.821 |
| Model 119 | 79.031 | 86.006 |
| a-multi-stage-memory-augmented-neural-network | 79.692 | 86.727 |
| deep-contextualized-word-representations | 78.58 | 85.833 |
| Model 122 | 85.944 | 92.425 |
| Model 123 | 73.303 | 81.754 |
| Model 124 | 82.062 | 88.947 |
| Model 125 | 65.163 | 74.555 |
| Model 126 | 80.426 | 86.912 |
| Model 127 | 53.698 | 64.036 |
| Model 128 | 76.240 | 84.599 |
| phase-conductor-on-multi-layered-attentions | 73.240 | 81.933 |
| xlnet-generalized-autoregressive-pretraining | 89.898 | 95.080 |
| bert-pre-training-of-deep-bidirectional | 85.083 | 91.835 |
| Model 132 | 81.401 | 88.122 |
| Model 133 | 72.600 | 81.011 |
| Model 134 | 78.328 | 85.682 |
| Model 135 | 81.307 | 88.909 |
| gated-self-matching-networks-for-reading | 76.461 | 84.265 |
| Model 137 | 67.544 | 76.429 |
| Model 138 | 77.342 | 84.925 |
| dynamic-coattention-networks-for-question | 71.625 | 80.383 |
| Model 140 | 80.164 | 86.721 |
| luke-deep-contextualized-entity | 90.202 | 95.379 |
| Model 142 | 66.516 | 76.349 |
| luke-deep-contextualized-entity | - | 95.4 |
| reinforced-mnemonic-reader-for-machine | 82.283 | 88.533 |
| Model 145 | 88.650 | 94.393 |
| Model 146 | 76.461 | 84.265 |
| Model 147 | 0.000 | 0.000 |
| Model 148 | 77.845 | 85.297 |
| Model 149 | 80.720 | 87.758 |
| Model 150 | 75.223 | 82.716 |
| Model 151 | 90.202 | 95.379 |
| Model 152 | 76.775 | 84.491 |
| Model 153 | 83.468 | 90.133 |
| Model 154 | 44.215 | 54.723 |
| machine-comprehension-using-match-lstm-and | 60.474 | 70.695 |
| Model 156 | 79.083 | 86.450 |
| a-fully-attention-based-information-retriever | 67.744 | 77.605 |
| dynamic-coattention-networks-for-question | 66.233 | 75.896 |
| Model 159 | 75.926 | 83.305 |
| Model 160 | 85.430 | 91.976 |
| Model 161 | 76.146 | 83.991 |
| reinforced-mnemonic-reader-for-machine | 79.545 | 86.654 |
| Model 163 | 67.502 | 76.786 |
| reinforced-mnemonic-reader-for-machine | 70.995 | 80.146 |
| reasonet-learning-to-stop-reading-in-machine | 75.034 | 82.552 |
| Model 166 | 81.496 | 87.557 |
| Model 167 | 81.045 | 87.999 |
| contextualized-word-representations-for | 77.583 | 84.163 |
| reinforced-mnemonic-reader-for-machine | 74.268 | 82.371 |
| Model 170 | 80.667 | 88.169 |
| Model 171 | 74.080 | 81.665 |
| Model 172 | 71.373 | 79.725 |
| Model 173 | 78.401 | 85.724 |
| Model 174 | 87.465 | 93.294 |
| Model 175 | 78.653 | 86.663 |
| Model 176 | 72.590 | 81.415 |
| luke-deep-contextualized-entity | 90.2 | - |
| Model 178 | 12.273 | 13.211 |
| Model 179 | 74.604 | 82.501 |
| phase-conductor-on-multi-layered-attentions | 76.996 | 84.630 |
| Model 181 | 78.580 | 85.833 |
| structural-embedding-of-syntactic-trees-for | 73.723 | 81.530 |
| Model 183 | 82.849 | 88.764 |
| Model 184 | 89.898 | 95.080 |
| bert-pre-training-of-deep-bidirectional | - | 91.8 |
| ruminating-reader-reasoning-with-gated-multi | 70.639 | 79.456 |
| structural-embedding-of-syntactic-trees-for | 68.163 | 77.527 |
| Model 188 | 82.471 | 89.306 |
| memen-multi-layer-embedding-with-memory | 78.234 | 85.344 |
| Model 190 | 77.573 | 84.858 |
| learning-to-compute-word-embeddings-on-the | 64.083 | 73.056 |
| linkbert-pretraining-language-models-with | 87.45 | 92.7 |
| Model 193 | 78.223 | 85.535 |
| Model 194 | 88.839 | 94.635 |
| Model 195 | 77.090 | 83.931 |
| Model 196 | 84.978 | 92.019 |
| learned-in-translation-contextualized-word | 71.3 | 79.9 |
| Model 198 | 79.859 | 88.263 |
| Model 199 | 82.136 | 88.126 |
| Model 200 | 86.940 | 92.641 |
| Model 201 | 79.597 | 87.374 |
| learning-recurrent-span-representations-for | 70.849 | 78.741 |
| simple-recurrent-units-for-highly | 71.4 | 80.2 |
| Model 204 | 83.930 | 90.613 |
| Model 205 | 80.436 | 86.912 |
| Model 206 | 82.650 | 88.493 |
| Model 207 | 79.083 | 86.288 |
| dcn-mixed-objective-and-deep-residual | 78.852 | 85.996 |
| Model 209 | 80.489 | 87.454 |
| Model 210 | 82.440 | 88.607 |
| Model 211 | 85.335 | 91.807 |
| Model 212 | 52.533 | 62.757 |
| machine-comprehension-using-match-lstm-and | 54.505 | 67.748 |