Question Answering On Squad11

评估指标

评测结果

各个模型在此基准测试上的表现结果

比较表格

模型名称	EM	F1
stochastic-answer-networks-for-machine	79.608	86.496
模型 2	71.908	81.023
harvesting-and-refining-question-answer-pairs	55.827	65.467
模型 4	68.331	77.783
information-theoretic-representation	77.7	85.8
adaptation-of-deep-bidirectional-multilingual	-	84.6
模型 7	84.926	91.932
模型 8	90.622	95.719
模型 9	85.314	91.756
phase-conductor-on-multi-layered-attentions	74.405	82.742
模型 11	78.664	85.780
a-large-batch-optimizer-reality-check	-	91.58
fusionnet-fusing-via-fully-aware-attention	75.968	83.900
模型 14	84.402	90.561
dyrex-dynamic-query-representation-for	-	91.01
模型 16	0.000	0.000
模型 17	74.121	82.342
contextualized-word-representations-for	75.789	83.261
harvesting-and-refining-question-answer-pairs	61.145	71.389
memen-multi-layer-embedding-with-memory	78.234	85.344
模型 21	84.328	91.281
end-to-end-answer-chunk-extraction-and	62.499	70.956
模型 23	81.003	87.432
multi-perspective-context-matching-for	73.765	81.257
模型 25	55.827	65.467
模型 26	86.458	92.645
exploring-machine-reading-comprehension-with	76.125	83.538
模型 28	79.996	86.711
stochastic-answer-networks-for-machine	76.828	84.396
模型 30	70.985	79.939
模型 31	75.034	83.405
模型 32	66.527	75.787
bert-pre-training-of-deep-bidirectional	87.4	93.2
模型 34	75.265	82.769
模型 35	71.016	79.835
information-theoretic-representation	81.5	88.5
making-neural-qa-as-simple-as-possible-but	70.849	78.857
模型 38	69.600	78.236
模型 39	59.058	69.436
模型 40	76.859	84.739
模型 41	72.485	80.550
multi-perspective-context-matching-for	70.387	78.784
模型 43	85.356	91.202
bert-pre-training-of-deep-bidirectional	87.433	93.160
textbox-2-0-a-text-generation-library-with	-	93.04
learning-to-compute-word-embeddings-on-the	62.897	72.016
模型 47	85.125	91.623
模型 48	73.639	81.931
simple-and-effective-multi-paragraph-reading	72.139	81.048
模型 50	61.145	71.389
dcn-mixed-objective-and-deep-residual	74.866	82.806
machine-comprehension-using-match-lstm-and	67.901	77.022
machine-comprehension-using-match-lstm-and	64.744	73.743
模型 54	80.615	87.311
fusionnet-fusing-via-fully-aware-attention	78.978	86.016
模型 56	79.199	86.590
模型 57	81.790	88.163
words-or-characters-fine-grained-gating-for	62.446	73.327
模型 59	83.426	89.218
efficientqa-a-roberta-based-phrase-indexed	74.9	83.1
模型 61	47.341	56.436
模型 62	0.000	0.000
模型 63	80.027	87.288
模型 64	52.544	62.780
learning-to-compute-word-embeddings-on-the	62.604	71.968
模型 66	79.692	86.727
deep-contextualized-word-representations	81.003	87.432
spanbert-improving-pre-training-by	88.8	94.6
smarnet-teaching-machines-to-read-and	71.415	80.160
模型 70	75.821	83.843
模型 71	89.709	94.859
模型 72	82.681	89.379
模型 73	79.083	86.288
模型 74	71.698	80.462
qanet-combining-local-convolution-with-global	76.2	84.6
making-neural-qa-as-simple-as-possible-but	68.436	77.070
reasonet-learning-to-stop-reading-in-machine	70.555	79.364
模型 78	69.443	78.358
模型 79	0.000	0.000
模型 80	71.898	79.989
structural-embedding-of-syntactic-trees-for	74.090	81.761
模型 82	67.618	77.151
模型 83	83.982	89.796
模型 84	80.436	87.021
模型 85	72.758	81.001
模型 86	0.000	0.000
模型 87	82.482	89.281
模型 88	89.646	94.930
memen-multi-layer-embedding-with-memory	75.370	82.658
模型 90	77.237	84.466
模型 91	85.944	92.425
模型 92	78.087	85.348
模型 93	88.912	94.584
bidirectional-attention-flow-for-machine	67.974	77.323
模型 95	83.804	90.429
模型 96	81.580	88.948
模型 97	68.132	77.569
模型 98	74.489	82.815
模型 99	89.856	94.903
模型 100	79.901	86.536
模型 101	64.439	73.921
模型 102	86.521	92.617
模型 103	77.646	84.905
模型 104	78.664	85.780
模型 105	75.989	83.475
reading-wikipedia-to-answer-open-domain	70.733	79.353
structural-embedding-of-syntactic-trees-for	68.478	77.971
模型 108	78.328	85.682
模型 109	64.932	74.594
exploring-question-understanding-and	73.010	81.517
模型 111	88.912	94.584
模型 112	78.171	85.543
模型 113	79.083	86.288
模型 114	63.306	73.463
模型 115	0.000	6.907
bidirectional-attention-flow-for-machine	73.744	81.525
模型 117	78.496	85.469
exploring-question-understanding-and	70.607	79.821
模型 119	79.031	86.006
a-multi-stage-memory-augmented-neural-network	79.692	86.727
deep-contextualized-word-representations	78.58	85.833
模型 122	85.944	92.425
模型 123	73.303	81.754
模型 124	82.062	88.947
模型 125	65.163	74.555
模型 126	80.426	86.912
模型 127	53.698	64.036
模型 128	76.240	84.599
phase-conductor-on-multi-layered-attentions	73.240	81.933
xlnet-generalized-autoregressive-pretraining	89.898	95.080
bert-pre-training-of-deep-bidirectional	85.083	91.835
模型 132	81.401	88.122
模型 133	72.600	81.011
模型 134	78.328	85.682
模型 135	81.307	88.909
gated-self-matching-networks-for-reading	76.461	84.265
模型 137	67.544	76.429
模型 138	77.342	84.925
dynamic-coattention-networks-for-question	71.625	80.383
模型 140	80.164	86.721
luke-deep-contextualized-entity	90.202	95.379
模型 142	66.516	76.349
luke-deep-contextualized-entity	-	95.4
reinforced-mnemonic-reader-for-machine	82.283	88.533
模型 145	88.650	94.393
模型 146	76.461	84.265
模型 147	0.000	0.000
模型 148	77.845	85.297
模型 149	80.720	87.758
模型 150	75.223	82.716
模型 151	90.202	95.379
模型 152	76.775	84.491
模型 153	83.468	90.133
模型 154	44.215	54.723
machine-comprehension-using-match-lstm-and	60.474	70.695
模型 156	79.083	86.450
a-fully-attention-based-information-retriever	67.744	77.605
dynamic-coattention-networks-for-question	66.233	75.896
模型 159	75.926	83.305
模型 160	85.430	91.976
模型 161	76.146	83.991
reinforced-mnemonic-reader-for-machine	79.545	86.654
模型 163	67.502	76.786
reinforced-mnemonic-reader-for-machine	70.995	80.146
reasonet-learning-to-stop-reading-in-machine	75.034	82.552
模型 166	81.496	87.557
模型 167	81.045	87.999
contextualized-word-representations-for	77.583	84.163
reinforced-mnemonic-reader-for-machine	74.268	82.371
模型 170	80.667	88.169
模型 171	74.080	81.665
模型 172	71.373	79.725
模型 173	78.401	85.724
模型 174	87.465	93.294
模型 175	78.653	86.663
模型 176	72.590	81.415
luke-deep-contextualized-entity	90.2	-
模型 178	12.273	13.211
模型 179	74.604	82.501
phase-conductor-on-multi-layered-attentions	76.996	84.630
模型 181	78.580	85.833
structural-embedding-of-syntactic-trees-for	73.723	81.530
模型 183	82.849	88.764
模型 184	89.898	95.080
bert-pre-training-of-deep-bidirectional	-	91.8
ruminating-reader-reasoning-with-gated-multi	70.639	79.456
structural-embedding-of-syntactic-trees-for	68.163	77.527
模型 188	82.471	89.306
memen-multi-layer-embedding-with-memory	78.234	85.344
模型 190	77.573	84.858
learning-to-compute-word-embeddings-on-the	64.083	73.056
linkbert-pretraining-language-models-with	87.45	92.7
模型 193	78.223	85.535
模型 194	88.839	94.635
模型 195	77.090	83.931
模型 196	84.978	92.019
learned-in-translation-contextualized-word	71.3	79.9
模型 198	79.859	88.263
模型 199	82.136	88.126
模型 200	86.940	92.641
模型 201	79.597	87.374
learning-recurrent-span-representations-for	70.849	78.741
simple-recurrent-units-for-highly	71.4	80.2
模型 204	83.930	90.613
模型 205	80.436	86.912
模型 206	82.650	88.493
模型 207	79.083	86.288
dcn-mixed-objective-and-deep-residual	78.852	85.996
模型 209	80.489	87.454
模型 210	82.440	88.607
模型 211	85.335	91.807
模型 212	52.533	62.757
machine-comprehension-using-match-lstm-and	54.505	67.748