Question Answering On Squad11

평가 지표

평가 결과

이 벤치마크에서 각 모델의 성능 결과

비교 표

모델 이름	EM	F1
stochastic-answer-networks-for-machine	79.608	86.496
모델 2	71.908	81.023
harvesting-and-refining-question-answer-pairs	55.827	65.467
모델 4	68.331	77.783
information-theoretic-representation	77.7	85.8
adaptation-of-deep-bidirectional-multilingual	-	84.6
모델 7	84.926	91.932
모델 8	90.622	95.719
모델 9	85.314	91.756
phase-conductor-on-multi-layered-attentions	74.405	82.742
모델 11	78.664	85.780
a-large-batch-optimizer-reality-check	-	91.58
fusionnet-fusing-via-fully-aware-attention	75.968	83.900
모델 14	84.402	90.561
dyrex-dynamic-query-representation-for	-	91.01
모델 16	0.000	0.000
모델 17	74.121	82.342
contextualized-word-representations-for	75.789	83.261
harvesting-and-refining-question-answer-pairs	61.145	71.389
memen-multi-layer-embedding-with-memory	78.234	85.344
모델 21	84.328	91.281
end-to-end-answer-chunk-extraction-and	62.499	70.956
모델 23	81.003	87.432
multi-perspective-context-matching-for	73.765	81.257
모델 25	55.827	65.467
모델 26	86.458	92.645
exploring-machine-reading-comprehension-with	76.125	83.538
모델 28	79.996	86.711
stochastic-answer-networks-for-machine	76.828	84.396
모델 30	70.985	79.939
모델 31	75.034	83.405
모델 32	66.527	75.787
bert-pre-training-of-deep-bidirectional	87.4	93.2
모델 34	75.265	82.769
모델 35	71.016	79.835
information-theoretic-representation	81.5	88.5
making-neural-qa-as-simple-as-possible-but	70.849	78.857
모델 38	69.600	78.236
모델 39	59.058	69.436
모델 40	76.859	84.739
모델 41	72.485	80.550
multi-perspective-context-matching-for	70.387	78.784
모델 43	85.356	91.202
bert-pre-training-of-deep-bidirectional	87.433	93.160
textbox-2-0-a-text-generation-library-with	-	93.04
learning-to-compute-word-embeddings-on-the	62.897	72.016
모델 47	85.125	91.623
모델 48	73.639	81.931
simple-and-effective-multi-paragraph-reading	72.139	81.048
모델 50	61.145	71.389
dcn-mixed-objective-and-deep-residual	74.866	82.806
machine-comprehension-using-match-lstm-and	67.901	77.022
machine-comprehension-using-match-lstm-and	64.744	73.743
모델 54	80.615	87.311
fusionnet-fusing-via-fully-aware-attention	78.978	86.016
모델 56	79.199	86.590
모델 57	81.790	88.163
words-or-characters-fine-grained-gating-for	62.446	73.327
모델 59	83.426	89.218
efficientqa-a-roberta-based-phrase-indexed	74.9	83.1
모델 61	47.341	56.436
모델 62	0.000	0.000
모델 63	80.027	87.288
모델 64	52.544	62.780
learning-to-compute-word-embeddings-on-the	62.604	71.968
모델 66	79.692	86.727
deep-contextualized-word-representations	81.003	87.432
spanbert-improving-pre-training-by	88.8	94.6
smarnet-teaching-machines-to-read-and	71.415	80.160
모델 70	75.821	83.843
모델 71	89.709	94.859
모델 72	82.681	89.379
모델 73	79.083	86.288
모델 74	71.698	80.462
qanet-combining-local-convolution-with-global	76.2	84.6
making-neural-qa-as-simple-as-possible-but	68.436	77.070
reasonet-learning-to-stop-reading-in-machine	70.555	79.364
모델 78	69.443	78.358
모델 79	0.000	0.000
모델 80	71.898	79.989
structural-embedding-of-syntactic-trees-for	74.090	81.761
모델 82	67.618	77.151
모델 83	83.982	89.796
모델 84	80.436	87.021
모델 85	72.758	81.001
모델 86	0.000	0.000
모델 87	82.482	89.281
모델 88	89.646	94.930
memen-multi-layer-embedding-with-memory	75.370	82.658
모델 90	77.237	84.466
모델 91	85.944	92.425
모델 92	78.087	85.348
모델 93	88.912	94.584
bidirectional-attention-flow-for-machine	67.974	77.323
모델 95	83.804	90.429
모델 96	81.580	88.948
모델 97	68.132	77.569
모델 98	74.489	82.815
모델 99	89.856	94.903
모델 100	79.901	86.536
모델 101	64.439	73.921
모델 102	86.521	92.617
모델 103	77.646	84.905
모델 104	78.664	85.780
모델 105	75.989	83.475
reading-wikipedia-to-answer-open-domain	70.733	79.353
structural-embedding-of-syntactic-trees-for	68.478	77.971
모델 108	78.328	85.682
모델 109	64.932	74.594
exploring-question-understanding-and	73.010	81.517
모델 111	88.912	94.584
모델 112	78.171	85.543
모델 113	79.083	86.288
모델 114	63.306	73.463
모델 115	0.000	6.907
bidirectional-attention-flow-for-machine	73.744	81.525
모델 117	78.496	85.469
exploring-question-understanding-and	70.607	79.821
모델 119	79.031	86.006
a-multi-stage-memory-augmented-neural-network	79.692	86.727
deep-contextualized-word-representations	78.58	85.833
모델 122	85.944	92.425
모델 123	73.303	81.754
모델 124	82.062	88.947
모델 125	65.163	74.555
모델 126	80.426	86.912
모델 127	53.698	64.036
모델 128	76.240	84.599
phase-conductor-on-multi-layered-attentions	73.240	81.933
xlnet-generalized-autoregressive-pretraining	89.898	95.080
bert-pre-training-of-deep-bidirectional	85.083	91.835
모델 132	81.401	88.122
모델 133	72.600	81.011
모델 134	78.328	85.682
모델 135	81.307	88.909
gated-self-matching-networks-for-reading	76.461	84.265
모델 137	67.544	76.429
모델 138	77.342	84.925
dynamic-coattention-networks-for-question	71.625	80.383
모델 140	80.164	86.721
luke-deep-contextualized-entity	90.202	95.379
모델 142	66.516	76.349
luke-deep-contextualized-entity	-	95.4
reinforced-mnemonic-reader-for-machine	82.283	88.533
모델 145	88.650	94.393
모델 146	76.461	84.265
모델 147	0.000	0.000
모델 148	77.845	85.297
모델 149	80.720	87.758
모델 150	75.223	82.716
모델 151	90.202	95.379
모델 152	76.775	84.491
모델 153	83.468	90.133
모델 154	44.215	54.723
machine-comprehension-using-match-lstm-and	60.474	70.695
모델 156	79.083	86.450
a-fully-attention-based-information-retriever	67.744	77.605
dynamic-coattention-networks-for-question	66.233	75.896
모델 159	75.926	83.305
모델 160	85.430	91.976
모델 161	76.146	83.991
reinforced-mnemonic-reader-for-machine	79.545	86.654
모델 163	67.502	76.786
reinforced-mnemonic-reader-for-machine	70.995	80.146
reasonet-learning-to-stop-reading-in-machine	75.034	82.552
모델 166	81.496	87.557
모델 167	81.045	87.999
contextualized-word-representations-for	77.583	84.163
reinforced-mnemonic-reader-for-machine	74.268	82.371
모델 170	80.667	88.169
모델 171	74.080	81.665
모델 172	71.373	79.725
모델 173	78.401	85.724
모델 174	87.465	93.294
모델 175	78.653	86.663
모델 176	72.590	81.415
luke-deep-contextualized-entity	90.2	-
모델 178	12.273	13.211
모델 179	74.604	82.501
phase-conductor-on-multi-layered-attentions	76.996	84.630
모델 181	78.580	85.833
structural-embedding-of-syntactic-trees-for	73.723	81.530
모델 183	82.849	88.764
모델 184	89.898	95.080
bert-pre-training-of-deep-bidirectional	-	91.8
ruminating-reader-reasoning-with-gated-multi	70.639	79.456
structural-embedding-of-syntactic-trees-for	68.163	77.527
모델 188	82.471	89.306
memen-multi-layer-embedding-with-memory	78.234	85.344
모델 190	77.573	84.858
learning-to-compute-word-embeddings-on-the	64.083	73.056
linkbert-pretraining-language-models-with	87.45	92.7
모델 193	78.223	85.535
모델 194	88.839	94.635
모델 195	77.090	83.931
모델 196	84.978	92.019
learned-in-translation-contextualized-word	71.3	79.9
모델 198	79.859	88.263
모델 199	82.136	88.126
모델 200	86.940	92.641
모델 201	79.597	87.374
learning-recurrent-span-representations-for	70.849	78.741
simple-recurrent-units-for-highly	71.4	80.2
모델 204	83.930	90.613
모델 205	80.436	86.912
모델 206	82.650	88.493
모델 207	79.083	86.288
dcn-mixed-objective-and-deep-residual	78.852	85.996
모델 209	80.489	87.454
모델 210	82.440	88.607
모델 211	85.335	91.807
모델 212	52.533	62.757
machine-comprehension-using-match-lstm-and	54.505	67.748