Visual Question Answering On Gqa Test2019

Metrics

Accuracy

Binary

Consistency

Distribution

Open

Plausibility

Validity

Results

Performance results of various models on this benchmark

Comparison Table

Model Name	Accuracy	Binary	Consistency	Distribution	Open	Plausibility	Validity
Model 1	60.98	77.32	90.77	5.36	46.55	84.93	96.38
Model 2	60.59	78.44	92.66	7.28	44.83	85.38	96.57
Model 3	46.55	63.26	74.57	7.46	31.8	84.25	96.02
Model 4	64.92	82.63	94.37	5.11	49.29	84.91	96.64
Model 5	59.37	77.53	88.63	6.06	43.35	84.71	96.18
Model 6	59.8	76.74	89.14	5.11	44.85	84.2	96.23
Model 7	60.17	77.19	89.61	5.83	45.14	84.46	96.36
Model 8	59.29	77.31	88.94	5.8	43.38	84.43	96.3
Model 9	37.03	56.61	63.96	28.4	19.74	85.12	95.76
Model 10	57.35	75.07	87.61	5.94	41.71	84.5	95.86
vinvl-l-enriching-visual-representation-with	64.85	82.59	94.0	4.59	49.19	84.91	96.62
Model 12	55.0	72.09	83.47	5.29	39.92	84.66	96.34
bottom-up-and-top-down-attention-for-image	49.74	66.64	78.71	5.98	34.83	84.57	96.18
Model 14	57.89	74.54	85.45	5.73	43.19	84.99	96.4
graph-reasoning-networks-for-visual-question	61.22	78.69	90.31	6.77	45.81	85.43	96.36
Model 16	56.95	74.62	87.71	5.81	41.36	84.57	95.98
Model 17	60.18	76.97	89.65	5.29	45.36	84.47	96.33
Model 18	63.17	78.94	93.25	3.71	49.25	84.28	96.41
Model 19	55.65	72.86	89.18	9.69	40.46	85.27	96.33
Model 20	56.96	74.97	85.12	7.13	41.06	84.85	96.38
Model 21	60.27	76.99	90.16	5.39	45.51	84.49	96.31
Model 22	60.02	76.37	90.05	5.63	45.59	84.34	96.29
Model 23	55.93	71.81	83.2	6.05	41.93	85.09	96.01
Model 24	59.93	79.09	93.72	10.1	43.02	85.92	96.41
Model 25	63.2	77.91	89.84	5.25	50.22	85.15	96.47
Model 26	89.3	91.2	98.4	0.0	87.4	97.2	98.9
Model 27	59.43	77.11	89.05	6.39	43.82	84.94	96.56
Model 28	56.18	72.84	85.46	5.42	41.47	84.04	96.18
Model 29	44.06	57.57	38.18	8.35	32.13	75.19	85.94
Model 30	59.54	77.98	89.21	6.01	43.26	84.94	96.24
Model 31	61.1	77.99	91.08	5.52	46.19	84.82	96.36
Model 32	56.65	73.65	84.35	6.07	41.64	84.37	95.94
Model 33	56.59	73.0	84.7	4.68	42.11	84.86	96.4
Model 34	51.22	69.36	82.44	6.45	35.2	83.82	96.12
Model 35	60.14	77.15	89.58	5.81	45.12	84.47	96.36
Model 36	57.1	76.0	91.7	10.52	40.41	85.58	96.16
Model 37	61.09	77.84	88.92	5.68	46.3	85.49	96.43
Model 38	60.67	78.02	89.81	6.41	45.36	84.84	96.31
Model 39	53.57	70.15	81.14	5.32	38.94	84.67	96.36
Model 40	61.05	78.02	89.77	5.24	46.06	84.95	96.5
Model 41	62.44	80.28	94.36	5.33	46.69	84.91	96.46
vinvl-making-visual-representations-matter-in	64.65	82.63	94.35	4.72	48.77	84.98	96.62
Model 43	57.21	74.46	87.6	5.6	41.99	84.87	96.2
Model 44	17.82	36.05	62.4	19.99	1.74	34.84	35.78
Model 45	54.15	69.3	82.36	5.41	40.79	85.15	95.99
Model 46	58.06	76.6	90.96	7.6	41.7	85.27	96.31
lxmert-learning-cross-modality-encoder	62.71	79.79	93.1	6.42	47.64	85.21	96.36
Model 48	59.81	78.02	91.43	6.0	43.75	84.77	96.5
Model 49	28.9	42.94	51.69	93.08	16.62	74.81	88.86
Model 50	55.7	72.88	83.52	5.32	40.53	84.81	96.39
Model 51	57.65	75.22	87.35	5.48	42.14	84.73	96.18
Model 52	40.3	61.18	74.11	40.44	21.88	86.13	96.14
Model 53	70.23	77.5	86.94	1.49	63.82	83.77	96.65
Model 54	57.14	75.07	87.36	5.29	41.31	84.49	95.87
Model 55	48.97	63.85	83.85	13.72	35.83	83.93	95.62
Model 56	59.12	76.69	88.9	5.6	43.6	84.78	96.43
Model 57	56.16	73.56	84.99	5.87	40.8	84.83	96.4
Model 58	53.31	70.41	80.33	6.4	38.23	84.32	95.99
Model 59	48.44	65.02	81.19	17.79	33.81	85.29	96.15
Model 60	60.07	76.84	89.32	6.21	45.27	84.55	96.35
Model 61	36.75	55.24	69.93	40.84	20.44	84.13	95.1
Model 62	60.42	77.12	89.69	6.03	45.68	84.56	96.35
Model 63	63.94	80.84	91.54	4.69	49.03	84.74	96.56
Model 64	51.87	67.99	80.2	6.77	37.64	84.35	96.25
Model 65	60.37	77.09	89.77	6.43	45.61	84.56	96.22
Model 66	52.19	69.15	78.34	5.69	37.22	83.44	95.45
Model 67	72.14	81.16	90.96	2.39	64.19	84.81	96.77
Model 68	61.12	78.07	91.13	5.55	46.16	84.8	96.36
Model 69	60.93	77.83	90.3	5.74	46.01	84.69	96.35
Model 70	60.51	76.87	88.2	8.48	46.06	85.19	96.15
Model 71	49.28	67.59	83.68	14.28	33.12	83.41	94.95
Model 72	76.04	84.46	91.47	3.68	68.6	83.75	96.42
Model 73	45.86	64.74	70.57	8.38	29.2	86.13	96.61
Model 74	41.63	55.12	82.21	13.01	29.73	77.4	92.27
Model 75	61.49	78.4	88.68	5.7	46.56	84.85	96.33
Model 76	55.41	72.87	83.06	5.48	39.99	84.74	96.35
Model 77	26.45	45.69	55.23	11.49	9.47	50.93	60.81
Model 78	53.89	72.52	87.47	8.66	37.44	85.05	96.39
Model 79	55.57	72.39	83.32	10.18	40.74	84.24	96.15
Model 80	60.89	78.07	93.02	5.31	45.73	84.05	96.0
Model 81	60.87	79.12	92.61	8.56	44.76	85.63	96.35
Model 82	31.24	47.9	54.04	13.98	16.66	84.31	84.33
Model 83	54.06	71.23	81.59	5.34	38.91	84.48	96.16
Model 84	60.7	77.41	89.65	6.09	45.96	84.55	96.37
Model 85	47.72	66.28	84.16	19.05	31.34	84.52	95.45
Model 86	49.27	66.57	78.51	6.91	34.0	84.58	95.78
Model 87	41.07	61.9	68.68	17.93	22.69	87.3	96.39
Model 88	51.51	67.82	79.7	6.16	37.11	83.69	95.82
Model 89	56.09	73.4	85.11	5.14	40.82	84.79	96.37
Model 90	56.11	72.65	85.51	5.42	41.52	84.36	96.25
Model 91	57.79	75.37	88.3	5.65	42.26	84.85	96.11
Model 92	62.45	80.91	93.95	5.36	46.15	84.15	96.33
Model 93	58.91	76.08	89.52	6.93	43.75	84.52	96.18
Model 94	56.95	75.01	90.49	9.5	41.02	85.46	96.37
Model 95	60.83	78.9	92.49	5.54	44.89	84.55	96.19
lxmert-learning-cross-modality-encoder	60.33	77.16	89.59	5.69	45.47	84.53	96.35
Model 97	60.95	78.41	89.08	4.86	45.54	84.27	96.35
Model 98	56.38	74.84	91.71	6.32	40.09	83.76	95.43
Model 99	58.2	75.91	88.25	5.81	42.57	84.72	96.08
Model 100	56.28	73.73	86.86	5.78	40.87	84.2	96.01
Model 101	57.77	75.78	86.85	5.36	41.86	84.97	96.44
Model 102	58.72	76.4	89.58	6.58	43.11	84.68	96.21
Model 103	42.75	61.21	63.51	7.63	26.45	84.2	95.99
Model 104	73.33	79.68	77.02	2.46	67.73	83.7	96.36
Model 105	54.79	72.42	86.1	6.01	39.23	84.55	95.92
Model 106	52.3	68.46	84.36	12.54	38.04	85.2	96.2
Model 107	59.84	76.79	89.52	6.06	44.89	84.72	96.2
Model 108	60.18	76.84	89.77	5.65	45.48	84.6	96.37
Model 109	43.84	59.24	67.71	10.99	30.24	84.01	95.32
Model 110	59.72	77.97	89.43	6.25	43.61	84.89	96.55
Model 111	60.28	77.13	89.47	5.38	45.41	84.45	96.33
Model 112	58.12	76.39	88.01	5.65	42.0	84.8	96.06
Model 113	58.88	75.07	84.64	5.54	44.58	84.86	96.23
Model 114	60.01	76.77	89.17	6.28	45.21	84.46	96.35
Model 115	59.06	76.07	89.81	6.14	44.04	82.76	93.82
Model 116	58.42	77.39	90.29	7.86	41.67	84.53	95.57
Model 117	55.35	72.65	84.17	5.22	40.08	84.56	96.32
Model 118	53.85	68.44	80.2	5.84	40.97	85.19	96.28
Model 119	54.94	71.7	82.71	5.1	40.14	84.78	96.4
Model 120	73.81	80.8	91.76	1.7	67.64	83.9	96.73
Model 121	67.55	80.45	93.83	2.78	56.16	84.16	96.53
Model 122	57.07	73.77	84.68	4.7	42.33	84.81	96.48
Model 123	56.0	73.9	87.16	6.02	40.2	84.45	96.01
Model 124	57.01	74.78	87.74	6.06	41.32	84.25	96.03
Model 125	52.02	67.35	80.44	5.64	38.5	83.94	95.75
Model 126	74.03	82.12	89.0	1.29	66.89	83.58	96.76
Model 127	47.38	58.76	73.71	6.29	37.34	81.75	94.55