Vision And Language Navigation On Vln

Metrics

error

length

oracle success

spl

success

Results

Performance results of various models on this benchmark

Comparison Table

Model Name	error	length	oracle success	spl	success
Model 1	2.99	686.86	0.99	0.01	0.74
Model 2	3.97	13.75	0.71	0.58	0.64
Model 3	5.45	10.65	0.6	0.48	0.52
Model 4	3.3	9.85	0.77	0.68	0.7
Model 5	4.32	13.43	0.69	0.55	0.61
Model 6	4.49	12.22	0.67	0.54	0.59
Model 7	4.96	12.51	0.61	0.48	0.54
Model 8	3.61	14.6	0.77	0.59	0.7
Model 9	4.57	10.99	0.65	0.5	0.57
vision-language-navigation-with-random	3.87	13.11	0.72	0.59	0.65
Model 11	3.38	15.47	0.79	0.59	0.71
self-monitoring-navigation-agent-via	5.67	18.04	0.59	0.35	0.48
Model 13	6.46	13.08	0.45	0.3	0.37
Model 14	5.37	10.0	0.59	0.5	0.53
Model 15	3.31	786.35	0.99	0.01	0.69
Model 16	4.65	12.19	0.62	0.52	0.56
Model 17	4.71	10.31	0.64	0.55	0.59
Model 18	3.73	15.89	0.73	0.6	0.66
Model 19	6.12	11.97	0.5	0.38	0.43
Model 20	3.44	16.14	0.79	0.6	0.72
Model 21	5.42	14.07	0.55	0.4	0.47
Model 22	5.63	12.13	0.61	0.49	0.53
Model 23	12.29	45.13	0.49	0.02	0.07
improving-vision-and-language-navigation-with	3.09	686.62	0.99	0.01	0.73
Model 25	4.57	1214.94	0.96	0.01	0.56
Model 26	5.5	14.42	0.57	0.41	0.49
Model 27	3.77	11.89	0.72	0.63	0.66
Model 28	7.85	8.13	0.27	0.18	0.2
Model 29	4.61	13.16	0.65	0.5	0.57
Model 30	4.21	9.48	0.67	0.59	0.6
Model 31	4.45	13.0	0.67	0.53	0.58
Model 32	4.27	12.13	0.67	0.56	0.61
Model 33	4.52	10.21	0.64	0.56	0.59
Model 34	6.01	10.47	0.53	0.4	0.45
Model 35	5.35	11.74	0.64	0.5	0.54
neighbor-view-enhanced-model-for-vision-and	4.37	12.98	0.66	0.54	0.58
Model 37	4.09	12.35	0.7	0.57	0.63
Model 38	4.53	10.52	0.63	0.53	0.57
environment-agnostic-multitask-learning-for	6.03	13.35	0.56	0.4	0.45
Model 40	3.78	15.9	0.71	0.59	0.65
Model 41	5.3	10.51	0.61	0.51	0.54
Model 42	7.47	10.92	0.34	0.21	0.26
Model 43	5.14	13.05	0.6	0.45	0.51
Model 44	5.3	10.4	0.61	0.5	0.53
Model 45	4.26	12.73	0.67	0.55	0.61
Model 46	4.48	373.09	0.97	0.02	0.61
Model 47	4.75	10.29	0.61	0.52	0.55
Model 48	4.18	10.22	0.67	0.58	0.62
Model 49	3.05	1250.89	1.0	0.01	0.72
Model 50	3.52	11.79	0.75	0.65	0.7
Model 51	6.61	13.85	0.46	0.3	0.36
vision-language-navigation-with-self	3.69	10.43	0.75	0.65	0.68
Model 53	9.79	9.89	0.18	0.12	0.13
Model 54	3.65	14.73	0.76	0.59	0.69
Model 55	4.02	10.66	0.72	0.55	0.61
Model 56	3.13	15.87	0.81	0.62	0.73
transferable-representation-learning-in	5.49	10.27	0.56	0.45	0.48
Model 58	3.99	12.62	0.71	0.58	0.63
Model 59	5.17	10.58	0.63	0.52	0.56
Model 60	6.55	9.81	0.45	0.31	0.35
Model 61	5.29	11.39	0.65	0.53	0.56
Model 62	4.59	12.34	0.65	0.53	0.57
Model 63	4.09	12.35	0.7	0.57	0.63
Model 64	4.53	13.07	0.66	0.53	0.59
Model 65	3.55	14.39	0.76	0.6	0.7
Model 66	3.82	12.71	0.72	0.6	0.65
Model 67	4.34	21.03	0.71	0.43	0.6
Model 68	4.9	12.96	0.62	0.5	0.55
Model 69	3.07	176.22	0.94	0.05	0.71
tactical-rewind-self-correction-via	5.14	22.08	0.64	0.41	0.54
Model 71	3.26	10.21	0.77	0.67	0.71
learning-to-navigate-unseen-environments-back	5.23	11.66	0.59	0.47	0.51
Model 73	5.32	10.9	0.63	0.51	0.55
Model 74	6.89	8.32	0.41	0.32	0.34
Model 75	3.55	625.27	0.99	0.01	0.74
the-regretful-agent-heuristic-aided	5.69	13.69	0.56	0.4	0.48
Model 77	4.09	12.35	0.7	0.57	0.63
Model 78	3.86	12.31	0.71	0.59	0.64
Model 79	3.94	12.3	0.71	0.59	0.64
Model 80	3.77	16.41	0.71	0.59	0.66
Model 81	5.8	16.73	0.57	0.36	0.47
Model 82	8.82	9.56	0.24	0.16	0.18
Model 83	7.63	22.14	0.61	0.2	0.3
Model 84	7.53	9.15	0.32	0.23	0.25
Model 85	3.41	12.07	0.76	0.6	0.67
Model 86	7.2	15.9	0.4	0.23	0.29
Model 87	5.64	10.42	0.53	0.43	0.47
airbert-in-domain-pretraining-for-vision-and	2.58	686.54	0.99	0.01	0.78
Model 89	6.95	10.44	0.43	0.31	0.36
Model 90	3.68	14.78	0.72	0.6	0.66
Model 91	4.27	16.94	0.72	0.49	0.62
Model 92	3.97	9.79	0.7	0.61	0.64
Model 93	4.03	357.62	0.96	0.02	0.63
Model 94	8.56	9.48	0.32	0.22	0.24
learning-to-navigate-unseen-environments-back	3.26	686.82	0.99	0.01	0.69
Model 96	3.67	13.07	0.73	0.6	0.66
Model 97	5.24	14.31	0.64	0.46	0.54
Model 98	4.0	16.44	0.69	0.58	0.63
Model 99	6.17	10.17	0.47	0.36	0.4
Model 100	4.95	10.71	0.65	0.55	0.58
Model 101	3.9	12.84	0.7	0.58	0.64
Model 102	4.02	13.57	0.71	0.57	0.63
tactical-rewind-self-correction-via	4.29	196.53	0.9	0.03	0.61
Model 104	4.22	13.2	0.69	0.56	0.61
Model 105	1.61	11.85	0.9	0.76	0.86
Model 106	3.59	11.9	0.73	0.64	0.68
Model 107	4.3	15.83	0.68	0.55	0.61
Model 108	3.86	12.75	0.72	0.6	0.66
Model 109	4.89	10.18	0.63	0.53	0.56
Model 110	4.84	15.74	0.69	0.48	0.56
Model 111	3.76	10.24	0.71	0.62	0.65
Model 112	3.0	10.2	0.8	0.69	0.73
Model 113	4.0	11.91	0.7	0.6	0.65
Model 114	3.78	15.9	0.71	0.59	0.65
Model 115	7.99	8.26	0.26	0.18	0.2
Model 116	3.85	19.62	0.74	0.55	0.66
Model 117	3.35	14.43	0.8	0.62	0.73
Model 118	3.74	13.55	0.73	0.61	0.67
Model 119	5.56	16.03	0.57	0.35	0.47
Model 120	5.45	11.15	0.57	0.47	0.51
Model 121	4.87	1257.38	0.96	0.01	0.53
Model 122	6.67	10.83	0.44	0.33	0.36
Model 123	9.8	9.91	0.19	0.12	0.14
Model 124	6.68	15.75	0.43	0.25	0.33
Model 125	7.03	15.1	0.4	0.25	0.31
Model 126	9.93	0.0	0.0	0.0	0.0
Model 127	3.21	690.61	0.99	0.01	0.7
Model 128	3.96	12.77	0.7	0.57	0.63
Model 129	2.5	686.45	0.99	0.01	0.79
Model 130	4.04	13.02	0.7	0.58	0.63
Model 131	4.16	12.51	0.69	0.58	0.63
vision-language-navigation-with-self	3.24	40.85	0.81	0.21	0.71
Model 133	4.57	20.39	0.7	0.46	0.61
Model 134	3.33	15.24	0.78	0.61	0.72
Model 135	3.86	12.22	0.71	0.61	0.65
Model 136	3.93	12.27	0.72	0.6	0.65
Model 137	5.11	10.06	0.62	0.52	0.54
Model 138	3.88	12.6	0.71	0.59	0.64
Model 139	5.7	11.47	0.57	0.47	0.51
Model 140	3.57	14.25	0.77	0.61	0.71
Model 141	4.07	12.3	0.7	0.58	0.63
Model 142	3.24	11.86	0.76	0.62	0.69
Model 143	4.26	14.29	0.66	0.55	0.59
Model 144	5.34	15.02	0.61	0.42	0.53
Model 145	3.98	13.54	0.7	0.58	0.63