Improving RetinaNet for CT Lesion Detection with Dense Masks from Weak RECIST Labels

Accurate, automated lesion detection in Computed Tomography (CT) is animportant yet challenging task due to the large variation of lesion types,sizes, locations and appearances. Recent work on CT lesion detection employstwo-stage region proposal based methods trained with centroid or bounding-boxannotations. We propose a highly accurate and efficient one-stage lesiondetector, by re-designing a RetinaNet to meet the particular challenges inmedical imaging. Specifically, we optimize the anchor configurations using adifferential evolution search algorithm. For training, we leverage the responseevaluation criteria in solid tumors (RECIST) annotation which are measured inclinical routine. We incorporate dense masks from weak RECIST labels, obtainedautomatically using GrabCut, into the training objective, which in combinationwith other advancements yields new state-of-the-art performance. We evaluateour method on the public DeepLesion benchmark, consisting of 32,735 lesionsacross the body. Our one-stage detector achieves a sensitivity of 90.77% at 4false positives per image, significantly outperforming the best reportedmethods by over 5%.