Anomaly Detection in Video via Self-Supervised and Multi-Task Learning

Anomaly detection in video is a challenging computer vision problem. Due tothe lack of anomalous events at training time, anomaly detection requires thedesign of learning methods without full supervision. In this paper, we approachanomalous event detection in video through self-supervised and multi-tasklearning at the object level. We first utilize a pre-trained detector to detectobjects. Then, we train a 3D convolutional neural network to producediscriminative anomaly-specific information by jointly learning multiple proxytasks: three self-supervised and one based on knowledge distillation. Theself-supervised tasks are: (i) discrimination of forward/backward movingobjects (arrow of time), (ii) discrimination of objects inconsecutive/intermittent frames (motion irregularity) and (iii) reconstructionof object-specific appearance information. The knowledge distillation tasktakes into account both classification and detection information, generatinglarge prediction discrepancies between teacher and student models whenanomalies occur. To the best of our knowledge, we are the first to approachanomalous event detection in video as a multi-task learning problem,integrating multiple self-supervised and knowledge distillation proxy tasks ina single architecture. Our lightweight architecture outperforms thestate-of-the-art methods on three benchmarks: Avenue, ShanghaiTech and UCSDPed2. Additionally, we perform an ablation study demonstrating the importanceof integrating self-supervised learning and normality-specific distillation ina multi-task learning setting.