Understanding broadcast videos is a challenging task in computer vision, asit requires generic reasoning capabilities to appreciate the content offered bythe video editing. In this work, we propose SoccerNet-v2, a novel large-scalecorpus of manual annotations for the SoccerNet video dataset, along with openchallenges to encourage more research in soccer understanding and broadcastproduction. Specifically, we release around 300k annotations within SoccerNet's500 untrimmed broadcast soccer videos. We extend current tasks in the realm ofsoccer to include action spotting, camera shot segmentation with boundarydetection, and we define a novel replay grounding task. For each task, weprovide and discuss benchmark results, reproducible with our open-sourceadapted implementations of the most relevant works in the field. SoccerNet-v2is presented to the broader research community to help push computer visioncloser to automatic solutions for more general video understanding andproduction purposes.