Unsustainable fishing practices worldwide pose a major threat to marineresources and ecosystems. Identifying vessels that do not show up inconventional monitoring systems -- known as ``dark vessels'' -- is key tomanaging and securing the health of marine environments. With the rise ofsatellite-based synthetic aperture radar (SAR) imaging and modern machinelearning (ML), it is now possible to automate detection of dark vessels day ornight, under all-weather conditions. SAR images, however, require adomain-specific treatment and are not widely accessible to the ML community.Maritime objects (vessels and offshore infrastructure) are relatively small andsparse, challenging traditional computer vision approaches. We present thelargest labeled dataset for training ML models to detect and characterizevessels and ocean structures in SAR imagery. xView3-SAR consists of nearly1,000 analysis-ready SAR images from the Sentinel-1 mission that are, onaverage, 29,400-by-24,400 pixels each. The images are annotated using acombination of automated and manual analysis. Co-located bathymetry and windstate rasters accompany every SAR image. We also provide an overview of thexView3 Computer Vision Challenge, an international competition using xView3-SARfor ship detection and characterization at large scale. We release the data(\href{https://iuu.xview.us/}{https://iuu.xview.us/}) and code(\href{https://github.com/DIUx-xView}{https://github.com/DIUx-xView}) tosupport ongoing development and evaluation of ML approaches for this importantapplication.