CARAFE: Content-Aware ReAssembly of FEatures

Feature upsampling is a key operation in a number of modern convolutionalnetwork architectures, e.g. feature pyramids. Its design is critical for denseprediction tasks such as object detection and semantic/instance segmentation.In this work, we propose Content-Aware ReAssembly of FEatures (CARAFE), auniversal, lightweight and highly effective operator to fulfill this goal.CARAFE has several appealing properties: (1) Large field of view. Unlikeprevious works (e.g. bilinear interpolation) that only exploit sub-pixelneighborhood, CARAFE can aggregate contextual information within a largereceptive field. (2) Content-aware handling. Instead of using a fixed kernelfor all samples (e.g. deconvolution), CARAFE enables instance-specificcontent-aware handling, which generates adaptive kernels on-the-fly. (3)Lightweight and fast to compute. CARAFE introduces little computationaloverhead and can be readily integrated into modern network architectures. Weconduct comprehensive evaluations on standard benchmarks in object detection,instance/semantic segmentation and inpainting. CARAFE shows consistent andsubstantial gains across all the tasks (1.2%, 1.3%, 1.8%, 1.1db respectively)with negligible computational overhead. It has great potential to serve as astrong building block for future research. It has great potential to serve as astrong building block for future research. Code and models are available athttps://github.com/open-mmlab/mmdetection.