Deep Matching Prior: Test-Time Optimization for Dense Correspondence

Conventional techniques to establish dense correspondences across visually orsemantically similar images focused on designing a task-specific matchingprior, which is difficult to model. To overcome this, recent learning-basedmethods have attempted to learn a good matching prior within a model itself onlarge training data. The performance improvement was apparent, but the need forsufficient training data and intensive learning hinders their applicability.Moreover, using the fixed model at test time does not account for the fact thata pair of images may require their own prior, thus providing limitedperformance and poor generalization to unseen images. In this paper, we showthat an image pair-specific prior can be captured by solely optimizing theuntrained matching networks on an input pair of images. Tailored for suchtest-time optimization for dense correspondence, we present a residual matchingnetwork and a confidence-aware contrastive loss to guarantee a meaningfulconvergence. Experiments demonstrate that our framework, dubbed Deep MatchingPrior (DMP), is competitive, or even outperforms, against the latestlearning-based methods on several benchmarks for geometric matching andsemantic matching, even though it requires neither large training data norintensive learning. With the networks pre-trained, DMP attains state-of-the-artperformance on all benchmarks.