The Self-Optimal-Transport Feature Transform

The Self-Optimal-Transport (SOT) feature transform is designed to upgrade theset of features of a data instance to facilitate downstream matching orgrouping related tasks. The transformed set encodes a rich representation ofhigh order relations between the instance features. Distances betweentransformed features capture their direct original similarity and their thirdparty agreement regarding similarity to other features in the set. A particularmin-cost-max-flow fractional matching problem, whose entropy regularizedversion can be approximated by an optimal transport (OT) optimization, resultsin our transductive transform which is efficient, differentiable, equivariant,parameterless and probabilistically interpretable. Empirically, the transformis highly effective and flexible in its use, consistently improving networks itis inserted into, in a variety of tasks and training schemes. We demonstrateits merits through the problem of unsupervised clustering and its efficiencyand wide applicability for few-shot-classification, with state-of-the-artresults, and large-scale person re-identification.