Semi-Supervised Visual Representation Learning for Fashion Compatibility

We consider the problem of complementary fashion prediction. Existingapproaches focus on learning an embedding space where fashion items fromdifferent categories that are visually compatible are closer to each other.However, creating such labeled outfits is intensive and also not feasible togenerate all possible outfit combinations, especially with large fashioncatalogs. In this work, we propose a semi-supervised learning approach where weleverage large unlabeled fashion corpus to create pseudo-positive andpseudo-negative outfits on the fly during training. For each labeled outfit ina training batch, we obtain a pseudo-outfit by matching each item in thelabeled outfit with unlabeled items. Additionally, we introduce consistencyregularization to ensure that representation of the original images and theirtransformations are consistent to implicitly incorporate colour and otherimportant attributes through self-supervision. We conduct extensive experimentson Polyvore, Polyvore-D and our newly created large-scale Fashion Outfitsdatasets, and show that our approach with only a fraction of labeled examplesperforms on-par with completely supervised methods.