Non-parametric Uni-modality Constraints for Deep Ordinal Classification

We propose a new constrained-optimization formulation for deep ordinalclassification, in which uni-modality of the label distribution is enforcedimplicitly via a set of inequality constraints over all the pairs of adjacentlabels. Based on (c-1) constraints for c labels, our model is non-parametricand, therefore, more flexible than the existing deep ordinal classificationtechniques. Unlike these, it does not restrict the learned representation to asingle and specific parametric model (or penalty) imposed on all the labels.Therefore, it enables the training to explore larger spaces of solutions, whileremoving the need for ad hoc choices and scaling up to large numbers of labels.It can be used in conjunction with any standard classification loss and anydeep architecture. To tackle the ensuing challenging optimization problem, wesolve a sequence of unconstrained losses based on a powerful extension of thelog-barrier method. This handles effectively competing constraints and accommodates standard SGDfor deep networks, while avoiding computationally expensive Lagrangian dualsteps and outperforming substantially penalty methods. Furthermore, we proposea new performance metric for ordinal classification, as a proxy to measuredistribution uni-modality, referred to as the Sides Order Index (SOI). Wereport comprehensive evaluations and comparisons to state-of-the-art methods onbenchmark public datasets for several ordinal classification tasks, showing themerits of our approach in terms of label consistency, classification accuracyand scalability. Importantly, enforcing label consistency with our model doesnot incur higher classification errors, unlike many existing ordinalclassification methods. A public reproducible PyTorch implementation isprovided.(https://github.com/sbelharbi/unimodal-prob-deep-oc-free-distribution)