Inter-category correspondences emerge from dense pose prediction. Our method discovers high-quality correspondences between different object classes automatically, as a byproduct of learning category-specific dense pose predictors. It does so by enforcing cycle consistency between reference 3D templates as well as by a new type of consistency between images and templates. This allows the model to transfer information between animal classes (eg the location of the eyes).

Abstract

We tackle the problem of learning the geometry of multiple categories of deformable objects jointly. Recent work has shown that it is possible to learn a unified dense pose predictor for several categories of related objects. However, training such models requires to initialize inter-category correspondences by hand. This is suboptimal and the resulting models fail to maintain correct correspondences as individual categories are learned. In this paper, we show that improved correspondences can be learned automatically as a natural byproduct of learning category-specific dense pose predictors. To do this, we express correspondences between different categories and between images and categories using a unified embedding. Then, we use the latter to enforce two constraints: symmetric inter-category cycle consistency and a new asymmetric image-to-category cycle consistency. Without any manual annotations for the inter-category correspondences, we obtain state-of-the-art alignment results, outperforming dedicated methods for matching 3D shapes. Moreover, the new model is also better at the task of dense pose prediction than prior work.

Resources

Video presentation

Citation

@InProceedings{Neverova_Sanakoyeu21,
  title={Discovering Relationships between Object Categories via Universal Canonical Maps},
  author={Natalia Neverova and Artsiom Sanakoyeu and Patrick Labatut and David Novotny and Andrea Vedaldi},
  booktitle={CVPR},
  year={2021},
}

Our previous work

Transferring Dense Pose to Proximal Animal Classes, CVPR 2020