An introduction to data distillation

Distillation ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. This is a kind of omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning offers the potential to surpass state-of-the-art fully supervised methods.

  • Introduction
  • Data Distillation
  • Data Distillation for Keypoint Detection
  • Experiments on Keypoint Detection
  • Thesis Work


In this series