An introduction to data distillation

Distillation ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. This is a kind of omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning offers the potential to surpass state-of-the-art fully supervised methods.

Introduction
Data Distillation
Data Distillation for Keypoint Detection
Experiments on Keypoint Detection
Thesis Work

References

[Rad17D]

Data Distillation: Towards Omni-Supervised Learning, Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, Kaiming He.

Dec 2017

We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lower-bounded by performance on existing labeled datasets, offering the potential to surpass state-of-the-art fully supervised methods. To exploit the omni-supervised setting, we …

Publication

References

In this series →