Video Summary

The paper studies the problem of training neural networks with limited data. The authors propose a method called data distillation, which uses a large teacher network to train a smaller student network. The teacher network is first trained on a large dataset, and then the student network is trained to mimic the predictions of the teacher network on a small dataset. The authors show that data distillation can improve the performance of the student network on the small dataset, even when the student network has much fewer parameters than the teacher network.