The Lottery Ticket Hypothesis, Simply Explained

Deep neural networks are usually significantly over-parameterized. Modern models contain millions or billions of parameters, yet it has been routinely observed that large portions of them can be removed after training with little loss in performance.

But the natural follow-up question is deeper: Are these extra parameters just redundant, or were they necessary to discover the final solution?

In their landmark paper, “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks,” Jonathan Frankle and Michael Carbin propose a bold answer.