There is a connection between gradient descent based optimizers and the dynamics of damped harmonic oscillators. What does that mean? We now have a better theory for optimization algorithms.

All the formulas I mention in the episode can be found in the post The physics of optimization algorithms

Enjoy the show.