Lecture 8: Gradient Descent (and Beyond)

Solved Gradient Descent for a given X, Y. Recall from

Linear regression with gradient descent

Gradient-based optimization for Deep Learning: a short introduction

Conrad Kennington - Class Notes

LipschitzLR: Using theoretically computed adaptive learning rates for fast convergence

Gradient Descent and Back-tracking Line Search, by Andreas Maier, CodeX

Effortless optimization through gradient flows – Machine Learning Research Blog

Lecture 8: Gradient Descent (and Beyond)

Part 8-Machine learning solvers BEYOND Gradient Descent (SGD, Momentum, Adagrad, Adam)

Gradient Descent. Machine Learning is Applied Calculus, by Tyler Elliot Bettilyon, Teb's Lab

Lecture 25: Stochastic Gradient Descent, Matrix Methods in Data Analysis, Signal Processing, and Machine Learning, Mathematics

4. Beyond Gradient Descent - Fundamentals of Deep Learning [Book]