Building an LLM from Scratch: Automatic Differentiation (2023)(bclarkson-code.github.io)297 points by netwrt 2 years ago | 16 comments
Finetuning LLMs on a Single GPU Using Gradient Accumulation(lightning.ai)76 points by ashvardanian 3 years ago | 2 comments
Neural network training makes beautiful fractals(sohl-dickstein.github.io)264 points by telotortium 2 years ago | 54 comments
An overview of gradient descent optimization algorithms (2016)(ruder.io)110 points by skidrow 1 year ago | 18 comments