Transformer Explainer(poloclub.github.io)
82 points by mdp2021 | 5 comments
First screen, captured .
Looks similar
Dust: Pretraining Transformers Without Backpropagation(qlabs.sh)
269 points by E-Reverance | 79 comments
Research page from QLabs presenting Dust, a method for pretraining transformer models without backpropagation, with authors, code link, and architecture diagram.
Related stories
What are transformer models and how do they work?(txt.cohere.ai)
380 points by tomcam | 91 comments
Cohere blog post explaining what transformer models are and how they work in natural language processing.
Understand how transformers work by demystifying the math behind them(osanseviero.github.io)
408 points by LaserPineapple | 94 comments
A hackerllama blog post walking through an end-to-end worked example of transformer math using small random vectors and matrices.
Manim: Animation engine for explanatory math videos(github.com)
388 points by pykello | 65 comments
GitHub repository for Manim, 3Blue1Brown's Python animation engine for creating explanatory math videos.
Transformers from Scratch (2021)(e2eml.school)
594 points by jasim | 46 comments
A long-form tutorial explaining how transformer neural networks work from the ground up, covering matrix multiplication, attention, and backpropagation.
Animated AI(animatedai.github.io)
642 points by qwertyforce | 55 comments
Animated AI is a site of 3D animations and instructional videos explaining neural network concepts, starting with the basic convolution algorithm.
You could have invented transformers from memory(alexwang.ai)
602 points by speckx | 133 comments
Blog post by Alex Wang explaining transformer layers (MLPs, gated MLPs, self-attention) as associative memory stores, with code and an interactive app.