First screen, captured .
Looks similar
Writing an LLM from scratch, part 8 – trainable self-attention(gilesthomas.com)
350 points by gpjt | 26 comments
Blog post, part 8 of a series working through Raschka's 'Build a Large Language Model (from Scratch)', explaining self-attention with trainable weights.
LLM from scratch, part 28 – training a base model from scratch on an RTX 3090(gilesthomas.com)
482 points by gpjt | 101 comments
Blog post, part 28 of a series, describing an experiment training a ~163M-parameter base LLM from scratch on a single RTX 3090 at home.
