First screen, captured .
Looks similar
Beyond self-attention: How a small language model predicts the next token(shyam.blog)
411 points by tplrbv | 72 comments
A long blog post analyzing how a small ~10M parameter transformer trained from Karpathy's tutorial turns attention outputs into next-token predictions.
Related stories
Language models can explain neurons in language models(openai.com)
659 points by mfiguiere | 416 comments
OpenAI research post describing using GPT-4 to automatically generate and score explanations of individual neuron behavior in GPT-2.
How Meta trains large language models at scale(engineering.fb.com)
362 points by mfiguiere | 183 comments
Engineering at Meta blog post describing the software, hardware, and network considerations behind training large language models at scale in production.
LLMs aren't world models(yosefk.com)
330 points by ingve | 189 comments
Blog post by Yossi Kreinin arguing that large language models are not world models, illustrated with examples like LLMs losing track of chess board state.
Training LLMs from ground zero as a startup(yitay.net)
360 points by swyx | 92 comments
Yi Tay's blog post on the experience of training large language models from scratch at a startup (Reka), covering compute, infrastructure, and lessons learned.
Language models for text classification: From bag-of-words to Jev(magazine.sebastianraschka.com)
181 points by Anon84 | 9 comments
Sebastian Raschka's Ahead of AI newsletter post giving a visual guide to text classification, from bag-of-words through RNNs, CNNs, and transformers to Jev-like APIs and calibration.
Something weird is happening with LLMs and chess(dynomight.substack.com)
626 points by crescit_eundo | 424 comments
A Dynomight newsletter post examining how well large language models play chess and why results vary strangely across models.