I trained a small transformer in 1.5hrs and it beats many LLMs(mvakde.github.io)
578 points by porridgeraisin | 149 comments
Personal blog post describing a small transformer trained from scratch in 1.5 hours on an RTX 5090 that scores 44% on ARC-AGI-1 for 67 cents.


