Is 'Attention' the Essence of Intelligence?(quantamagazine.org)
58 points by bcaulfield | 52 comments
Quanta Magazine article exploring whether the attention mechanism is central to intelligence in AI and brains.
#attention on Hacker News
22 stories tagged attention, newest first.
Transformers Explained Visually(poloclub.github.io)
576 points by aray07 | 85 comments
Transformer Explainer is an interactive in-browser visualization that runs GPT-2 and shows how embeddings, multi-head self-attention, and MLP layers process text.
Attention is all you have(alicegg.tech)
996 points by zer0tonin | 298 comments
A personal blog essay by Alice GG titled "Attention is all you have", opening with the Tetris effect to discuss how what we focus on shapes perception.
De-Brainrot Vacations(devz.cl)
437 points by DanielVZ | 176 comments
A personal blog post about spending a vacation trying to recover from "brainrot" caused by excessive screen and internet use.
It's getting harder to focus every day(glyphack.com)
675 points by peykar | 381 comments
A personal blog post by Glyphack reflecting on increasing difficulty focusing and getting only about an hour of focused time a day.
How LLMs work(0xkato.xyz)
834 points by 0xkato | 228 comments
A blog post giving a math-light walkthrough of how transformer-based large language models work internally.
Men who stare at walls(alexselimov.com)
662 points by aselimov3 | 305 comments
Personal blog post by Alex Selimov about staring at walls as a practice to improve focus and productivity.
The Illustrated Transformer(jalammar.github.io)
468 points by auraham | 85 comments
Jay Alammar's illustrated blog post explaining the Transformer neural network architecture and its self-attention mechanism with visual diagrams.
Attention lapses due to sleep deprivation due to flushing fluid from brain(news.mit.edu)
525 points by gmays | 254 comments
MIT News article on research showing attention lapses during sleep deprivation coincide with fluid flushing from the brain, a process normally occurring during sleep.
Smartphones and being present(herman.bearblog.dev)
389 points by articsputnik | 236 comments
Herman's blog post reflecting on average daily smartphone use statistics and his desire to avoid carrying the internet in his pocket to stay present.
Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it(github.com)
327 points by mmastrac | 143 comments
GitHub pull request in triton-lang/triton rewriting the Gluon attention tutorial kernel to be persistent, noting fp8 runs ~100 TFLOPS faster when the kernel name contains "cutlass".
Almost anything you give sustained attention to will begin to loop on itself(henrikkarlsson.xyz)
707 points by jger15 | 194 comments
An essay by Henrik Karlsson on Escaping Flatland arguing that sustained attention to almost anything makes it loop on itself and bloom into richness.
LLM architecture comparison(magazine.sebastianraschka.com)
357 points by mdp2021 | 23 comments
Sebastian Raschka's Ahead of AI article comparing the architectures of modern LLMs, from DeepSeek-V3 to Kimi K2.
Writing an LLM from scratch, part 13 – attention heads are dumb(gilesthomas.com)
322 points by gpjt | 61 comments
Blog post, part 13 of a series working through Raschka's 'Build a Large Language Model (from Scratch)', explaining why self-attention's matrix multiplications work.
The Death of Daydreaming(afterbabel.com)
587 points by isolli | 224 comments
An After Babel essay by Christine Rosen, introduced by Jon Haidt, on how smartphones eliminate boredom and interstitial time, and what is lost with daydreaming.
Write to Escape Your Default Setting(kupajo.com)
370 points by kolyder | 85 comments
A personal blog essay on kupajo arguing that writing gives structure to thinking and helps escape distraction and default mental patterns.
Calm tech certification "rewards" less distracting tech(spectrum.ieee.org)
358 points by headalgorithm | 114 comments
IEEE Spectrum news article on the Calm Tech certification for less distracting devices, with the first certified products shown at CES 2025.
The quiet art of attention(billwear.github.io)
717 points by billwear | 251 comments
A personal essay on cultivating attention, observing the mind without judgment, and simplifying life by focusing on essentials.
Understand how transformers work by demystifying the math behind them(osanseviero.github.io)
408 points by LaserPineapple | 94 comments
A hackerllama blog post walking through an end-to-end worked example of transformer math using small random vectors and matrices.
The Secret Sauce behind 100K context window in LLMs: all tricks in one place(blog.gopenai.com)
406 points by T-A | 92 comments
A Medium blog post explaining techniques for speeding up large language models and enabling context windows of 100K tokens.
Transformers from Scratch (2021)(e2eml.school)
594 points by jasim | 46 comments
A long-form tutorial explaining how transformer neural networks work from the ground up, covering matrix multiplication, attention, and backpropagation.
Treat your to-read pile like a river, not a bucket(oliverburkeman.com)
352 points by pps | 170 comments
Oliver Burkeman essay arguing readers should treat their backlog of books, articles and podcasts as a flowing river to dip into rather than a bucket to empty.