15 stories tagged nlp, newest first.
194 points by canjobear | 167 comments
arXiv paper analyzing psycholinguistic features that make AI-generated text harder to understand, attributing it to optimization for information density.
57 points by jochenmadler | 14 comments
arXiv paper "SlopShape" presenting a classifier that detects AI-generated commercial blog posts and attributes their source model using structural features rather than wording.
585 points by syrusakbary | 116 comments
GitHub repository for gigatoken, an open-source library for fast language model tokenization at GB/s speeds, MIT licensed.
529 points by kristianpaul | 49 comments
Course homepage for Stanford's CS336: Language Modeling from Scratch (Spring 2026), listing instructors Tatsunori Hashimoto and Percy Liang and course staff.
468 points by auraham | 85 comments
Jay Alammar's illustrated blog post explaining the Transformer neural network architecture and its self-attention mechanism with visual diagrams.
427 points by chilipepperhott | 154 comments
A blog post by Elijah Potter explaining Markov chains as early language models and how they generate text.
402 points by eric-burel | 78 comments
A Hugging Face Space hosting a visual, intuitive guide explaining how embeddings work in large language models.
372 points by brig90 | 123 comments
GitHub repository applying SBERT embeddings, KMeans clustering, and Markov transition analysis to the Voynich Manuscript to test for linguistic structure.
330 points by simonpure | 206 comments
arXiv paper showing top LLMs perform about 39% worse in multi-turn, underspecified conversations than in single-turn, fully specified settings.
441 points by jxmorris12 | 133 comments
An essay arguing that text embeddings are an underrated machine learning tool, especially for technical writing and documentation work.
366 points by zxexz | 83 comments
Meta AI research publication page for the Byte Latent Transformer paper, which proposes byte-level patches instead of tokens for scaling language models.
1620 points by tomthe | 483 comments
SUNSET.md in the wordfreq GitHub repository explaining why the word-frequency dataset, a snapshot through 2021, will no longer be updated.
344 points by felipemesquita | 45 comments
GitHub repository for transformers-ruby by ankane, a Ruby port of state-of-the-art transformer models, licensed under Apache-2.0.
594 points by jasim | 46 comments
A long-form tutorial explaining how transformer neural networks work from the ground up, covering matrix multiplication, attention, and backpropagation.
380 points by tomcam | 91 comments
Cohere blog post explaining what transformer models are and how they work in natural language processing.