Tokens too cheap to meter(jyn.dev)
326 points by teoruiz | 214 comments
Blog post on jyn.dev arguing LLM token prices are falling by orders of magnitude, so quality and access, not token cost, will limit AI use.
Screenshot pending…
Related stories
Measuring Claude 4.7's tokenizer costs(claudecodecamp.com)
678 points by aray07 | 474 comments
Blog post measuring Claude Opus 4.7's new tokenizer, finding about 1.47x more tokens than 4.6 on real content versus the documented 1.0–1.35x, and the resulting cost impact.
GigaToken: ~1000x faster Language model tokenization(github.com)
585 points by syrusakbary | 116 comments
GitHub repository for gigatoken, an open-source library for fast language model tokenization at GB/s speeds, MIT licensed.
OpenCoder: Open Cookbook for Top-Tier Code Large Language Models(opencoder-llm.github.io)
573 points by pil0u | 71 comments
arXiv paper introducing OpenCoder, an open code LLM released with its training data, data pipeline, and protocols as an open cookbook for research.
100K Context Windows(anthropic.com)
853 points by samwillis | 364 comments
Anthropic announcement from May 11, 2023 introducing 100K token context windows for its Claude language model.
Tokens are getting more expensive(ethanding.substack.com)
341 points by admp | 230 comments
Substack essay by Ethan Ding arguing that AI token usage costs are rising and cheaper models won't save consumer AI subscription margins, citing Claude Code and Windsurf.