Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs(baseten.co)

215 points by philipkiely 1 year ago | 142 comments

Show HN: I made a GPU VRAM calculator for transformer-based models(vram.asmirnov.xyz)

114 points by furiousteabag 2 years ago | 33 comments

Gemma3 – The current strongest model that fits on a single GPU(ollama.com)

212 points by brylie 1 year ago | 86 comments

Prompt caching for cheaper LLM tokens(ngrok.com)

268 points by samwho 10 months ago | 67 comments

GPUHammer: Rowhammer attacks on GPU memories are practical(gpuhammer.com)

260 points by jonbaer 1 year ago | 89 comments

SedonaDB 0.4: GPU-accelerated spatial joins(sedona.apache.org)

32 points by dr-jia-yu 4 months ago | 4 comments

Caveman: Why use many token when few token do trick(github.com)

809 points by tosh 6 months ago | 344 comments

GitHub repository for Caveman, a Claude Code skill/plugin that cuts output tokens about 75% by making the model respond in terse caveman-style speech.

100K Context Windows(anthropic.com)

853 points by samwillis 3 years ago | 364 comments

Anthropic announcement from May 11, 2023 introducing 100K token context windows for its Claude language model.

Measuring Claude 4.7's tokenizer costs(claudecodecamp.com)

678 points by aray07 6 months ago | 474 comments

Blog post measuring Claude Opus 4.7's new tokenizer, finding about 1.47x more tokens than 4.6 on real content versus the documented 1.0–1.35x, and the resulting cost impact.

How fast is N tokens per second really?(mikeveerman.github.io)

459 points by hexagr 5 months ago | 94 comments

Interactive web tool that simulates LLM output streaming at selectable tokens-per-second rates for code, text, thinking, and agent modes.

gpuvis: GPU Trace Visualizer(github.com)

67 points by luu 23 days ago | 10 comments

GitHub repository for gpuvis, an MIT-licensed GPU trace visualizer tool for viewing GPU and CPU event timelines.

Building Meta's GenAI infrastructure(engineering.fb.com)

636 points by mootpt 2 years ago | 291 comments

Meta engineering blog post describing its large GPU training clusters for generative AI, including RDMA and InfiniBand networking, Grand Teton hardware, and storage.