Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs(baseten.co)

215 points by philipkiely 1 year ago | 142 comments

Rack makes Pion SCTP 71% faster with 27% less latency(pion.ly)

10 points by mosura 10 months ago | 0 comments

Show HN: Pion SCTP with RACK is 70% faster with 30% less latency(pion.ly)

51 points by pch07 10 months ago | 5 comments

Two RTX 5090 hosts sharing a KV cache over Ethernet(viiwork.pulu.app)

4 points by velmu 2 days ago | 1 comments

A viiwork blog post benchmarking KV-cache sharing between two RTX 5090 LLM hosts over 1 GbE versus 10 GbE links for coding-agent workloads.

Nvidia Nemotron 3.5 Lightning and NeMo Switchyard(blogs.nvidia.com)

246 points by droidjj 2 months ago | 123 comments

H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark(blogs.nvidia.com)

137 points by Anon84 3 years ago | 62 comments

GPUs Go Brrr(hazyresearch.stanford.edu)

1056 points by nmstoker 2 years ago | 243 comments

Stanford Hazy Research blog post on what GPU hardware wants for efficient AI kernels, introducing the ThunderKittens CUDA kernel library.

What every developer should know about GPU computing(codeconfessions.substack.com)

422 points by Anon84 3 years ago | 149 comments

A Substack primer by Abhinav Upadhyay explaining GPU architecture and computing concepts for software developers familiar mainly with CPUs.

Fly.io has GPUs now(fly.io)

552 points by andes314 2 years ago | 196 comments

Fly.io blog post announcing that GPU machines are now available on its cloud platform.

Nvidia DGX GH200: 100 Terabyte GPU Memory System(developer.nvidia.com)

492 points by MacsHeadroom 3 years ago | 328 comments

NVIDIA Technical Blog post announcing the DGX GH200, a Grace Hopper-based system offering 100 terabytes of shared GPU memory for large AI workloads.

We were wrong about GPUs(fly.io)

834 points by mxstbr 1 year ago | 529 comments

Fly.io blog post reflecting on how the company's bet on offering GPU machines to developers did not pan out as expected.

We bought the whole GPU, so we're damn well going to use the whole GPU(hazyresearch.stanford.edu)

472 points by sydriax 1 year ago | 94 comments

Stanford Hazy Research blog post introducing a throughput-optimized megakernel for tensor-parallel Llama-70B inference on H100 GPUs that overlaps compute, memory, and communication.