How many GPUs is 1M/B/T tokens?(cedana.com)
14 points by kmavm | 4 comments
Cedana's interactive calculator that converts a token volume and model choice into an estimated number of GPUs (e.g., H100s) needed to serve it.
#h100 on Hacker News
3 stories tagged h100, newest first.
We bought the whole GPU, so we're damn well going to use the whole GPU(hazyresearch.stanford.edu)
472 points by sydriax | 94 comments
Stanford Hazy Research blog post introducing a throughput-optimized megakernel for tensor-parallel Llama-70B inference on H100 GPUs that overlaps compute, memory, and communication.
Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups(sfcompute.org)
672 points by flaque | 161 comments
Landing page for The San Francisco Compute Group, a shared GPU cluster that startups and labs co-buy and schedule for bursty large-model training runs.