Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs(baseten.co)215 points by philipkiely 1 year ago | 142 comments
Continuous batching to increase LLM inference throughput and reduce p50 latency(anyscale.com)98 points by michellezzz 3 years ago | 19 comments
How much do amd64 microarchitecture levels help in Go?(lemire.me)61 points by zdw 4 months ago | 39 comments
Intelligent Kubernetes Load Balancing at Databricks(databricks.com)99 points by ayf 1 year ago | 20 comments
How fast can browsers process base64 data?(lemire.me)35 points by mfiguiere 11 months ago | 20 comments