Real-time LLM Inference on Standard GPUs: 3k tokens/s per request (blog.kog.ai) | HN Screenshots

YHacker Newsscreenshots

Screenshot of blog.kog.ai: Real-time LLM Inference on Standard GPUs: 3k tokens/s per request

First screen, captured 5 months ago.

Looks similar

Screenshot of baseten.co: Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs

Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs (baseten.co)

215 points by philipkiely 1 year ago | 142 comments

Screenshot of blog.mlc.ai: Making AMD GPUs competitive for LLM inference (2023)

Making AMD GPUs competitive for LLM inference (2023)(blog.mlc.ai)

297 points by plasticchris 1 year ago | 199 comments

Screenshot of blog.mlc.ai: Making AMD GPUs competitive for LLM inference

Making AMD GPUs competitive for LLM inference (blog.mlc.ai)

302 points by djoldman 3 years ago | 106 comments

Screenshot of telnyx.com: Kimi K3 Now Available via Telnyx Inference API

Kimi K3 Now Available via Telnyx Inference API (telnyx.com)

123 points by fionaattelnyx 3 months ago | 72 comments

Screenshot of artfintel.com: Efficient LLM Inference (2023)

Efficient LLM Inference (2023)(artfintel.com)

87 points by inaciom 2 years ago | 9 comments

Screenshot of blogs.nvidia.com: H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark

H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark (blogs.nvidia.com)

137 points by Anon84 3 years ago | 62 comments