Continuous batching to increase LLM inference throughput and reduce p50 latency (anyscale.com) | HN Screenshots

YHacker Newsscreenshots

Screenshot of anyscale.com: Continuous batching to increase LLM inference throughput and reduce p50 latency

First screen, captured 3 years ago.

Looks similar

Screenshot of anyscale.com: Numbers every LLM Developer should know

Numbers every LLM Developer should know (anyscale.com)

70 points by davidwu 3 years ago | 13 comments

Screenshot of anyscale.com: ThirdAI Uses Ray for Parallel Training of Billion-Parameter NN on Commodity CPUs

ThirdAI Uses Ray for Parallel Training of Billion-Parameter NN on Commodity CPUs (anyscale.com)

52 points by thirdailab 3 years ago | 11 comments

Screenshot of anyscale.com: Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper (anyscale.com)

71 points by pallas_athena 3 years ago | 19 comments

Screenshot of lemire.me: How much do amd64 microarchitecture levels help in Go?

How much do amd64 microarchitecture levels help in Go?(lemire.me)

61 points by zdw 4 months ago | 39 comments

Screenshot of blogs.remobjects.com: AVX2 is slower than SSE2-4.x under Windows ARM emulation

AVX2 is slower than SSE2-4.x under Windows ARM emulation (blogs.remobjects.com)

97 points by vintagedave 8 months ago | 78 comments

Screenshot of chrisebert.net: Comparing AWS Lambda ARM64 vs. x86_64 Performance Across Runtimes in Late 2025

Comparing AWS Lambda ARM64 vs. x86_64 Performance Across Runtimes in Late 2025 (chrisebert.net)

111 points by hasanhaja 10 months ago | 51 comments