Life of an inference request (vLLM V1): How LLMs are served efficiently at scale (ubicloud.com) | HN Screenshots

YHacker Newsscreenshots

Screenshot of ubicloud.com: Life of an inference request (vLLM V1): How LLMs are served efficiently at scale

First screen, captured 1 year ago.

Looks similar

Screenshot of ubicloud.com: How we enabled ARM64 VMs

How we enabled ARM64 VMs (ubicloud.com)

60 points by ozgune 2 years ago | 10 comments

Screenshot of minimaxir.com: The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin

The mysterious Hy3 LLM is topping OpenRouter Model Rankings by a large margin (minimaxir.com)

131 points by freediver 5 months ago | 103 comments

Screenshot of alessiofanelli.com: In the LLM space, "open source" is being used to mean "downloadable weights"

In the LLM space, "open source" is being used to mean "downloadable weights"(alessiofanelli.com)

353 points by FanaHOVA 3 years ago | 210 comments

Blog post by Alessio Fanelli arguing that LLaMA2 isn't truly open source under the OSS definition, and why that distinction may not matter much.

llama-2 open-source-licensing llm meta

Screenshot of blog.cloudflare.com: When “idle” isn't idle: how a Linux kernel optimization became a QUIC bug

When “idle” isn't idle: how a Linux kernel optimization became a QUIC bug (blog.cloudflare.com)

145 points by sbulaev 5 months ago | 26 comments

Screenshot of artfintel.com: Efficient LLM Inference (2023)

Efficient LLM Inference (2023)(artfintel.com)

87 points by inaciom 2 years ago | 9 comments

Screenshot of ubicloud.com: Rethinking PostgreSQL Storage

Rethinking PostgreSQL Storage (ubicloud.com)

22 points by furkansahin 1 year ago | 8 comments