First screen, captured 6 days ago.

DeepInfer: High-performance inference framework for DeepSeek models

What HN thinks

A framework release becomes a referendum on whether AI makes money, with detours through Chinese electricity bills and Uber.

Inference is already profitable
DeepSeek's $38 per card-hour revenue and profit estimators suggest serving models makes healthy margins, especially on owned, well-utilized hardware.
Margins will be competed away
Inference is an undifferentiated, price-sensitive commodity, so margins will fall toward utility levels, and training costs can't be separated from serving costs.
Open models are good enough
Many find cheap open models like DeepSeek Flash or GLM cover most needs, though some say frontier models like Claude Sonnet save tokens and time overall.

Consensus: Most agree inference itself is profitable today, but the business as a whole, once training and research are counted, is not clearly profitable, and open models are pushing toward commoditization.

Recommended in the comments

InferenceX Profit Estimator ↗ (service): Cited as hard numbers showing inference margins.
OpenRouter ↗ (service): Shows many providers undercutting each other on open models, evidence of commoditization.
Cloudflare Workers AI ↗ (service): Used at work to explore cheap open models with acceptable contractual and security terms.
GLM (product): One commenter says it covers nearly all their needs without frontier models.

Summarised by Claude from the comment thread 3 days ago.

Points: 2273
Comments: 890
Posted: 2026-10-20 by ggcr
Category: Ai ml

deepseek llm-inference inference-framework open-source github

All captures:

Looks similar

Screenshot of github.com: DualPipe: Bidirectional pipeline parallelism algorithm

DualPipe: Bidirectional pipeline parallelism algorithm (github.com)

165 points by mfiguiere 1 year ago | 19 comments

Screenshot of github.com: DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling (github.com)

381 points by mfiguiere 1 year ago | 65 comments

GitHub repository for DeepSeek's DeepGEMM, a library of clean and efficient FP8 GEMM kernels with fine-grained scaling, released under the MIT license.

deepseek fp8 gemm cuda

Screenshot of github.com: DeepSeek open source DeepEP – library for MoE training and Inference

DeepSeek open source DeepEP – library for MoE training and Inference (github.com)

473 points by helloericsf 1 year ago | 66 comments

GitHub repository for DeepEP, DeepSeek's open-source expert-parallel communication library for Mixture-of-Experts model training and inference.

deepseek mixture-of-experts expert-parallelism gpu-communication

Screenshot of github.com: Fire-Flyer File System from DeepSeek

Fire-Flyer File System from DeepSeek (github.com)

319 points by wenyuanyu 1 year ago | 80 comments

GitHub repository for DeepSeek's 3FS (Fire-Flyer File System), a high-performance distributed file system for AI training and inference workloads.

deepseek 3fs distributed-file-system storage

DeepSeek-V3 (github.com)

21 points by mgt19937 1 year ago | 5 comments

Screenshot of github.com: Janus: Decoupling Visual Encoding for Multimodal Understanding and Generation

Janus: Decoupling Visual Encoding for Multimodal Understanding and Generation (github.com)

21 points by jinqueeny 2 years ago | 3 comments

Looks similar

DualPipe: Bidirectional pipeline parallelism algorithm (github.com)

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling (github.com)

DeepSeek open source DeepEP – library for MoE training and Inference (github.com)

Fire-Flyer File System from DeepSeek (github.com)

DeepSeek-V3 (github.com)

Janus: Decoupling Visual Encoding for Multimodal Understanding and Generation (github.com)

Related stories

DeepSeek open source DeepEP – library for MoE training and Inference (github.com)

DeepSeek-Prover-V2 (github.com)

The path to open-sourcing the DeepSeek inference engine (github.com)

BitNet: Inference framework for 1-bit LLMs (github.com)

DeepSeek-R1 (github.com)

DeepSeek Open Source FlashMLA – MLA Decoding Kernel for Hopper GPUs (github.com)