Fixing Tool-Call Accuracy in vLLM(ubicloud.com)
20 points by coffeemug | 1 comments
Ubicloud blog post explaining tool-call schema accuracy problems when serving open-weight models like Kimi K2.6 in vLLM and how constrained decoding fixes them.
#vllm on Hacker News
4 stories tagged vllm, newest first.
PyTorch-native TPU back end for vLLM(github.com)
4 points by zniturah | 0 comments
GitHub repository for vllm-torchtpu, a PyTorch-native TPU backend for the vLLM LLM inference engine.
Turning GLM-5.3-Flash into a Jev-like decision model(privatemode.ai)
116 points by flxflx | 50 comments
Privatemode AI blog post describing how to prompt GLM-5.3-Flash so its first output token gives a decision in a single forward pass, benchmarked against Jev and Laya.
The path to open-sourcing the DeepSeek inference engine(github.com)
531 points by Palmik | 58 comments
GitHub directory in DeepSeek's open-infra-index repo with a README describing their plan for open-sourcing the DeepSeek inference engine.