nvme-streaming on Hacker News: 1 stories with screenshots

Screenshot of github.com: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

Run Kimi K3 using 29 GB of RAM at 0.50 tok/s (github.com)

318 points by marcobambini 2 months ago | 152 comments

GitHub repository for waste, a dependency-free embeddable C inference engine that runs the 2.78-trillion-parameter Kimi K3 model by streaming weights from NVMe.

kimi-k3 llm-inference c nvme-streaming

#nvme-streaming on Hacker News

Run Kimi K3 using 29 GB of RAM at 0.50 tok/s(github.com)

Run Kimi K3 using 29 GB of RAM at 0.50 tok/s (github.com)