gpu-inference on Hacker News: 1 stories with screenshots

Screenshot of arxiv.org: Lossless LLM compression for efficient GPU inference via dynamic-length float

Lossless LLM compression for efficient GPU inference via dynamic-length float (arxiv.org)

392 points by CharlesW 1 year ago | 109 comments

arXiv paper introducing DFloat11, a lossless compression format that shrinks LLM weights by about 30% using entropy coding, with a custom GPU kernel for inference.

llm-compression dfloat11 bfloat16 entropy-coding

#gpu-inference on Hacker News

Lossless LLM compression for efficient GPU inference via dynamic-length float(arxiv.org)

Lossless LLM compression for efficient GPU inference via dynamic-length float (arxiv.org)