Lossless LLM compression for efficient GPU inference via dynamic-length float(arxiv.org)
392 points by CharlesW | 109 comments
arXiv paper introducing DFloat11, a lossless compression format that shrinks LLM weights by about 30% using entropy coding, with a custom GPU kernel for inference.