Lossless LLM compression for efficient GPU inference via dynamic-length float (arxiv.org) | HN Screenshots

YHacker Newsscreenshots

Screenshot of arxiv.org: Lossless LLM compression for efficient GPU inference via dynamic-length float

First screen, captured 1 year ago.

Looks similar

Screenshot of arxiv.org: SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs

SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs (arxiv.org)

25 points by PaulHoule 1 year ago | 5 comments

Screenshot of arxiv.org: Parameter-free KV cache compression for memory-efficient long-context LLMs

Parameter-free KV cache compression for memory-efficient long-context LLMs (arxiv.org)

60 points by PaulHoule 1 year ago | 13 comments

Screenshot of arxiv.org: LoMA: Lossless Compressed Memory Attention

LoMA: Lossless Compressed Memory Attention (arxiv.org)

90 points by PaulHoule 2 years ago | 8 comments

Screenshot of arxiv.org: QLoRA: Efficient Finetuning of Quantized LLMs

QLoRA: Efficient Finetuning of Quantized LLMs (arxiv.org)

270 points by Garcia98 3 years ago | 90 comments

Screenshot of arxiv.org: Efficient LLM inference solution on Intel GPU

Efficient LLM inference solution on Intel GPU (arxiv.org)

87 points by PaulHoule 2 years ago | 20 comments

Screenshot of arxiv.org: LM2: Large Memory Models

LM2: Large Memory Models (arxiv.org)

105 points by fzliu 1 year ago | 30 comments

Related stories

Screenshot of arxiv.org: Deep Learning Is Not So Mysterious or Different

Deep Learning Is Not So Mysterious or Different (arxiv.org)

445 points by wuubuu 1 year ago | 112 comments

arXiv paper by Andrew Gordon Wilson arguing deep learning generalization phenomena like benign overfitting and double descent are explainable by soft inductive biases.

deep-learning generalization double-descent pac-bayes

Screenshot of github.com: Lossless video compression using Bloom filters

Lossless video compression using Bloom filters (github.com)

327 points by rh3939 1 year ago | 112 comments

GitHub README for a project implementing lossless video compression using rational Bloom filters, with source code and explanation.

bloom-filters video-compression lossless-compression data-structures

Screenshot of research.google: TurboQuant: Redefining AI efficiency with extreme compression

TurboQuant: Redefining AI efficiency with extreme compression (research.google)

495 points by ray__ 7 months ago | 134 comments

Google Research blog post introducing TurboQuant, a method for extreme compression to improve AI model efficiency.

turboquant quantization model-compression google-research

Screenshot of github.com: Nvidia Warp: A Python framework for high performance GPU simulation and graphics

Nvidia Warp: A Python framework for high performance GPU simulation and graphics (github.com)

453 points by jarmitage 2 years ago | 125 comments

GitHub repository for NVIDIA Warp, a Python framework for writing high-performance GPU simulation and graphics code.

nvidia-warp python gpu cuda

Screenshot of data.ornn.com: The Economics of Open-Weight Inference

The Economics of Open-Weight Inference (data.ornn.com)

36 points by marinesebastian 1 month ago | 25 comments

Ornn Data research paper arguing that open-weight model demand extends the economic useful life of older NVIDIA GPU families, comparing inference costs to closed models.

open-weight-models llm-inference nvidia gpu-depreciation

Screenshot of tomshardware.com: FFmpeg devs boast of another 100x leap thanks to handwritten assembly code

FFmpeg devs boast of another 100x leap thanks to handwritten assembly code (tomshardware.com)

341 points by harambae 1 year ago | 96 comments

Tom's Hardware article reporting FFmpeg developers achieving a roughly 100x speedup in a function via handwritten assembly code.

ffmpeg assembly simd performance-optimization