26× Faster Inference with Layer-Condensed KV Cache for Large Language Models (arxiv.org) | HN Screenshots

YHacker Newsscreenshots

Screenshot of arxiv.org: 26× Faster Inference with Layer-Condensed KV Cache for Large Language Models

First screen, captured 2 years ago.

Looks similar

Screenshot of arxiv.org: Parameter-free KV cache compression for memory-efficient long-context LLMs

Parameter-free KV cache compression for memory-efficient long-context LLMs (arxiv.org)

60 points by PaulHoule 1 year ago | 13 comments

Screenshot of arxiv.org: HMT: Hierarchical Memory Transformer for Long Context Language Processing

HMT: Hierarchical Memory Transformer for Long Context Language Processing (arxiv.org)

83 points by jasondavies 2 years ago | 5 comments

Screenshot of arxiv.org: 16-Bit to 1-Bit: Visual KV Cache Quantization for Efficient Multimodal LLMs

16-Bit to 1-Bit: Visual KV Cache Quantization for Efficient Multimodal LLMs (arxiv.org)

83 points by PaulHoule 1 year ago | 1 comments

Screenshot of arxiv.org: δ-mem: Efficient Online Memory for Large Language Models

δ-mem: Efficient Online Memory for Large Language Models (arxiv.org)

227 points by 44za12 5 months ago | 58 comments

Screenshot of arxiv.org: Efficient Memory Management for Large Language Model Serving with PagedAttention

Efficient Memory Management for Large Language Model Serving with PagedAttention (arxiv.org)

87 points by jmorgan 3 years ago | 17 comments

Screenshot of arxiv.org: Recursively summarizing enables long-term dialogue memory in LLMs

Recursively summarizing enables long-term dialogue memory in LLMs (arxiv.org)

245 points by PaulHoule 3 years ago | 137 comments