Efficient Memory Management for Large Language Model Serving with PagedAttention(arxiv.org)87 points by jmorgan 3 years ago | 17 comments
Recursively summarizing enables long-term dialogue memory in LLMs(arxiv.org)245 points by PaulHoule 3 years ago | 137 comments
26× Faster Inference with Layer-Condensed KV Cache for Large Language Models(arxiv.org)112 points by georgehill 2 years ago | 16 comments
Efficient LLM inference solution on Intel GPU(arxiv.org)87 points by PaulHoule 2 years ago | 20 comments