Serving 70B-scale LLMs efficiently on low-resource edge devices [pdf] (arxiv.org)

Screenshot of arxiv.org: Serving 70B-scale LLMs efficiently on low-resource edge devices [pdf]

First screen, captured 2 years ago.

Looks similar

Screenshot of arxiv.org: SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs

SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs (arxiv.org)

25 points by PaulHoule 1 year ago | 5 comments

Screenshot of arxiv.org: Were RNNs all we needed?

Were RNNs all we needed?(arxiv.org)

466 points by beefman 2 years ago | 245 comments

arXiv paper proposing minimal LSTM and GRU variants without hidden-state gate dependencies that can be trained in parallel, rivaling recent sequence models.

rnn lstm gru transformers

Screenshot of arxiv.org: MemGPT: Towards LLMs as Operating Systems

MemGPT: Towards LLMs as Operating Systems (arxiv.org)

203 points by belter 3 years ago | 105 comments

Screenshot of arxiv.org: Lossless LLM compression for efficient GPU inference via dynamic-length float

Lossless LLM compression for efficient GPU inference via dynamic-length float (arxiv.org)

392 points by CharlesW 1 year ago | 109 comments

arXiv paper introducing DFloat11, a lossless compression format that shrinks LLM weights by about 30% using entropy coding, with a custom GPU kernel for inference.

llm-compression dfloat11 bfloat16 entropy-coding

Screenshot of arxiv.org: Parameter-free KV cache compression for memory-efficient long-context LLMs

Parameter-free KV cache compression for memory-efficient long-context LLMs (arxiv.org)

60 points by PaulHoule 1 year ago | 13 comments

Screenshot of arxiv.org: Efficient LLM inference solution on Intel GPU

Efficient LLM inference solution on Intel GPU (arxiv.org)

87 points by PaulHoule 2 years ago | 20 comments

Looks similar

SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs(arxiv.org)

Were RNNs all we needed?(arxiv.org)

MemGPT: Towards LLMs as Operating Systems(arxiv.org)

Lossless LLM compression for efficient GPU inference via dynamic-length float(arxiv.org)

Parameter-free KV cache compression for memory-efficient long-context LLMs(arxiv.org)

Efficient LLM inference solution on Intel GPU(arxiv.org)

SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs (arxiv.org)

MemGPT: Towards LLMs as Operating Systems (arxiv.org)

Lossless LLM compression for efficient GPU inference via dynamic-length float (arxiv.org)

Parameter-free KV cache compression for memory-efficient long-context LLMs (arxiv.org)

Efficient LLM inference solution on Intel GPU (arxiv.org)