How attention sinks keep language models stable (hanlab.mit.edu)

Screenshot of hanlab.mit.edu: How attention sinks keep language models stable

First screen, captured 1 year ago.

Looks similar

Screenshot of arxiv.org: StreamingLLM: Efficient streaming technique enable infinite sequence lengths

StreamingLLM: Efficient streaming technique enable infinite sequence lengths (arxiv.org)

97 points by TheJCDenton 3 years ago | 11 comments

Screenshot of hanlab.mit.edu: SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup

SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup (hanlab.mit.edu)

167 points by lmxyy 1 year ago | 56 comments

Screenshot of hao-ai-lab.github.io: Consistency LLM: converting LLMs to parallel decoders accelerates inference 3.5x

Consistency LLM: converting LLMs to parallel decoders accelerates inference 3.5x (hao-ai-lab.github.io)

447 points by zhisbug 2 years ago | 94 comments

Hao AI Lab blog post introducing Consistency LLMs, a method that fine-tunes LLMs into parallel Jacobi decoders to speed up inference about 3.5x.

consistency-llm llm-inference parallel-decoding jacobi-decoding

Screenshot of shyam.blog: Beyond self-attention: How a small language model predicts the next token

Beyond self-attention: How a small language model predicts the next token (shyam.blog)

411 points by tplrbv 2 years ago | 72 comments

A long blog post analyzing how a small ~10M parameter transformer trained from Karpathy's tutorial turns attention outputs into next-token predictions.

transformers interpretability language-models self-attention

Screenshot of openai.com: Why language models hallucinate

Why language models hallucinate (openai.com)

224 points by simianwords 1 year ago | 188 comments

Screenshot of arxiv.org: Recursively summarizing enables long-term dialogue memory in LLMs

Recursively summarizing enables long-term dialogue memory in LLMs (arxiv.org)

245 points by PaulHoule 3 years ago | 137 comments

Looks similar

StreamingLLM: Efficient streaming technique enable infinite sequence lengths(arxiv.org)

SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup(hanlab.mit.edu)

Consistency LLM: converting LLMs to parallel decoders accelerates inference 3.5x(hao-ai-lab.github.io)

Beyond self-attention: How a small language model predicts the next token(shyam.blog)

Why language models hallucinate(openai.com)

Recursively summarizing enables long-term dialogue memory in LLMs(arxiv.org)