Efficient streaming language models with attention sinks(github.com)
401 points by guywithabowtie | 65 comments
GitHub repository from MIT HAN Lab for StreamingLLM, code for efficient streaming language models using attention sinks.
#attention-sinks on Hacker News
1 story tagged attention-sinks, newest first.