Eagle 7B: Soaring past Transformers(blog.rwkv.com)
341 points by guybedo | 73 comments
RWKV blog post announcing Eagle 7B, an RWKV-v5 linear transformer model trained on 1 trillion tokens across 100+ languages.
#rwkv on Hacker News
2 stories tagged rwkv, newest first.
RWKV: Reinventing RNNs for the Transformer Era(arxiv.org)
318 points by ianbutler | 164 comments
arXiv paper introducing RWKV, a model architecture combining Transformer-style parallel training with efficient RNN inference via linear attention.