New attention mechanisms that outperform standard multi-head attention (arxiv.org) | HN Screenshots

YHacker Newsscreenshots

Screenshot of arxiv.org: New attention mechanisms that outperform standard multi-head attention

First screen, captured 2 years ago.

Looks similar

Screenshot of arxiv.org: TransMLA: Multi-head latent attention is all you need

TransMLA: Multi-head latent attention is all you need (arxiv.org)

113 points by ocean_moist 1 year ago | 34 comments

Screenshot of arxiv.org: Log-Linear Attention

Log-Linear Attention (arxiv.org)

19 points by sva_ 1 year ago | 3 comments

Screenshot of arxiv.org: Multi-Token Attention

Multi-Token Attention (arxiv.org)

131 points by fzliu 1 year ago | 35 comments

Screenshot of arxiv.org: HyperAttention: Long-Context Attention in Near-Linear Time

HyperAttention: Long-Context Attention in Near-Linear Time (arxiv.org)

61 points by kelseyfrog 3 years ago | 10 comments

Screenshot of aclanthology.org: Native Sparse Attention

Native Sparse Attention (aclanthology.org)

127 points by CalmStorm 1 year ago | 25 comments

Screenshot of arxiv.org: Tensor Product Attention Is All You Need

Tensor Product Attention Is All You Need (arxiv.org)

136 points by eunos 1 year ago | 93 comments