Transformers Are Inherently Succinct (2025) (arxiv.org)

Screenshot of arxiv.org: Transformers Are Inherently Succinct (2025)

First screen, captured 5 months ago.

Looks similar

Screenshot of arxiv.org: Do transformers need three projections? Systematic study of QKV variants

Do transformers need three projections? Systematic study of QKV variants (arxiv.org)

202 points by Anon84 4 months ago | 39 comments

Screenshot of arxiv.org: Transformers Are Bayesian Networks

Transformers Are Bayesian Networks (arxiv.org)

27 points by Anon84 7 months ago | 16 comments

Screenshot of arxiv.org: Transformers as Support Vector Machines

Transformers as Support Vector Machines (arxiv.org)

209 points by fofoz 3 years ago | 129 comments

Screenshot of arxiv.org: Language models are injective and hence invertible

Language models are injective and hence invertible (arxiv.org)

194 points by mazsa 1 year ago | 136 comments

Screenshot of arxiv.org: CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs (arxiv.org)

96 points by matt_d 5 months ago | 12 comments

Screenshot of arxiv.org: Differential Transformer

Differential Transformer (arxiv.org)

525 points by weirdcat 2 years ago | 166 comments

arXiv paper introducing Diff Transformer, which computes attention as the difference of two softmax maps to cancel noise and improve long-context modeling and reduce hallucination.

differential-transformer attention-mechanism transformers large-language-models

Looks similar

Do transformers need three projections? Systematic study of QKV variants(arxiv.org)

Transformers Are Bayesian Networks(arxiv.org)

Transformers as Support Vector Machines(arxiv.org)

Language models are injective and hence invertible(arxiv.org)

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs(arxiv.org)

Differential Transformer(arxiv.org)