Mixture-of-Depths: Dynamically allocating compute in transformers (arxiv.org) | HN Screenshots

YHacker Newsscreenshots

Screenshot of arxiv.org: Mixture-of-Depths: Dynamically allocating compute in transformers

First screen, captured 2 years ago.

Looks similar

Screenshot of arxiv.org: DenseFormer: Enhancing Information Flow in Transformers

DenseFormer: Enhancing Information Flow in Transformers (arxiv.org)

109 points by tipsytoad 2 years ago | 28 comments

Screenshot of arxiv.org: Sorbet: A neuromorphic hardware-compatible transformer-based spiking model

Sorbet: A neuromorphic hardware-compatible transformer-based spiking model (arxiv.org)

47 points by PaulHoule 2 years ago | 24 comments

Screenshot of arxiv.org: Jamba: A Hybrid Transformer-Mamba Language Model

Jamba: A Hybrid Transformer-Mamba Language Model (arxiv.org)

58 points by eitanturok 2 years ago | 6 comments

Screenshot of arxiv.org: Mixtral 8x7B: A sparse Mixture of Experts language model

Mixtral 8x7B: A sparse Mixture of Experts language model (arxiv.org)

293 points by ignoramous 2 years ago | 110 comments

Screenshot of arxiv.org: LLM.int8(): 8-Bit Matrix Multiplication for Transformers at Scale (2022)

LLM.int8(): 8-Bit Matrix Multiplication for Transformers at Scale (2022)(arxiv.org)

125 points by tosh 3 years ago | 15 comments

Screenshot of arxiv.org: Scalable MatMul-Free Language Modeling

Scalable MatMul-Free Language Modeling (arxiv.org)

177 points by lykahb 2 years ago | 27 comments