Scaling Transformers to 1B Tokens (arxiv.org) | HN Screenshots

YHacker Newsscreenshots

Screenshot of arxiv.org: Scaling Transformers to 1B Tokens

First screen, captured 3 years ago.

Looks similar

Screenshot of arxiv.org: Scaling Transformer to 1M tokens and beyond with RMT

Scaling Transformer to 1M tokens and beyond with RMT (arxiv.org)

232 points by panabee 3 years ago | 115 comments

Screenshot of arxiv.org: Unlimiformer: Long-Range Transformers with Unlimited Length Input

Unlimiformer: Long-Range Transformers with Unlimited Length Input (arxiv.org)

315 points by shishy 3 years ago | 97 comments

Screenshot of arxiv.org: LongRoPE: Extending LLM Context Window Beyond 2M Tokens

LongRoPE: Extending LLM Context Window Beyond 2M Tokens (arxiv.org)

127 points by nojito 2 years ago | 38 comments

Screenshot of arxiv.org: HMT: Hierarchical Memory Transformer for Long Context Language Processing

HMT: Hierarchical Memory Transformer for Long Context Language Processing (arxiv.org)

83 points by jasondavies 2 years ago | 5 comments

Screenshot of arxiv.org: Scaling Data-Constrained Language Models

Scaling Data-Constrained Language Models (arxiv.org)

44 points by ijk 3 years ago | 5 comments

Screenshot of arxiv.org: TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters

TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters (arxiv.org)

170 points by og_kalu 2 years ago | 28 comments