Training LLMs from ground zero as a startup(yitay.net)

360 points by swyx 2 years ago | 92 comments

Yi Tay's blog post on the experience of training large language models from scratch at a startup (Reka), covering compute, infrastructure, and lessons learned.

Talking to LLMs has improved my thinking(philipotoole.com)

117 points by otoolep 9 months ago | 95 comments

Autoregressive Language Model on the 6502 Processor(mattbeton.com)

122 points by nmstoker 2 months ago | 12 comments

Summing ASCII encoded integers on Haswell at almost the speed of memcpy(blog.mattstuchlik.com)

114 points by iliekcomputers 2 years ago | 34 comments

Tokenization for language modeling: BPE vs. Unigram Language Modeling (2020)(ndingwall.github.io)

65 points by phewlink 1 year ago | 19 comments

TabPFN and TabICL vs. tuned XGBoost: the model that doesn't train won 14/14(efraingaray.com)

21 points by EfrainGaray 28 days ago | 9 comments

Blog post benchmarking TabPFN and TabICL foundation models against tuned XGBoost on fourteen tabular datasets, finding the no-training models won all fourteen.