ProgramBench: Can Language Models Rebuild Programs from Scratch?(arxiv.org)

110 points by jonbaer 5 months ago | 60 comments

MIT Course: Generative AI for Constructive Communication(ai4comm.media.mit.edu)

121 points by tim_sw 3 years ago | 7 comments

Scalable MatMul-Free Language Modeling(arxiv.org)

177 points by lykahb 2 years ago | 27 comments

Teaching Large Language Models to Self-Debug(arxiv.org)

79 points by saurabh20n 3 years ago | 44 comments

Scaling Data-Constrained Language Models(arxiv.org)

44 points by ijk 3 years ago | 5 comments

I built a large language model "from scratch"(brettgfitzgerald.com)

66 points by controversy187 1 year ago | 4 comments

Training LLMs from ground zero as a startup(yitay.net)

360 points by swyx 2 years ago | 92 comments

Yi Tay's blog post on the experience of training large language models from scratch at a startup (Reka), covering compute, infrastructure, and lessons learned.

C++26: The New Debugging Header(sandordargo.com)

98 points by jandeboevrie 9 days ago | 23 comments

Blog post by Sandor Dargo introducing the C++26 <debugging> header for programmatic breakpoints, replacing compiler-specific code like __debugbreak.

PSSA: A non-transformer language model written from scratch in Rust(github.com)

76 points by sparticle62 26 days ago | 32 comments

GitHub repository for PSSA, a custom non-transformer language model architecture written from scratch in Rust.

LLM from scratch, part 28 – training a base model from scratch on an RTX 3090(gilesthomas.com)

482 points by gpjt 10 months ago | 101 comments

Blog post, part 28 of a series, describing an experiment training a ~163M-parameter base LLM from scratch on a single RTX 3090 at home.

MIT 6.S184: Introduction to Flow Matching and Diffusion Models(diffusion.csail.mit.edu)

362 points by __rito__ 1 year ago | 21 comments

Course page for MIT 6.S184, an introduction to flow matching and diffusion models built from stochastic differential equations, with notes, lectures, and labs.

Grok(github.com)

1034 points by pierre 2 years ago | 375 comments

GitHub repository from xAI containing the open release of the Grok language model code and checkpoint loading scripts under Apache-2.0.