DeepSeek Elastic Compute (DSec)(arxiv.org)
308 points by shenli3514 | 99 comments
arXiv paper from DeepSeek describing DSec, an elastic sandbox compute infrastructure for training AI agents at scale.
#llm-training on Hacker News
10 stories tagged llm-training, newest first.
LLM from scratch, part 28 – training a base model from scratch on an RTX 3090(gilesthomas.com)
482 points by gpjt | 101 comments
Blog post, part 28 of a series, describing an experiment training a ~163M-parameter base LLM from scratch on a single RTX 3090 at home.
NanoChat – The best ChatGPT that $100 can buy(github.com)
1409 points by huseyinkeles | 283 comments
GitHub repository for nanochat, Andrej Karpathy's minimal full-stack codebase for training and serving a small ChatGPT-like LLM for about $100.
Train Your Own O1 Preview Model Within $450(sky.cs.berkeley.edu)
399 points by 9woc | 62 comments
UC Berkeley Sky Computing Lab project page for Sky-T1, a reasoning model trained for under $450 that rivals OpenAI's o1-preview.
S1: A $6 R1 competitor?(timkellogg.me)
796 points by tkellogg | 365 comments
Blog post by Tim Kellogg discussing S1, a reasoning model reportedly trained for about $6 that competes with DeepSeek R1.
AI companies cause most of traffic on forums(pod.geraspora.de)
566 points by ta988 | 335 comments
A diaspora* post by Dennis Schubert reporting that LLM training crawlers like GPTBot, Amazonbot and ClaudeBot account for about 70% of the project's web infrastructure traffic.
How Meta trains large language models at scale(engineering.fb.com)
362 points by mfiguiere | 183 comments
Engineering at Meta blog post describing the software, hardware, and network considerations behind training large language models at scale in production.
Reproducing GPT-2 in llm.c(github.com)
578 points by tosh | 109 comments
A GitHub discussion in Andrej Karpathy's llm.c repository about reproducing GPT-2 training from scratch in C/CUDA.
Training LLMs from ground zero as a startup(yitay.net)
360 points by swyx | 92 comments
Yi Tay's blog post on the experience of training large language models from scratch at a startup (Reka), covering compute, infrastructure, and lessons learned.
Replit's new Code LLM: Open Source, 77% smaller than Codex, trained in 1 week(latent.space)
848 points by swyx | 214 comments
Latent Space podcast episode with Replit's Reza Shabani about training Replit's open-source code LLM in one week and evaluating it.