Train Your Own LLM from Scratch(github.com)
376 points by kristianpaul | 43 comments
GitHub repository by angelos-p providing code and docs for training your own large language model from scratch.
#language-model-training on Hacker News
3 stories tagged language-model-training, newest first.
TimeCapsuleLLM: LLM trained only on data from 1800-1875(github.com)
704 points by admp | 290 comments
GitHub repository for TimeCapsuleLLM, a language model trained only on texts from 1800-1875 London to reduce modern bias.
SmolGPT: A minimal PyTorch implementation for training a small LLM from scratch(github.com)
415 points by amrrs | 52 comments
GitHub repository for smolGPT, a minimal PyTorch implementation for training a small GPT-style language model from scratch.