Self-Distillation Enables Continual Learning [pdf](arxiv.org)

78 points by teleforce 5 months ago | 20 comments

Embarrassingly simple self-distillation improves code generation(arxiv.org)

605 points by Anon84 6 months ago | 182 comments

arXiv paper showing that fine-tuning an LLM on its own sampled code outputs (simple self-distillation) improves code generation, e.g. Qwen3-30B on LiveCodeBench.

Dream-RSI: Recursive Self-Improvement through Evolving Worlds(arxiv.org)

200 points by bananaflag 1 month ago | 49 comments

A Comprehensive Survey of Self-Evolving AI Agents [pdf](arxiv.org)

68 points by SerCe 1 year ago | 16 comments

Self-Adapting Language Models(arxiv.org)

210 points by archon1410 1 year ago | 61 comments

Self-Taught Optimizer (Stop): Recursively Self-Improving Code Generation(arxiv.org)

39 points by birriel 3 years ago | 8 comments

Training language models to be warm and empathetic makes them less reliable(arxiv.org)

334 points by Cynddl 1 year ago | 333 comments

arXiv paper finding that fine-tuning language models to be warm and empathetic raises error rates and makes them more sycophantic on safety-critical tasks.

Training LLMs from ground zero as a startup(yitay.net)

360 points by swyx 2 years ago | 92 comments

Yi Tay's blog post on the experience of training large language models from scratch at a startup (Reka), covering compute, infrastructure, and lessons learned.

SmolGPT: A minimal PyTorch implementation for training a small LLM from scratch(github.com)

415 points by amrrs 1 year ago | 52 comments

GitHub repository for smolGPT, a minimal PyTorch implementation for training a small GPT-style language model from scratch.

The Linux Kernel Module Programming Guide(sysprog21.github.io)

340 points by asicsp 3 years ago | 26 comments

Online edition of The Linux Kernel Module Programming Guide by Salzman, Burian, Pomerantz, Mottram and Huang, with GitHub source and PDF download.

Thinking fast and slow in AI: The role of metacognition (2021)(arxiv.org)

165 points by teleforce 28 days ago | 68 comments

An arXiv paper proposing a multi-agent AI architecture inspired by Kahneman's fast and slow thinking, with a metacognitive module choosing between system 1 and system 2 agents.

Turning GLM-5.3-Flash into a Jev-like decision model(privatemode.ai)

116 points by flxflx 1 month ago | 50 comments

Privatemode AI blog post describing how to prompt GLM-5.3-Flash so its first output token gives a decision in a single forward pass, benchmarked against Jev and Laya.