Self-Distillation Enables Continual Learning [pdf] (arxiv.org)

Screenshot of arxiv.org: Self-Distillation Enables Continual Learning [pdf]

First screen, captured 5 months ago.

Looks similar

Screenshot of arxiv.org: Self-Adapting Language Models

Self-Adapting Language Models (arxiv.org)

210 points by archon1410 1 year ago | 61 comments

Screenshot of arxiv.org: LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics

LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics (arxiv.org)

61 points by nothrowaways 11 months ago | 14 comments

Screenshot of arxiv.org: Embarrassingly simple self-distillation improves code generation

Embarrassingly simple self-distillation improves code generation (arxiv.org)

605 points by Anon84 6 months ago | 182 comments

arXiv paper showing that fine-tuning an LLM on its own sampled code outputs (simple self-distillation) improves code generation, e.g. Qwen3-30B on LiveCodeBench.

self-distillation code-generation llm fine-tuning

Screenshot of arxiv.org: Alignment pretraining: AI discourse creates self-fulfilling (mis)alignment

Alignment pretraining: AI discourse creates self-fulfilling (mis)alignment (arxiv.org)

67 points by anigbrowl 5 months ago | 25 comments

Screenshot of arxiv.org: CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs (arxiv.org)

96 points by matt_d 5 months ago | 12 comments

Screenshot of arxiv.org: Self-Rewarding Language Models

Self-Rewarding Language Models (arxiv.org)

56 points by jph00 2 years ago | 49 comments

Looks similar

Self-Adapting Language Models(arxiv.org)

LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics(arxiv.org)

Embarrassingly simple self-distillation improves code generation(arxiv.org)

Alignment pretraining: AI discourse creates self-fulfilling (mis)alignment(arxiv.org)

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs(arxiv.org)

Self-Rewarding Language Models(arxiv.org)