Graph Language Models(aclanthology.org)

111 points by Anon84 2 years ago | 9 comments

Context Language Models(arxiv.org)

164 points by emersonmacro 25 days ago | 44 comments

arXiv paper introducing Context Language Models, LLMs that natively manage their own context by treating it as an editable file, with benchmarks on agent tasks.

Vision language models are blind(vlmsareblind.github.io)

425 points by taesiri 2 years ago | 172 comments

Project page for the research paper "Vision language models are blind" from Auburn University and University of Alberta, with links to the arXiv paper, code, and dataset.

Diffusion language models are super data learners(jinjieni.notion.site)

202 points by babelfish 1 year ago | 14 comments

Language Models Represent Space and Time(arxiv.org)

103 points by birriel 3 years ago | 96 comments

Continuous Diffusion Language Models (CDLM's)(sander.ai)

115 points by peter_d_sherman 1 month ago | 37 comments

Language models can explain neurons in language models(openai.com)

659 points by mfiguiere 3 years ago | 416 comments

OpenAI research post describing using GPT-4 to automatically generate and score explanations of individual neuron behavior in GPT-2.

LLM Visualization(bbycroft.net)

910 points by jonbaer 2 years ago | 1 comments

An interactive 3D walkthrough visualizing how a GPT-style large language model works, step by step, using nano-gpt, GPT-2, and GPT-3 examples.

Language models pack billions of concepts into 12k dimensions(nickyoder.com)

336 points by lawrenceyan 1 year ago | 119 comments

A blog post explaining how language models represent vast numbers of concepts in 12,000-dimensional embedding spaces using near-orthogonality and the Johnson-Lindenstrauss lemma.

Decomposing language models into understandable components(anthropic.com)

402 points by tompark 3 years ago | 58 comments

Anthropic research post from Oct 2023 on decomposing language model neurons into more interpretable features.

Training LLMs from ground zero as a startup(yitay.net)

360 points by swyx 2 years ago | 92 comments

Yi Tay's blog post on the experience of training large language models from scratch at a startup (Reka), covering compute, infrastructure, and lessons learned.

Deep Learning Is Not So Mysterious or Different(arxiv.org)

445 points by wuubuu 1 year ago | 112 comments

arXiv paper by Andrew Gordon Wilson arguing deep learning generalization phenomena like benign overfitting and double descent are explainable by soft inductive biases.