Language models pack billions of concepts into 12k dimensions(nickyoder.com)
336 points by lawrenceyan | 119 comments
A blog post explaining how language models represent vast numbers of concepts in 12,000-dimensional embedding spaces using near-orthogonality and the Johnson-Lindenstrauss lemma.