Scalable MatMul-Free Language Modeling(arxiv.org)
177 points by lykahb | 27 comments
177 points by lykahb | 27 comments
447 points by zhisbug | 94 comments
Hao AI Lab blog post introducing Consistency LLMs, a method that fine-tunes LLMs into parallel Jacobi decoders to speed up inference about 3.5x.
108 points by apsec112 | 23 comments