How to Think About GPUs(jax-ml.github.io)
361 points by alphabetting | 107 comments
Part 12 of Google's "How To Scale Your Model" book, a deep dive into how NVIDIA GPUs work and are networked, and what that means for LLMs compared to TPUs.
#jax on Hacker News
3 stories tagged jax, newest first.
We fine-tuned Llama 405B on AMD GPUs(publish.obsidian.md)
452 points by felarof | 87 comments
Felafax blog post describing how they fine-tuned Llama 3.1 405B on 8 AMD MI300X GPUs using JAX, covering parameter sharding and LoRA.
Grok(github.com)
1034 points by pierre | 375 comments
GitHub repository from xAI containing the open release of the Grok language model code and checkpoint loading scripts under Apache-2.0.