We fine-tuned Llama 405B on AMD GPUs(publish.obsidian.md)
452 points by felarof | 87 comments
Felafax blog post describing how they fine-tuned Llama 3.1 405B on 8 AMD MI300X GPUs using JAX, covering parameter sharding and LoRA.
#model-sharding on Hacker News
1 story tagged model-sharding, newest first.