felafax on Hacker News: 1 stories with screenshots

We fine-tuned Llama 405B on AMD GPUs (publish.obsidian.md)

452 points by felarof 2 years ago | 87 comments

Felafax blog post describing how they fine-tuned Llama 3.1 405B on 8 AMD MI300X GPUs using JAX, covering parameter sharding and LoRA.