PyTorch Monarch(pytorch.org)
360 points by jarbus | 42 comments
PyTorch blog post from Meta introducing Monarch, a framework moving beyond multi-controller SPMD for heterogeneous, asynchronous, fault-tolerant ML workflows.
#distributed-training on Hacker News
2 stories tagged distributed-training, newest first.
How Meta trains large language models at scale(engineering.fb.com)
362 points by mfiguiere | 183 comments
Engineering at Meta blog post describing the software, hardware, and network considerations behind training large language models at scale in production.