PyTorch Monarch(pytorch.org)
360 points by jarbus | 42 comments
PyTorch blog post from Meta introducing Monarch, a framework moving beyond multi-controller SPMD for heterogeneous, asynchronous, fault-tolerant ML workflows.
360 points by jarbus | 42 comments
PyTorch blog post from Meta introducing Monarch, a framework moving beyond multi-controller SPMD for heterogeneous, asynchronous, fault-tolerant ML workflows.
190 points by steveklabnik | 93 comments
426 points by bblcla | 78 comments
Blog post recounting how a training loss plateau turned out to be a PyTorch bug, traced through optimizer internals down to GPU kernels.