3 stories tagged distributed-inference, newest first.
355 points by antipaul | 188 comments
Apple Machine Learning Research post on running large LLM inference locally and distributed across Mac Studio machines using MLX and RDMA.
326 points by tionis | 76 comments
Blog post from the iroh (n0) team introducing Mesh LLM, a system for distributed AI model inference across devices networked with iroh.
396 points by simonpure | 137 comments
GitHub repository for exo, an open-source tool for running a distributed AI inference cluster at home across everyday devices like phones and laptops.