Maintaining large-scale AI capacity at Meta(engineering.fb.com)

86 points by samber 2 years ago | 50 comments

Building Meta's GenAI infrastructure(engineering.fb.com)

636 points by mootpt 2 years ago | 291 comments

Meta engineering blog post describing its large GPU training clusters for generative AI, including RDMA and InfiniBand networking, Grand Teton hardware, and storage.

Meta developer tools: Working at scale(engineering.fb.com)

210 points by ot 3 years ago | 162 comments

Meta’s renewed commitment to jemalloc(engineering.fb.com)

495 points by hahahacorn 7 months ago | 226 comments

Engineering at Meta blog post announcing Meta's renewed investment in and commitment to maintaining the open-source jemalloc memory allocator.

Meta's GB{2,3}00 Cluster(engineering.fb.com)

75 points by stokedmartin 7 days ago | 31 comments

Engineering at Meta blog post describing Prometheus, Meta's gigawatt-scale AI supercluster and the data center and networking work behind it.

Meta enforces purpose limitation via Privacy Aware Infrastructure at scale(engineering.fb.com)

32 points by seanieb 2 years ago | 39 comments

Buck2: Our open source build system(engineering.fb.com)

327 points by mfiguiere 3 years ago | 213 comments

Meta Engineering blog post announcing the open source release of Buck2, its large-scale build system.

Building the heap: racking 30 petabytes of hard drives for pretraining(si.inc)

392 points by nee1r 1 year ago | 267 comments

Standard Intelligence blog post describing how they built a 30-petabyte hard drive storage cluster in a San Francisco colocation center for under $500k to store video pretraining data.

Stop syncing everything(sqlsync.dev)

607 points by neilk 1 year ago | 110 comments

Carl Sverre's blog post launching Graft, an open-source transactional storage engine for lazy, partial replication with strong consistency backed by object storage.

MLow: Meta's low bitrate audio codec(engineering.fb.com)

551 points by mikece 2 years ago | 188 comments

Engineering at Meta blog post introducing MLow, Meta's low bitrate audio codec for real-time calling.

Twelve years of distributed SQLite(philipotoole.com)

365 points by otoolep 2 days ago | 62 comments

Philip O'Toole's blog post recounting the history of distributed SQLite since he launched rqlite in 2014, covering dqlite, Litestream, LiteFS, libSQL/Turso, cr-sqlite and Cloudflare D1.

DuckDB Doesn't Need Data to Be a Database(nikolasgoebel.com)

353 points by tosh 2 years ago | 115 comments

A blog post by Nikolas Göbel discussing how DuckDB can act as a database without storing data itself, querying external data sources directly.