Crawling a quarter billion webpages in 40 hours (2012)(michaelnielsen.org)184 points by swyx 3 years ago | 63 comments
