An update on residential proxies and the scraper situation(lwn.net)
328 points by chmaynard | 343 comments
LWN article by Jonathan Corbet updating on AI scraper bot traffic overwhelming websites and the role of residential proxy networks.
#ai-scrapers on Hacker News
4 stories tagged ai-scrapers, newest first.
We can't have nice things because of AI scrapers(blog.metabrainz.org)
372 points by LorenDB | 197 comments
MetaBrainz Foundation blog post explaining how AI scrapers are forcing it to restrict or change services it offers to the community.
Ban me at the IP level if you don't like me(boston.conman.org)
526 points by classichasclass | 392 comments
A Boston Diaries blog post complaining about the Thinkbot AI web crawler, which ignores robots.txt and asks to be blocked by IP across 74 addresses from 41 network blocks.
The Web Is Broken – Botnet Part 2(jan.wildeboer.net)
371 points by todsacerdoti | 225 comments
Blog post by Jan Wildeboer, second in a series, arguing the web is broken by botnets, likely covering apps embedding SDKs that turn user devices into scraping proxies.