Amazon's AI crawler is making my Git server unstable(xeiaso.net)
532 points by OptionOfT | 207 comments
Xe Iaso's blog note describing how Amazon's AI crawler is overloading their self-hosted git server and an nginx ingress snippet used to block it.
#amazonbot on Hacker News
2 stories tagged amazonbot, newest first.
AI companies cause most of traffic on forums(pod.geraspora.de)
566 points by ta988 | 335 comments
A diaspora* post by Dennis Schubert reporting that LLM training crawlers like GPTBot, Amazonbot and ClaudeBot account for about 70% of the project's web infrastructure traffic.