Perplexity is using stealth, undeclared crawlers to evade no-crawl directives(blog.cloudflare.com)
1183 points by rrampage | 652 comments
Cloudflare blog post reporting that Perplexity uses stealth, undeclared crawlers to evade websites' robots.txt and no-crawl directives.


























![Screenshot of lweb.cfa.harvard.edu: Is the interstellar object 3I/ATLAS alien technology? [pdf]](https://hn.urlbox.com/shots/renders/2025/8/4/891e8ef6-028c-4807-b278-688b4ef81dd9.jpg)
































