Perplexity is using stealth, undeclared crawlers to evade no-crawl directives(blog.cloudflare.com)
1183 points by rrampage | 652 comments
Cloudflare blog post reporting that Perplexity uses stealth, undeclared crawlers to evade websites' robots.txt and no-crawl directives.












