{"id":133914,"date":"2025-08-04T19:16:26","date_gmt":"2025-08-04T19:16:26","guid":{"rendered":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/"},"modified":"2025-08-04T19:16:26","modified_gmt":"2025-08-04T19:16:26","slug":"ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says","status":"publish","type":"post","link":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/","title":{"rendered":"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says"},"content":{"rendered":"<div>\n<p>AI search engine Perplexity is using stealth bots and other tactics to evade websites\u2019 no-crawl directives, an allegation that if true violates Internet norms that have been in place for more than three decades, network security and optimization service Cloudflare said Monday.<\/p>\n<p>In a <a href=\"https:\/\/blog.cloudflare.com\/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives\/\">blog post<\/a>, Cloudflare researchers said the company received complaints from customers who had disallowed Perplexity scraping bots by implementing settings in their sites\u2019 robots.txt files and through Web application firewalls that blocked the declared Perplexity crawlers. Despite those steps, Cloudflare said, Perplexity continued to access the sites\u2019 content.<\/p>\n<p>The researchers said they then set out to test it for themselves and found that when known Perplexity crawlers encountered blocks from robots.txt files or firewall rules, Perplexity then searched the sites using a stealth bot that followed a range of tactics to mask its activity.<\/p>\n<p><a href=\"https:\/\/arstechnica.com\/information-technology\/2025\/08\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/\">Read full article<\/a><\/p>\n<p><a href=\"https:\/\/arstechnica.com\/information-technology\/2025\/08\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/#comments\">Comments<\/a><\/p>\n<\/div>\n<p class=\"wpematico_credit\"><small>Powered by <a href=\"http:\/\/www.wpematico.com\" target=\"_blank\">WPeMatico<\/a><\/small><\/p>\n","protected":false},"excerpt":{"rendered":"<p>AI search engine Perplexity is using stealth bots and other tactics to evade websites\u2019 no-crawl directives, an allegation that if true violates Internet norms that have been in place for more than three decades, network security and optimization service Cloudflare said Monday. In a blog post, Cloudflare researchers said the company received complaints from customers who had disallowed Perplexity scraping bots by implementing settings in their sites\u2019 robots.txt files and&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[241],"tags":[],"class_list":["post-133914","post","type-post","status-publish","format-standard","hentry","category-technology"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says - UshopWell.com<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says - UshopWell.com\" \/>\n<meta property=\"og:description\" content=\"AI search engine Perplexity is using stealth bots and other tactics to evade websites\u2019 no-crawl directives, an allegation that if true violates Internet norms that have been in place for more than three decades, network security and optimization service Cloudflare said Monday. In a blog post, Cloudflare researchers said the company received complaints from customers who had disallowed Perplexity scraping bots by implementing settings in their sites\u2019 robots.txt files and...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/\" \/>\n<meta property=\"og:site_name\" content=\"UshopWell.com\" \/>\n<meta property=\"article:published_time\" content=\"2025-08-04T19:16:26+00:00\" \/>\n<meta name=\"author\" content=\"UShopWell\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"UShopWell\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\\\/\"},\"author\":{\"name\":\"UShopWell\",\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/#\\\/schema\\\/person\\\/6fd1f9e0ff932e534c86c70d5acff0fc\"},\"headline\":\"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says\",\"datePublished\":\"2025-08-04T19:16:26+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\\\/\"},\"wordCount\":159,\"publisher\":{\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/#organization\"},\"articleSection\":[\"Technology\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\\\/\",\"url\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\\\/\",\"name\":\"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says - UshopWell.com\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/#website\"},\"datePublished\":\"2025-08-04T19:16:26+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/#website\",\"url\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/\",\"name\":\"UshopWell.com\",\"description\":\"The Premiere Online Marketplace\",\"publisher\":{\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/#organization\",\"name\":\"UshopWell\",\"url\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/wp-content\\\/uploads\\\/2018\\\/01\\\/pandaSwea.png\",\"contentUrl\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/wp-content\\\/uploads\\\/2018\\\/01\\\/pandaSwea.png\",\"width\":365,\"height\":359,\"caption\":\"UshopWell\"},\"image\":{\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/#\\\/schema\\\/person\\\/6fd1f9e0ff932e534c86c70d5acff0fc\",\"name\":\"UShopWell\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/4adb372cadd43b4d4c57964dab95b0f69618bf960d131c4acf49d96d6bbc9c6e?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/4adb372cadd43b4d4c57964dab95b0f69618bf960d131c4acf49d96d6bbc9c6e?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/4adb372cadd43b4d4c57964dab95b0f69618bf960d131c4acf49d96d6bbc9c6e?s=96&d=mm&r=g\",\"caption\":\"UShopWell\"},\"url\":\"https:\\\/\\\/ushopwell.com\\\/ublog\\\/author\\\/kburnettu\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says - UshopWell.com","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/","og_locale":"en_US","og_type":"article","og_title":"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says - UshopWell.com","og_description":"AI search engine Perplexity is using stealth bots and other tactics to evade websites\u2019 no-crawl directives, an allegation that if true violates Internet norms that have been in place for more than three decades, network security and optimization service Cloudflare said Monday. In a blog post, Cloudflare researchers said the company received complaints from customers who had disallowed Perplexity scraping bots by implementing settings in their sites\u2019 robots.txt files and...","og_url":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/","og_site_name":"UshopWell.com","article_published_time":"2025-08-04T19:16:26+00:00","author":"UShopWell","twitter_card":"summary_large_image","twitter_misc":{"Written by":"UShopWell","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/#article","isPartOf":{"@id":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/"},"author":{"name":"UShopWell","@id":"https:\/\/ushopwell.com\/ublog\/#\/schema\/person\/6fd1f9e0ff932e534c86c70d5acff0fc"},"headline":"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says","datePublished":"2025-08-04T19:16:26+00:00","mainEntityOfPage":{"@id":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/"},"wordCount":159,"publisher":{"@id":"https:\/\/ushopwell.com\/ublog\/#organization"},"articleSection":["Technology"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/","url":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/","name":"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says - UshopWell.com","isPartOf":{"@id":"https:\/\/ushopwell.com\/ublog\/#website"},"datePublished":"2025-08-04T19:16:26+00:00","breadcrumb":{"@id":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ushopwell.com\/ublog\/ai-site-perplexity-uses-stealth-tactics-to-flout-no-crawl-edicts-cloudflare-says\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ushopwell.com\/ublog\/"},{"@type":"ListItem","position":2,"name":"AI site Perplexity uses \u201cstealth tactics\u201d to flout no-crawl edicts, Cloudflare says"}]},{"@type":"WebSite","@id":"https:\/\/ushopwell.com\/ublog\/#website","url":"https:\/\/ushopwell.com\/ublog\/","name":"UshopWell.com","description":"The Premiere Online Marketplace","publisher":{"@id":"https:\/\/ushopwell.com\/ublog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ushopwell.com\/ublog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ushopwell.com\/ublog\/#organization","name":"UshopWell","url":"https:\/\/ushopwell.com\/ublog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ushopwell.com\/ublog\/#\/schema\/logo\/image\/","url":"https:\/\/ushopwell.com\/ublog\/wp-content\/uploads\/2018\/01\/pandaSwea.png","contentUrl":"https:\/\/ushopwell.com\/ublog\/wp-content\/uploads\/2018\/01\/pandaSwea.png","width":365,"height":359,"caption":"UshopWell"},"image":{"@id":"https:\/\/ushopwell.com\/ublog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/ushopwell.com\/ublog\/#\/schema\/person\/6fd1f9e0ff932e534c86c70d5acff0fc","name":"UShopWell","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/4adb372cadd43b4d4c57964dab95b0f69618bf960d131c4acf49d96d6bbc9c6e?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/4adb372cadd43b4d4c57964dab95b0f69618bf960d131c4acf49d96d6bbc9c6e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4adb372cadd43b4d4c57964dab95b0f69618bf960d131c4acf49d96d6bbc9c6e?s=96&d=mm&r=g","caption":"UShopWell"},"url":"https:\/\/ushopwell.com\/ublog\/author\/kburnettu\/"}]}},"_links":{"self":[{"href":"https:\/\/ushopwell.com\/ublog\/wp-json\/wp\/v2\/posts\/133914","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ushopwell.com\/ublog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ushopwell.com\/ublog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ushopwell.com\/ublog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ushopwell.com\/ublog\/wp-json\/wp\/v2\/comments?post=133914"}],"version-history":[{"count":0,"href":"https:\/\/ushopwell.com\/ublog\/wp-json\/wp\/v2\/posts\/133914\/revisions"}],"wp:attachment":[{"href":"https:\/\/ushopwell.com\/ublog\/wp-json\/wp\/v2\/media?parent=133914"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ushopwell.com\/ublog\/wp-json\/wp\/v2\/categories?post=133914"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ushopwell.com\/ublog\/wp-json\/wp\/v2\/tags?post=133914"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}