How to stop your CDN or security plugin blocking AI bots

Even when robots.txt allows AI crawlers, your CDN, firewall, or security plugin can block them at the network level. AI Search Score checks for signs of this.

Why this matters

robots.txt is a polite request; a CDN/WAF block is a locked door. Services like Cloudflare and many security plugins now offer “block AI bots” toggles, sometimes on by default. If one is active, AI crawlers get turned away before they ever read your robots.txt — and you silently vanish from AI citation.

How to fix it

• Cloudflare: check for a “Block AI Scrapers and Crawlers” / bot-management setting and ensure it’s not blocking the crawlers you want.
• Security plugins (Wordfence, etc.): review their bot/crawler blocking rules.
• Host/CDN firewall: check for user-agent rules that target AI bots.

If you use Quic.cloud or LiteSpeed, review any crawler/bot controls there too. Allow the AI user-agents you want citing you (GPTBot, ClaudeBot, PerplexityBot, Google-Extended).

How to check it worked

Re-scan. “Not blocking AI crawlers” should turn green. For a deeper check, you can test how your site responds to an AI user-agent string and confirm it returns your page, not a block page.

Prefer this diagnosed for you? See our services.