Is robots.txt Blocking
GPTBot? How to Check
If your robots.txt file is unknowingly blocking GPTBot, ChatGPT can never cite your site as a source. This is one of the most common and costly SEO mistakes in 2026. This guide shows you step by step how to check whether your robots.txt blocks GPTBot and how to fix it.
Quick check: Open yoursite.com/robots.txt in your browser. If you see "User-agent: GPTBot" followed by "Disallow: /", you are completely blocking ChatGPT.
What is GPTBot and Why Does It Matter?
GPTBot is OpenAI's web crawler. It allows ChatGPT to pull fresh information from the web and cite your site as a source. If you block GPTBot, your site will never be recommended when users ask ChatGPT about your industry.
Which AI Bots Exist?
| Bot | Company | Purpose |
|---|---|---|
| GPTBot | OpenAI | ChatGPT source crawling |
| ClaudeBot | Anthropic | Claude source crawling |
| PerplexityBot | Perplexity | Perplexity answer sources |
| Google-Extended | Google AI / Gemini training |
Is robots.txt Blocking GPTBot? Check Steps
1. Open Your robots.txt File
Type your site address followed by /robots.txt. Example: https://yoursite.com/robots.txt
2. Look for GPTBot Lines
If you see lines like this, you are blocking the bots:
User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: /
3. Correct Configuration
For AI bots to read your site, it should look like this:
User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: /
Warning: Some CMS plugins and security tools automatically block AI bots. Always check your robots.txt file manually.
Automatic Check: Scanliq
If you don't want to inspect robots.txt manually, you can check it in seconds with Scanliq. It automatically detects whether GPTBot, ClaudeBot and PerplexityBot are blocked, and analyzes 15 other AI SEO criteria too.
Also check our AI SEO guide and llms.txt guide to fully optimize your site.
Frequently Asked Questions
Is your robots.txt blocking AI bots?
Check it free with Scanliq — find out in seconds.
Test for Free →