Free tool · no sign-up for your score
Robots.txt AI bot checker
The SiteSignal robots.txt AI bot checker is a free tool that reads your robots.txt and reports, bot by bot, whether known AI crawlers such as GPTBot, ClaudeBot, Google-Extended and CCBot are allowed, blocked or unclear.
Website readiness only: we do not query ChatGPT, Perplexity or Google live, and we do not estimate lost customers. About 30 seconds as an estimate; some sites take up to 90.
Same engine, focused view
Every SiteSignal tool runs the full scan. This one opens your results on the part you came for, and keeps that focus after you unlock the full report.
See a sample reportWhy robots.txt is where AI visibility starts
robots.txt is a small public file that tells automated bots what they may read. Many site templates and security plugins add blanket blocks without the owner knowing. One line can stop an AI product from learning anything from your site.
Bots that do different jobs
Some bots gather pages to train models. Others fetch pages live when a user asks a question. Google-Extended controls certain Google AI uses and does not affect normal Google Search. Treat them separately.
- GPTBot: OpenAI's crawler
- ClaudeBot: Anthropic's crawler
- Google-Extended: a control token for some Google AI features
- CCBot: Common Crawl, a public dataset many models use
Questions owners ask
Not always. It is a business choice. The problem is blocking them by accident. The full report shows each rule so you can decide on purpose.
Usually not. Without the file, bots assume they may read your public pages. We report it as unclear rather than failed.
Other free tools · same scan engine