All checks › Bot Access Control
AI bot rules
Why it matters
Generic bots and AI bots have different interests — one indexes for search, the other may be collecting training data or fetching context to answer a question in real time. Having specific rules for bots such as GPTBot, ClaudeBot or Google-Extended lets the site state its policy explicitly (allow, block or differentiate) instead of leaving each AI crawler to decide on its own how to behave, with no signal from the site.
Want to see how your site does?
The free scan runs this and every other check in about 30 seconds and shows the evidence for each one.
Scan my site for free