Free crawler check
Can ChatGPT access your website?
Check the robots.txt rules and HTTP responses for OpenAI’s crawlers. Find an access problem before spending time rewriting your content.
Live HTTP check — we request your page with each bot’s user agent and show what comes back. No signup needed.
No signup for the quick check. Requests come from BotView, not OpenAI’s network. An allowed response does not guarantee a ChatGPT mention.
Need screenshots and a GEO content score? Run a full scan.Your next step · Pro
Get your AI business analysis
Find out how LLMs describe your business and who they recommend for services like yours. We test questions based on your business and services, with web search.
- Your business: what AI gets right, misses or gets wrong.
- Your competitors: who appears instead, and the reasons the answer gives.
- Your next steps: answers and sources to help you improve your site.
$9/month · 20 AI checks + 50 crawler scans · Cancel anytime
Start with a five-question analysis (5 checks). Rerun a question for 1 check. Currently powered by OpenAI with web search; results are sampled answers, not measured rankings.
What to do with your result
Search crawler allowed
The check found no access restriction for OAI-SearchBot in the signals it could verify.
Next step: Run a full scan to check the page content. Access is only one requirement for being useful in search.
Disallowed in robots.txt
A matching rule asks OAI-SearchBot not to crawl this URL.
Next step: If that restriction is accidental, edit the matching rule. Keep any private paths restricted, then check the same URL again.
Challenge, error or could not verify
BotView could not confirm access. A firewall may treat our request differently from a verified OpenAI crawler.
Next step: Check your CDN security events and compare requests against OpenAI’s published crawler IPs before changing firewall rules.
GPTBot and OAI-SearchBot have different jobs
OAI-SearchBot supports ChatGPT search. GPTBot collects content that may be used for model training. ChatGPT-User retrieves pages for some user requests; OpenAI says robots.txt may not apply to those requests.
OpenAI documents independent controls for these agents. Blocking GPTBot is a training preference; it is not a substitute for checking your OAI-SearchBot rules.
Allow search while opting out of training
This example applies to public content on a site with no additional path restrictions. Merge the policy into your existing robots.txt instead of replacing the file; a crawler-specific group can change how wildcard rules apply.
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /A successful check does not explain every missing citation
If the page is accessible, check whether its main answer is present in the initial HTML and whether it adds something worth referencing. BotView’s full report helps inspect those on-page signals. It does not query ChatGPT to count mentions or tell you your position in an answer.
Questions about this check
Will allowing OAI-SearchBot make ChatGPT recommend my business?
No. Allowing a crawler permits access; it does not choose which pages a search system retrieves or cites. A recommendation also depends on the question and the content available.
Can ChatGPT still link to a site that blocks OAI-SearchBot?
OpenAI says opted-out sites will not appear in ChatGPT search answers, but may still appear as navigational links. That is narrower than saying the site disappears from ChatGPT altogether.
How do I know whether a firewall blocked the real crawler?
Inspect server or CDN logs and verify the request’s source IP against the vendor’s current published list. A user-agent string alone is not proof of identity. BotView’s request is a diagnostic probe.
Sources and further checks
Crawler documentation checked September 6, 2026.
Check the content behind the response
The full scan compares screenshots, initial HTML and rendered content, and checks on-page GEO signals. Free scans cover up to three pages. Results describe this scan; they do not measure search rankings or AI citations.