AI crawler · OpenAI
OAI-SearchBot: what it is and how to block it
OAI-SearchBot is the crawler behind ChatGPT search. If you block it, OpenAI says your pages won't be shown in ChatGPT search answers, though they can still appear as navigational links.
BrandVector Editorial · Last verified Oct 8, 2026
- Operator
- OpenAI
- Type
- Search
- robots.txt token
OAI-SearchBot- Follows robots.txt
- Yes, per OpenAI
- IP ranges
- Published
- Blocked by name
- 5.1% of top sites
What is OAI-SearchBot?
OAI-SearchBot is OpenAI's search crawler: it indexes pages for search results. In OpenAI's words:
“OAI-SearchBot is used to surface websites in search results in ChatGPT's search features.”
“For search results, please note it can take ~24 hours from a site's robots.txt update for our systems to adjust.”
OAI-SearchBot user agent string
OpenAI documents these user agent strings for OAI-SearchBot:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbot
In robots.txt, use the token OAI-SearchBot, not the full string. Crawlers match on the token, and version numbers in the full string can change.
How many websites block OAI-SearchBot?
In our Oct 8, 2026 check of 817 of the most-visited websites, 5.1% block OAI-SearchBot by name for their whole site (42 sites).
| robots.txt rule for OAI-SearchBot | Sites | Share |
|---|---|---|
| Blocks the whole site by name | 42 | 5.1% |
| Blocks part of the site by name | 35 | 4.3% |
| Names it but doesn't block it | 21 | 2.6% |
| Blocked only by a catch-all (*) rule | 28 | 3.4% |
For comparison, 0.1% of the same sites block Googlebot by name. The sites are the top 1,000 origins in the Chrome UX Report for August 2026; see how we measured.
How to block OAI-SearchBot in robots.txt
Add this group to the robots.txt file at the root of your domain to block OAI-SearchBot from your whole site:
User-agent: OAI-SearchBot Disallow: /
To block only part of your site, list those paths instead:
User-agent: OAI-SearchBot Disallow: /private/
A group that names OAI-SearchBot replaces your User-agent: * rules for it entirely (RFC 9309). If your catch-all group blocks paths that OAI-SearchBot should also stay out of, repeat them in its group. To let OAI-SearchBot in while a catch-all rule blocks others:
User-agent: * Disallow: / User-agent: OAI-SearchBot Allow: /
“OpenAI uses OAI-SearchBot and GPTBot robots.txt tags to enable webmasters to manage how their sites and content work with AI.”
What happens if you block OAI-SearchBot?
“Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.”
How to verify requests from OAI-SearchBot
OpenAI publishes the IP addresses OAI-SearchBot uses. Check a request's IP address against that list before trusting its user agent, which any client can fake.
Other OpenAI crawlers
- GPTBot · AI training
- ChatGPT-User · User-triggered
- OAI-AdsBot · Ad review
See every AI crawler and how often top sites block it
Sources
Rather have an expert do it?
Tell us about your site and goals, and we'll match you with an AI visibility (GEO) specialist. Free, no obligation.