AI crawler · Perplexity
PerplexityBot: what it is and how to block it
PerplexityBot crawls the web to surface and link sites in Perplexity's search results. Perplexity says it isn't used to train AI models, and that blocked pages may still get a domain, headline, and short summary indexed.
BrandVector Editorial · Last verified Oct 8, 2026
- Operator
- Perplexity
- Type
- Search
- robots.txt token
PerplexityBot- Follows robots.txt
- Yes, per Perplexity
- IP ranges
- Published
- Blocked by name
- 8.8% of top sites
What is PerplexityBot?
PerplexityBot is Perplexity's search crawler: it indexes pages for search results. In Perplexity's words:
“PerplexityBot is designed to surface and link websites in search results on Perplexity. It is not used to crawl content for AI foundation models.”
“Each setting works independently, and it may take up to 24 hours for our systems to reflect changes.”
PerplexityBot user agent string
Perplexity documents this user agent string for PerplexityBot:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
In robots.txt, use the token PerplexityBot, not the full string. Crawlers match on the token, and version numbers in the full string can change.
How many websites block PerplexityBot?
In our Oct 8, 2026 check of 817 of the most-visited websites, 8.8% block PerplexityBot by name for their whole site (72 sites).
| robots.txt rule for PerplexityBot | Sites | Share |
|---|---|---|
| Blocks the whole site by name | 72 | 8.8% |
| Blocks part of the site by name | 41 | 5.0% |
| Names it but doesn't block it | 16 | 2.0% |
| Blocked only by a catch-all (*) rule | 21 | 2.6% |
For comparison, 0.1% of the same sites block Googlebot by name. The sites are the top 1,000 origins in the Chrome UX Report for August 2026; see how we measured.
How to block PerplexityBot in robots.txt
Add this group to the robots.txt file at the root of your domain to block PerplexityBot from your whole site:
User-agent: PerplexityBot Disallow: /
To block only part of your site, list those paths instead:
User-agent: PerplexityBot Disallow: /private/
A group that names PerplexityBot replaces your User-agent: * rules for it entirely (RFC 9309). If your catch-all group blocks paths that PerplexityBot should also stay out of, repeat them in its group. To let PerplexityBot in while a catch-all rule blocks others:
User-agent: * Disallow: / User-agent: PerplexityBot Allow: /
“Our crawler, PerplexityBot, will not index the full or partial text content of any site that disallows it via robots.txt.”
What happens if you block PerplexityBot?
“However, if a page is blocked, we may still index the domain, headline, and a brief factual summary.”
How to verify requests from PerplexityBot
Perplexity publishes the IP addresses PerplexityBot uses. Check a request's IP address against that list before trusting its user agent, which any client can fake.
“combine both User-Agent string matching and IP address verification for enhanced security”
Other Perplexity crawlers
- Perplexity-User · User-triggered
See every AI crawler and how often top sites block it
Sources
Rather have an expert do it?
Tell us about your site and goals, and we'll match you with an AI visibility (GEO) specialist. Free, no obligation.