Perplexity: Unblock AI Crawlers from Your Website
How to fix the problem of ai crawlers blocked by robots.txt specifically for Perplexity. Platform-specific solutions and prevention strategies.
Perplexity에서 이 문제가 발생하는 이유
Perplexity uses PerplexityBot to discover and index content. Blanket 'Disallow: /' rules affecting AI bots. To fix this, you need to ensure Perplexity's crawler can access your content and that your pages are structured in a way Perplexity can understand.
Perplexity에서 이 문제의 징후
PerplexityBot is blocked from crawling your site
Perplexity's real-time search can't access your pages
Server logs show no PerplexityBot visits
Perplexity can't cite you because it can't reach your content
Perplexity에서 이 문제를 해결하는 방법
다음 Perplexity 전용 솔루션을 따라 하세요:
Add to robots.txt: User-agent: PerplexityBot\nAllow: /
Ensure real-time crawling isn't blocked
Remove restrictions affecting PerplexityBot
Monitor server logs for PerplexityBot activity
Test Perplexity search to verify access
Keep robots.txt rules permissive for AI crawlers
Perplexity 기술 세부사항
크롤러 이름
PerplexityBot
User Agent
PerplexityBot
필수 robots.txt 설정:
User-agent: PerplexityBot
Allow: /Perplexity에서 이 문제를 예방하는 방법
Audit robots.txt (for PerplexityBot) quarterly
Stay updated on new Perplexity crawler user agents
Test Perplexity crawler access after any robots.txt (for PerplexityBot) changes
Balance security needs with Perplexity visibility goals