ClaudeBot
AI & BotsAnthropic's web crawler, used to gather data referenced by Claude models.
ClaudeBot identifies itself in the User-Agent header. Like other named AI crawlers, its access can be permitted or restricted independently via robots.txt.
Anthropic's web crawler, used to gather data referenced by Claude models.
ClaudeBot identifies itself in the User-Agent header. Like other named AI crawlers, its access can be permitted or restricted independently via robots.txt.