GPTBot
AI & BotsOpenAI's web crawler, used to gather training and browsing data for its models.
GPTBot identifies itself in the User-Agent header and respects robots.txt directives targeting it by name. Sites can allow or block GPTBot specifically without affecting standard search crawlers.