User-agent: * Allow: / # AI crawlers. Two distinct decisions, stated separately: # - search/answer crawlers (OAI-SearchBot, PerplexityBot, ClaudeBot) may # index and cite, with attribution and the canonical URL; # - training crawlers (GPTBot, Google-Extended, CCBot) are allowed too. # Attribution is a request, not something robots.txt can enforce. User-agent: OAI-SearchBot Allow: / User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: Google-Extended Allow: / User-agent: CCBot Allow: / User-agent: PerplexityBot Allow: / Sitemap: https://margiovanni.it/sitemap.xml # LLM-friendly content index (llmstxt.org standard) # Index: https://margiovanni.it/llms.txt # Full dump:https://margiovanni.it/llms-full.txt