etsy.com: what it tells AI crawlers
On 27 October 2026, etsy.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot; it also published llms.txt.
Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.
- Panel
- top 1k (Tranco)
- robots.txt
- ok (HTTP 200) · https://www.etsy.com/robots.txt
unchanged since 2026-10-27 · sha2566ebe2465427dc327…· view as recorded - The * group
- allows applies to every agent without its own group
Per AI bot, 2026-10-27
| User agent | Verdict | How |
|---|---|---|
| GPTBot | allows | through the * group |
| OAI-SearchBot | allows | through the * group |
| ChatGPT-User | allows | through the * group |
| ClaudeBot | allows | through the * group |
| Claude-SearchBot | allows | through the * group |
| Claude-User | allows | through the * group |
| PerplexityBot | allows | through the * group |
| Perplexity-User | allows | through the * group |
| Googlebot | allows | through the * group |
| Google-Extended | allows | through the * group |
| Applebot-Extended | allows | through the * group |
| CCBot | allows | through the * group |
| Bytespider | allows | through the * group |
| Meta-ExternalAgent | allows | through the * group |
| Amazonbot | allows | through the * group |
Other signals
| Publishes /llms.txt | yes |
| Publishes /ai.txt | no |
| Publishes /.well-known/tdmrep.json | no |
| Reserves TDM rights (TDMRep file, header or meta) | not observed |
| Declares Content Signals (robots.txt or header) | not observed |
| Says ai-train=no in a Content Signal | not observed |
| Points to an RSL licence (robots.txt License line or Link header) | not observed |
| Answers with an x402 payment offer (lower bound: only the homepage is probed) | not observed |
| Answers HTTP 402 Payment Required (lower bound: only the homepage is probed) | not observed |
| Sends noai / noimageai directives | not observed |
| Refused our fetcher (403 / challenge) | no |
llms.txt: "Etsy: official reference for AI assistants", 26,713 bytes, 76 links.
robots.txt as recorded sha256 a950913bfb87553f…
User-agent: Spinn3r Disallow: / User-agent: * Disallow: /api/ Disallow: /uk/api/ Disallow: /au/api/ Disallow: /ca/api/ Disallow: /dk-en/api/ Disallow: /hk-en/api/ Disallow: /ie/api/ Disallow: /il-en/api/ Disallow: /in-en/api/ Disallow: /no-en/api/ Disallow: /nz/api/ Disallow: /se-en/api/ Disallow: /sg-en/api/ Disallow: /fr/api/ Disallow: /ca-fr/api/ Disallow: /de/api/ Disallow: /nl/api/ Disallow: /es/api/ Disallow: /mx/api/ Disallow: /it/api/ Disallow: /jp/api/ Disallow: /pl/api/ Disallow: /your/ Disallow: /uk/your/ Disallow: /au/your/ Disallow: /ca/your/ Disallow: /dk-en/your/ Disallow: /hk-en/your/ Disallow: /ie/your/ Disallow: /il-en/your/ Disallow: /in-en/your/ Disallow: /no-en/your/ Disallow: /nz/your/ Disallow: /se-en/your/ Disallow: /sg-en/your/ Disallow: /fr/your/ Disallow: /ca-fr/your/ Disallow: /de/your/ Disallow: /nl/your/ Disallow: /es/your/ Disallow: /mx/your/ Disallow: /it/your/ Disallow: /jp/your/ Disallow: /pl/your/ Disallow: /transactions/ Disallow: /uk/transactions/ Disallow: /au/transactions/ Disallow: /ca/transactions/ Disallow: /dk-en/transactions/ Disallow: /hk-en/transactions/ Disallow: /ie/transactions/ Disallow: /il-en/transactions/ Disallow: /in-en/transactions/ Disallow: /no-en/transactions/ Disallow: /nz/transactions/ Disallow: /se-en/transactions/ Disallow: /sg-en/transactions/ Disallow: /fr/transactions/ Disallow: /ca-fr/transactions/ Disallow: /de/transactions/ Disallow: /nl/transactions/ Disallow: /es/transactions/ Disallow: /mx/transactions/ Disallow: /it/transactions/ Disallow: /jp/transactions/ Disallow: /pl/transactions/ Disallow: /treasury Disallow: /uk/treasury Disallow: /au/treasury Disallow: /ca/treasury Disallow: /dk-en/treasury Disallow: /hk-en/treasury Disallow: /ie/treasury Disallow: /il-en/treasury Disallow: /in-en/treasury Disallow: /no-en/treasury Disallow: /nz/treasury Disallow: /se-en/treasury Disallow: /sg-en/treasury Disallow: /fr/treasury Disallow: /ca-fr/treasury Disallow: /de/treasury Disallow: /nl/treasury Disallow: /es/treasury Disallow: /mx/treasury Disallow: /it/treasury Disallow: /jp/treasury Disallow: /pl/treasury Disallow: /teams Disallow: /uk/teams Disallow: /au/teams Disallow: /ca/teams Disallow: /dk-en/teams Disallow: /hk-en/teams Disallow: /ie/teams Disallow: /il-en/teams Disallow: /in-en/teams Disallow: /no-en/teams Disallow: /nz/teams Disallow: /se-en/teams Disallow: /sg-en/teams Disallow: /fr/teams Disallow: /ca-fr/teams Disallow: /de/teams Disallow: /nl/teams Disallow: /es/teams Disallow: /mx/teams Disallow: /it/teams Disallow: /jp/teams Disallow: /pl/teams Disallow: /cart Disallow: /uk/cart Disallow: /au/cart Disallow: /ca/cart Disallow: /dk-en/cart Disallow: /hk-en/cart Disallow: /ie/cart Disallow: /il-en/cart Disallow: /in-en/cart Disallow: /no-en/cart Disallow: /nz/cart Disallow: /se-en/cart Disallow: /sg-en/cart Disallow: /fr/cart Disallow: /ca-fr/cart Disallow: /de/cart Disallow: /nl/cart Disallow: /es/cart Disallow: /mx/cart Disallow: /it/cart Disallow: /jp/cart Disallow: /pl/cart Disallow: /conversations Disallow: /uk/conversations Disallow: /au/conversations Disallow: /ca/conversations Disallow: /dk-en/conversations Disallow: /hk-en/conversations Disallow: /ie/conversations Disallow: /il-en/conversations Disallow: /in-en/conversations Disallow: /no-en/conversations Disallow: /nz/conversations Disallow: /se-en/conversations Disallow: /sg-en/conversations Disallow: /fr/conversations Disallow: /ca-fr/conversations Disallow: /de/conversations Disallow: /nl/conversations Disallow: /es/conversations Disallow: /mx/conversations Disallow: /it/conversations Disallow: /jp/conversations Disallow: /pl/conversations Disallow: /conversations/ Disallow: /uk/conversations/ Disallow: /au/conversations/ Disallow: /ca/conversations/ Disallow: /dk-en/conversations/ Disallow: /hk-en/conversations/ Disallow: /ie/conversations/ Disallow: /il-en/conversations/ Disallow: /in-en/conversations/ Disallow: /no-en/conversations/ Disallow: /nz/conversations/ Disallow: /se-en/conversations/ Disallow: /sg-en/conversations/ Disallow: /fr/conversations/ Disallow: /ca-fr/conversations/ Disallow: /de/conversations/ Disallow: /nl/conversations/ Disallow: /es/conversations/ Disallow: /mx/conversations/ Disallow: /it/conversations/ Disallow: /jp/conversations/ Disallow: /pl/conversations/ Disallow: /locale/overlay Disallow: /uk/locale/overlay Disallow: /au/locale/overlay Disallow: /ca/locale/overlay Disallow: /dk-en/locale/overlay Disallow: /hk-en/locale/overlay Disallow: /ie/locale/overlay Disallow: /il-en/locale/overlay Disallow: /in-en/locale/overlay Disallow: /no-en/locale/overlay Disallow: /nz/locale/overlay Disallow: /se-en/locale/overlay Disallow: /sg-en/locale/overlay Disallow: /fr/locale/overlay Disallow: /ca-fr/locale/overlay Disallow: /de/locale/overlay Disallow: /nl/locale/overlay Disallow: /es/locale/overlay Disallow: /mx/locale/overlay Disallow: /it/locale/overlay Disallow: /jp/locale/overlay Disallow: /pl/locale/overlay Disallow: /locale/overlay_secure Disallow: /uk/locale/overlay_secure Disallow: /au/locale/overlay_secure Disallow: /ca/locale/overlay_secure Disallow: /dk-en/locale/overlay_secure Disallow: /hk-en/locale/overlay_secure Disallow: /ie/locale/overlay_secure Disallow: /il-en/locale/overlay_secure Disallow: /in-en/locale/overlay_secure Disallow: /no-en/locale/overlay_secure Disallow: /nz/locale/overlay_secure Disallow: /se-en/locale/overlay_secure Disallow: /sg-en/locale/overlay_secure Disallow: /fr/locale/overlay_secure Disallow: /ca-fr/locale/overlay_secure Disallow: /de/locale/overlay_secure Disallow: /nl/locale/overlay_secure Disallow: /es/locale/overlay_secure Disallow: /mx/locale/overlay_secure Disallow: /it/locale/overlay_secure Disallow: /jp/locale/overlay_secure Disallow: /pl/locale/overlay_secure Disallow: /guest-pass/ Disallow: /uk/guest-pass/ Disallow: /au/guest-pass/ Disallow: /ca/guest-pass/ Disallow: /dk-en/guest-pass/ Disallow: /hk-en/guest-pass/ Disallow: /ie/guest-pass/ Disallow: /il-en/guest-pass/ Disallow: /in-en/guest-pass/ Disallow: /no-en/guest-pass/ Disallow: /nz/guest-pass/ Disallow: /se-en/guest-pass/ Disallow: /sg-en/guest-pass/ Disallow: /fr/guest-pass/ Disallow: /ca-fr/guest-pass/ Disallow: /de/guest-pass/ Disallow: /nl/guest-pass/ Disallow: /es/guest-pass/ Disallow: /mx/guest-pass/ Disallow: /it/guest-pass/ Disallow: /jp/guest-pass/ Disallow: /pl/guest-pass/ Disallow: /mailinglist/ Disallow: /uk/mailinglist/ Disallow: /au/mailinglist/ Disallow: /ca/mailinglist/ Disallow: /dk-en/mailinglist/ Disallow: /hk-en/mailinglist/ Disallow: /ie/mailinglist/ Disallow: /il-en/mailinglist/ Disallow: /in-en/mailinglist/ Disallow: /no-en/mailinglist/ Disallow: /nz/mailinglist/ Disallow: /se-en/mailinglist/ Disallow: /sg-en/mailinglist/ Disallow: /fr/mailinglist/ Disallow: /ca-fr/mailinglist/ Disallow: /de/mailinglist/ Disallow: /nl/mailinglist/ Disallow: /es/mailinglist/ Disallow: /mx/mailinglist/ Disallow: /it/mailinglist/ Disallow: /jp/mailinglist/ Disallow: /pl/mailinglist/ Disallow: /set_mobile_cookie.php Disallow: /uk/set_mobile_cookie.php Disallow: /au/set_mobile_cookie.php Disallow: /ca/set_mobile_cookie.php Disallow: /dk-en/set_mobile_cookie.php Disallow: /hk-en/set_mobile_cookie.php Disallow: /ie/set_mobile_cookie.php Disallow: /il-en/set_mobile_cookie.php Disallow: /in-en/set_mobile_cookie.php Disallow: /no-en/set_mobile_cookie.php Disallow: /nz/set_mobile_cookie.php Disallow: /se-en/set_mobile_cookie.php Disallow: /sg-en/set_mobile_cookie.php Disallow: /fr/set_mobile_cookie.php Disallow: /ca-fr/set_mobile_cookie.php Disallow: /de/set_mobile_cookie.php Disallow: /nl/set_mobile_cookie.php Disallow: /es/set_mobile_cookie.php Disallow: /mx/set_mobile_cookie.php Disallow: /it/set_mobile_cookie.php Disallow: /jp/set_mobile_cookie.php Disallow: /pl/set_mobile_cookie.php Disallow: /mini.php Disallow: /uk/mini.php Disallow: /au/mini.php Disallow: /ca/mini.php Disallow: /dk-en/mini.php Disallow: /hk-en/mini.php Disallow: /ie/mini.php Disallow: /il-en/mini.php Disallow: /in-en/mini.php Disallow: /no-en/mini.php Disallow: /nz/mini.php Disallow: /se-en/mini.php Disallow: /sg-en/mini.php Disallow: /fr/mini.php Disallow: /ca-fr/mini.php Disallow: /de/mini.php Disallow: /nl/mini.php Disallow: /es/mini.php Disallow: /mx/mini.php Disallow: /it/mini.php Disallow: /jp/mini.php Disallow: /pl/mini.php Disallow: /signin Disallow: /uk/signin Disallow: /au/signin Disallow: /ca/signin Disallow: /dk-en/signin Disallow: /hk-en/signin Disallow: /ie/signin Disallow: /il-en/signin Disallow: /in-en/signin Disallow: /no-en/signin Disallow: /nz/signin Disallow: /se-en/signin Disallow: /sg-en/signin Disallow: /fr/signin Disallow: /ca-fr/signin Disallow: /de/signin Disallow: /nl/signin Disallow: /es/signin Disallow: /mx/signin Disallow: /it/signin Disallow: /jp/signin Disallow: /pl/signin Disallow: /invites/buy Disallow: /uk/invites/buy Disallow: /au/invites/buy Disallow: /ca/invites/buy Disallow: /dk-en/invites/buy Disallow: /hk-en/invites/buy Disallow: /ie/invites/buy Disallow: /il-en/invites/buy Disallow: /in-en/invites/buy Disallow: /no-en/invites/buy Disallow: /nz/invites/buy Disallow: /se-en/invites/buy Disallow: /sg-en/invites/buy Disallow: /fr/invites/buy Disallow: /ca-fr/invites/buy Disallow: /de/invites/buy Disallow: /nl/invites/buy Disallow: /es/invites/buy Disallow: /mx/invites/buy Disallow: /it/invites/buy Disallow: /jp/invites/buy Disallow: /pl/invites/buy Disallow: /share Disallow: /uk/share Disallow: /au/share Disallow: /ca/share Disallow: /dk-en/share Disallow: /hk-en/share Disallow: /ie/share Disallow: /il-en/share Disallow: /in-en/share Disallow: /no-en/share Disallow: /nz/share Disallow: /se-en/share Disallow: /sg-en/share Disallow: /fr/share Disallow: /ca-fr/share Disallow: /de/share Disallow: /nl/share Disallow: /es/share Disallow: /mx/share Disallow: /it/share Disallow: /jp/share Disallow: /pl/share Disallow: /bcn Disallow: /uk/bcn Disallow: /au/bcn Disallow: /ca/bcn Disallow: /dk-en/bcn Disallow: /hk-en/bcn Disallow: /ie/bcn Disallow: /il-en/bcn Disallow: /in-en/bcn Disallow: /no-en/bcn Disallow: /nz/bcn Disallow: /se-en/bcn Disallow: /sg-en/bcn Disallow: /fr/bcn Disallow: /ca-fr/bcn Disallow: /de/bcn Disallow: /nl/bcn Disallow: /es/bcn Disallow: /mx/bcn Disallow: /it/bcn Disallow: /jp/bcn Disallow: /pl/bcn Disallow: /listing/*/stubs/* Disallow: /uk/listing/*/stubs/* Disallow: /au/listing/*/stubs/* Disallow: /ca/listing/*/stubs/* Disallow: /dk-en/listing/*/stubs/* Disallow: /hk-en/listing/*/stubs/* Disallow: /ie/listing/*/stubs/* Disallow: /il-en/listing/*/stubs/* Disallow: /in-en/listing/*/stubs/* Disallow: /no-en/listing/*/stubs/* Disallow: /nz/listing/*/stubs/* Disallow: /se-en/listing/*/stubs/* Disallow: /sg-en/listing/*/stubs/* Disallow: /fr/listing/*/stubs/* Disallow: /ca-fr/listing/*/stubs/* Disallow: /de/listing/*/stubs/* Disallow: /nl/listing/*/stubs/* Disallow: /es/listing/*/stubs/* Disallow: /mx/listing/*/stubs/* Disallow: /it/listing/*/stubs/* Disallow: /jp/listing/*/stubs/* Disallow: /pl/listing/*/stubs/* Disallow: /listing/*/similar?* Disallow: /uk/listing/*/similar?* Disallow: /au/listing/*/similar?* Disallow: /ca/listing/*/similar?* Disallow: /dk-en/listing/*/similar?* Disallow: /hk-en/listing/*/similar?* Disallow: /ie/listing/*/similar?* Disallow: /il-en/listing/*/similar?* Disallow: /in-en/listing/*/similar?* Disallow: /no-en/listing/*/similar?* Disallow: /nz/listing/*/similar?* Disallow: /se-en/listing/*/similar?* Disallow: /sg-en/listing/*/similar?* Disallow: /fr/listing/*/similar?* Disallow: /ca-fr/listing/*/similar?* Disallow: /de/listing/*/similar?* Disallow: /nl/listing/*/similar?* Disallow: /es/listing/*/similar?* Disallow: /mx/listing/*/similar?* Disallow: /it/listing/*/similar?* Disallow: /jp/listing/*/similar?* Disallow: /pl/listing/*/similar?* Disallow: /market/confederate_flag Disallow: /uk/market/confederate_flag Disallow: /au/market/confederate_flag Disallow: /ca/market/confederate_flag Disallow: /dk-en/market/confederate_flag Disallow: /hk-en/market/confederate_flag Disallow: /ie/market/confederate_flag Disallow: /il-en/market/confederate_flag Disallow: /in-en/market/confederate_flag Disallow: /no-en/market/confederate_flag Disallow: /nz/market/confederate_flag Disallow: /se-en/market/confederate_flag Disallow: /sg-en/market/confederate_flag Disallow: /fr/market/confederate_flag Disallow: /ca-fr/market/confederate_flag Disallow: /de/market/confederate_flag Disallow: /nl/market/confederate_flag Disallow: /es/market/confederate_flag Disallow: /mx/market/confederate_flag Disallow: /it/market/confederate_flag Disallow: /jp/market/confederate_flag Disallow: /pl/market/confederate_flag Disallow: /market/rebel_flag Disallow: /uk/market/rebel_flag Disallow: /au/market/rebel_flag Disallow: /ca/market/rebel_flag Disallow: /dk-en/market/rebel_flag Disallow: /hk-en/market/rebel_flag Disallow: /ie/market/rebel_flag Disallow: /il-en/market/rebel_flag Disallow: /in-en/market/rebel_flag Disallow: /no-en/market/rebel_flag Disallow: /nz/market/rebel_flag Disallow: /se-en/market/rebel_flag Disallow: /sg-en/market/rebel_flag Disallow: /fr/market/rebel_flag Disallow: /ca-fr/market/rebel_flag Disallow: /de/market/rebel_flag Disallow: /nl/market/rebel_flag Disallow: /es/market/rebel_flag Disallow: /mx/market/rebel_flag Disallow: /it/market/rebel_flag Disallow: /jp/market/rebel_flag Disallow: /pl/market/rebel_flag Disallow: /market/dixie_flag Disallow: /uk/market/dixie_flag Disallow: /au/market/dixie_flag Disallow: /ca/market/dixie_flag Disallow: /dk-en/market/dixie_flag Disallow: /hk-en/market/dixie_flag Disallow: /ie/market/dixie_flag Disallow: /il-en/market/dixie_flag Disallow: /in-en/market/dixie_flag Disallow: /no-en/market/dixie_flag Disallow: /nz/market/dixie_flag Disallow: /se-en/market/dixie_flag Disallow: /sg-en/market/dixie_flag Disallow: /fr/market/dixie_flag Disallow: /ca-fr/market/dixie_flag Disallow: /de/market/dixie_flag Disallow: /nl/market/dixie_flag Disallow: /es/market/dixie_flag Disallow: /mx/market/dixie_flag Disallow: /it/market/dixie_flag Disallow: /jp/market/dixie_flag Disallow: /pl/market/dixie_flag Disallow: /help/sitemap.php Disallow: /uk/help/sitemap.php Disallow: /au/help/sitemap.php Disallow: /ca/help/sitemap.php Disallow: /dk-en/help/sitemap.php Disallow: /hk-en/help/sitemap.php Disallow: /ie/help/sitemap.php Disallow: /il-en/help/sitemap.php Disallow: /in-en/help/sitemap.php Disallow: /no-en/help/sitemap.php Disallow: /nz/help/sitemap.php Disallow: /se-en/help/sitemap.php Disallow: /sg-en/help/sitemap.php Disallow: /fr/help/sitemap.php Disallow: /ca-fr/help/sitemap.php Disallow: /de/help/sitemap.php Disallow: /nl/help/sitemap.php Disallow: /es/help/sitemap.php Disallow: /mx/help/sitemap.php Disallow: /it/help/sitemap.php Disallow: /jp/help/sitemap.php Disallow: /pl/help/sitemap.php Disallow: /help/sitemap/* Disallow: /uk/help/sitemap/* Disallow: /au/help/sitemap/* Disallow: /ca/help/sitemap/* Disallow: /dk-en/help/sitemap/* Disallow: /hk-en/help/sitemap/* Disallow: /ie/help/sitemap/* Disallow: /il-en/help/sitemap/* Disallow: /in-en/help/sitemap/* Disallow: /no-en/help/sitemap/* Disallow: /nz/help/sitemap/* Disallow: /se-en/help/sitemap/* Disallow: /sg-en/help/sitemap/* Disallow: /fr/help/sitemap/* Disallow: /ca-fr/help/sitemap/* Disallow: /de/help/sitemap/* Disallow: /nl/help/sitemap/* Disallow: /es/help/sitemap/* Disallow: /mx/help/sitemap/* Disallow: /it/help/sitemap/* Disallow: /jp/help/sitemap/* Disallow: /pl/help/sitemap/* Disallow: /*?order=* Disallow: /uk/*?order=* Disallow: /au/*?order=* Disallow: /ca/*?order=* Disallow: /dk-en/*?order=* Disallow: /hk-en/*?order=* Disallow: /ie/*?order=* Disallow: /il-en/*?order=* Disallow: /in-en/*?order=* Disallow: /no-en/*?order=* Disallow: /nz/*?order=* Disallow: /se-en/*?order=* Disallow: /sg-en/*?order=* Disallow: /fr/*?order=* Disallow: /ca-fr/*?order=* Disallow: /de/*?order=* Disallow: /nl/*?order=* Disallow: /es/*?order=* Disallow: /mx/*?order=* Disallow: /it/*?order=* Disallow: /jp/*?order=* Disallow: /pl/*?order=* Disallow: /*handmade/ Disallow: /uk/*handmade/ Disallow: /au/*handmade/ Disallow: /ca/*handmade/ Disallow: /dk-en/*handmade/ Disallow: /hk-en/*handmade/ Disallow: /ie/*handmade/ Disallow: /il-en/*handmade/ Disallow: /in-en/*handmade/ Disallow: /no-en/*handmade/ Disallow: /nz/*handmade/ Disallow: /se-en/*handmade/ Disallow: /sg-en/*handmade/ Disallow: /fr/*handmade/ Disallow: /ca-fr/*handmade/ Disallow: /de/*handmade/ Disallow: /nl/*handmade/ Disallow: /es/*handmade/ Disallow: /mx/*handmade/ Disallow: /it/*handmade/ Disallow: /jp/*handmade/ Disallow: /pl/*handmade/ Disallow: /*color=* Disallow: /uk/*color=* Disallow: /au/*color=* Disallow: /ca/*color=* Disallow: /dk-en/*color=* Disallow: /hk-en/*color=* Disallow: /ie/*color=* Disallow: /il-en/*color=* Disallow: /in-en/*color=* Disallow: /no-en/*color=* Disallow: /nz/*color=* Disallow: /se-en/*color=* Disallow: /sg-en/*color=* Disallow: /fr/*color=* Disallow: /ca-fr/*color=* Disallow: /de/*color=* Disallow: /nl/*color=* Disallow: /es/*color=* Disallow: /mx/*color=* Disallow: /it/*color=* Disallow: /jp/*color=* Disallow: /pl/*color=* Disallow: /*my_etsy=* Disallow: /uk/*my_etsy=* Disallow: /au/*my_etsy=* Disallow: /ca/*my_etsy=* Disallow: /dk-en/*my_etsy=* Disallow: /hk-en/*my_etsy=* Disallow: /ie/*my_etsy=* Disallow: /il-en/*my_etsy=* Disallow: /in-en/*my_etsy=* Disallow: /no-en/*my_etsy=* Disallow: /nz/*my_etsy=* Disallow: /se-en/*my_etsy=* Disallow: /sg-en/*my_etsy=* Disallow: /fr/*my_etsy=* Disallow: /ca-fr/*my_etsy=* Disallow: /de/*my_etsy=* Disallow: /nl/*my_etsy=* Disallow: /es/*my_etsy=* Disallow: /mx/*my_etsy=* Disallow: /it/*my_etsy=* Disallow: /jp/*my_etsy=* Disallow: /pl/*my_etsy=* Disallow: /*gift_card=* Disallow: /uk/*gift_card=* Disallow: /au/*gift_card=* Disallow: /ca/*gift_card=* Disallow: /dk-en/*gift_card=* Disallow: /hk-en/*gift_card=* Disallow: /ie/*gift_card=* Disallow: /il-en/*gift_card=* Disallow: /in-en/*gift_card=* Disallow: /no-en/*gift_card=* Disallow: /nz/*gift_card=* Disallow: /se-en/*gift_card=* Disallow: /sg-en/*gift_card=* Disallow: /fr/*gift_card=* Disallow: /ca-fr/*gift_card=* Disallow: /de/*gift_card=* Disallow: /nl/*gift_card=* Disallow: /es/*gift_card=* Disallow: /mx/*gift_card=* Disallow: /it/*gift_card=* Disallow: /jp/*gift_card=* Disallow: /pl/*gift_card=* Disallow: /*customizable=* Disallow: /uk/*customizable=* Disallow: /au/*customizable=* Disallow: /ca/*customizable=* Disallow: /dk-en/*customizable=* Disallow: /hk-en/*customizable=* Disallow: /ie/*customizable=* Disallow: /il-en/*customizable=* Disallow: /in-en/*customizable=* Disallow: /no-en/*customizable=* Disallow: /nz/*customizable=* Disallow: /se-en/*customizable=* Disallow: /sg-en/*customizable=* Disallow: /fr/*customizable=* Disallow: /ca-fr/*customizable=* Disallow: /de/*customizable=* Disallow: /nl/*customizable=* Disallow: /es/*customizable=* Disallow: /mx/*customizable=* Disallow: /it/*customizable=* Disallow: /jp/*customizable=* Disallow: /pl/*customizable=* Disallow: /*locationQuery=* Disallow: /uk/*locationQuery=* Disallow: /au/*locationQuery=* Disallow: /ca/*locationQuery=* Disallow: /dk-en/*locationQuery=* Disallow: /hk-en/*locationQuery=* Disallow: /ie/*locationQuery=* Disallow: /il-en/*locationQuery=* Disallow: /in-en/*locationQuery=* Disallow: /no-en/*locationQuery=* Disallow: /nz/*locationQuery=* Disallow: /se-en/*locationQuery=* Disallow: /sg-en/*locationQuery=* Disallow: /fr/*locationQuery=* Disallow: /ca-fr/*locationQuery=* Disallow: /de/*locationQuery=* Disallow: /nl/*locationQuery=* Disallow: /es/*locationQuery=* Disallow: /mx/*locationQuery=* Disallow: /it/*locationQuery=* Disallow: /jp/*locationQuery=* Disallow: /pl/*locationQuery=* Disallow: /*min=* Disallow: /uk/*min=* Disallow: /au/*min=* Disallow: /ca/*min=* Disallow: /dk-en/*min=* Disallow: /hk-en/*min=* Disallow: /ie/*min=* Disallow: /il-en/*min=* Disallow: /in-en/*min=* Disallow: /no-en/*min=* Disallow: /nz/*min=* Disallow: /se-en/*min=* Disallow: /sg-en/*min=* Disallow: /fr/*min=* Disallow: /ca-fr/*min=* Disallow: /de/*min=* Disallow: /nl/*min=* Disallow: /es/*min=* Disallow: /mx/*min=* Disallow: /it/*min=* Disallow: /jp/*min=* Disallow: /pl/*min=* Disallow: /*max=* Disallow: /uk/*max=* Disallow: /au/*max=* Disallow: /ca/*max=* Disallow: /dk-en/*max=* Disallow: /hk-en/*max=* Disallow: /ie/*max=* Disallow: /il-en/*max=* Disallow: /in-en/*max=* Disallow: /no-en/*max=* Disallow: /nz/*max=* Disallow: /se-en/*max=* Disallow: /sg-en/*max=* Disallow: /fr/*max=* Disallow: /ca-fr/*max=* Disallow: /de/*max=* Disallow: /nl/*max=* Disallow: /es/*max=* Disallow: /mx/*max=* Disallow: /it/*max=* Disallow: /jp/*max=* Disallow: /pl/*max=* Disallow: */people/*/circle* Disallow: /uk*/people/*/circle* Disallow: /au*/people/*/circle* Disallow: /ca*/people/*/circle* Disallow: /dk-en*/people/*/circle* Disallow: /hk-en*/people/*/circle* Disallow: /ie*/people/*/circle* Disallow: /il-en*/people/*/circle* Disallow: /in-en*/people/*/circle* Disallow: /no-en*/people/*/circle* Disallow: /nz*/people/*/circle* Disallow: /se-en*/people/*/circle* Disallow: /sg-en*/people/*/circle* Disallow: /fr*/people/*/circle* Disallow: /ca-fr*/people/*/circle* Disallow: /de*/people/*/circle* Disallow: /nl*/people/*/circle* Disallow: /es*/people/*/circle* Disallow: /mx*/people/*/circle* Disallow: /it*/people/*/circle* Disallow: /jp*/people/*/circle* Disallow: /pl*/people/*/circle* Disallow: */people/*/favorites* Disallow: /uk*/people/*/favorites* Disallow: /au*/people/*/favorites* Disallow: /ca*/people/*/favorites* Disallow: /dk-en*/people/*/favorites* Disallow: /hk-en*/people/*/favorites* Disallow: /ie*/people/*/favorites* Disallow: /il-en*/people/*/favorites* Disallow: /in-en*/people/*/favorites* Disallow: /no-en*/people/*/favorites* Disallow: /nz*/people/*/favorites* Disallow: /se-en*/people/*/favorites* Disallow: /sg-en*/people/*/favorites* Disallow: /fr*/people/*/favorites* Disallow: /ca-fr*/people/*/favorites* Disallow: /de*/people/*/favorites* Disallow: /nl*/people/*/favorites* Disallow: /es*/people/*/favorites* Disallow: /mx*/people/*/favorites* Disallow: /it*/people/*/favorites* Disallow: /jp*/people/*/favorites* Disallow: /pl*/people/*/favorites* Disallow: */shop/*/sold* Disallow: /uk*/shop/*/sold* Disallow: /au*/shop/*/sold* Disallow: /ca*/shop/*/sold* Disallow: /dk-en*/shop/*/sold* Disallow: /hk-en*/shop/*/sold* Disallow: /ie*/shop/*/sold* Disallow: /il-en*/shop/*/sold* Disallow: /in-en*/shop/*/sold* Disallow: /no-en*/shop/*/sold* Disallow: /nz*/shop/*/sold* Disallow: /se-en*/shop/*/sold* Disallow: /sg-en*/shop/*/sold* Disallow: /fr*/shop/*/sold* Disallow: /ca-fr*/shop/*/sold* Disallow: /de*/shop/*/sold* Disallow: /nl*/shop/*/sold* Disallow: /es*/shop/*/sold* Disallow: /mx*/shop/*/sold* Disallow: /it*/shop/*/sold* Disallow: /jp*/shop/*/sold* Disallow: /pl*/shop/*/sold* Disallow: */shop/*/favoriters* Disallow: /uk*/shop/*/favoriters* Disallow: /au*/shop/*/favoriters* Disallow: /ca*/shop/*/favoriters* Disallow: /dk-en*/shop/*/favoriters* Disallow: /hk-en*/shop/*/favoriters* Disallow: /ie*/shop/*/favoriters* Disallow: /il-en*/shop/*/favoriters* Disallow: /in-en*/shop/*/favoriters* Disallow: /no-en*/shop/*/favoriters* Disallow: /nz*/shop/*/favoriters* Disallow: /se-en*/shop/*/favoriters* Disallow: /sg-en*/shop/*/favoriters* Disallow: /fr*/shop/*/favoriters* Disallow: /ca-fr*/shop/*/favoriters* Disallow: /de*/shop/*/favoriters* Disallow: /nl*/shop/*/favoriters* Disallow: /es*/shop/*/favoriters* Disallow: /mx*/shop/*/favoriters* Disallow: /it*/shop/*/favoriters* Disallow: /jp*/shop/*/favoriters* Disallow: /pl*/shop/*/favoriters* Disallow: /search*attr_*=* Disallow: /uk/search*attr_*=* Disallow: /au/search*attr_*=* Disallow: /ca/search*attr_*=* Disallow: /dk-en/search*attr_*=* Disallow: /hk-en/search*attr_*=* Disallow: /ie/search*attr_*=* Disallow: /il-en/search*attr_*=* Disallow: /in-en/search*attr_*=* Disallow: /no-en/search*attr_*=* Disallow: /nz/search*attr_*=* Disallow: /se-en/search*attr_*=* Disallow: /sg-en/search*attr_*=* Disallow: /fr/search*attr_*=* Disallow: /ca-fr/search*attr_*=* Disallow: /de/search*attr_*=* Disallow: /nl/search*attr_*=* Disallow: /es/search*attr_*=* Disallow: /mx/search*attr_*=* Disallow: /it/search*attr_*=* Disallow: /jp/search*attr_*=* Disallow: /pl/search*attr_*=* Disallow: /c/*attr_*=* Disallow: /uk/c/*attr_*=* Disallow: /au/c/*attr_*=* Disallow: /ca/c/*attr_*=* Disallow: /dk-en/c/*attr_*=* Disallow: /hk-en/c/*attr_*=* Disallow: /ie/c/*attr_*=* Disallow: /il-en/c/*attr_*=* Disallow: /in-en/c/*attr_*=* Disallow: /no-en/c/*attr_*=* Disallow: /nz/c/*attr_*=* Disallow: /se-en/c/*attr_*=* Disallow: /sg-en/c/*attr_*=* Disallow: /fr/c/*attr_*=* Disallow: /ca-fr/c/*attr_*=* Disallow: /de/c/*attr_*=* Disallow: /nl/c/*attr_*=* Disallow: /es/c/*attr_*=* Disallow: /mx/c/*attr_*=* Disallow: /it/c/*attr_*=* Disallow: /jp/c/*attr_*=* Disallow: /pl/c/*attr_*=* Disallow: /search*price_bucket=* Disallow: /uk/search*price_bucket=* Disallow: /au/search*price_bucket=* Disallow: /ca/search*price_bucket=* Disallow: /dk-en/search*price_bucket=* Disallow: /hk-en/search*price_bucket=* Disallow: /ie/search*price_bucket=* Disallow: /il-en/search*price_bucket=* Disallow: /in-en/search*price_bucket=* Disallow: /no-en/search*price_bucket=* Disallow: /nz/search*price_bucket=* Disallow: /se-en/search*price_bucket=* Disallow: /sg-en/search*price_bucket=* Disallow: /fr/search*price_bucket=* Disallow: /ca-fr/search*price_bucket=* Disallow: /de/search*price_bucket=* Disallow: /nl/search*price_bucket=* Disallow: /es/search*price_bucket=* Disallow: /mx/search*price_bucket=* Disallow: /it/search*price_bucket=* Disallow: /jp/search*price_bucket=* Disallow: /pl/search*price_bucket=* Disallow: /c/*price_bucket=* Disallow: /uk/c/*price_bucket=* Disallow: /au/c/*price_bucket=* Disallow: /ca/c/*price_bucket=* Disallow: /dk-en/c/*price_bucket=* Disallow: /hk-en/c/*price_bucket=* Disallow: /ie/c/*price_bucket=* Disallow: /il-en/c/*price_bucket=* Disallow: /in-en/c/*price_bucket=* Disallow: /no-en/c/*price_bucket=* Disallow: /nz/c/*price_bucket=* Disallow: /se-en/c/*price_bucket=* Disallow: /sg-en/c/*price_bucket=* Disallow: /fr/c/*price_bucket=* Disallow: /ca-fr/c/*price_bucket=* Disallow: /de/c/*price_bucket=* Disallow: /nl/c/*price_bucket=* Disallow: /es/c/*price_bucket=* Disallow: /mx/c/*price_bucket=* Disallow: /it/c/*price_bucket=* Disallow: /jp/c/*price_bucket=* Disallow: /pl/c/*price_bucket=* Disallow: /search*ship_to=* Disallow: /uk/search*ship_to=* Disallow: /au/search*ship_to=* Disallow: /ca/search*ship_to=* Disallow: /dk-en/search*ship_to=* Disallow: /hk-en/search*ship_to=* Disallow: /ie/search*ship_to=* Disallow: /il-en/search*ship_to=* Disallow: /in-en/search*ship_to=* Disallow: /no-en/search*ship_to=* Disallow: /nz/search*ship_to=* Disallow: /se-en/search*ship_to=* Disallow: /sg-en/search*ship_to=* Disallow: /fr/search*ship_to=* Disallow: /ca-fr/search*ship_to=* Disallow: /de/search*ship_to=* Disallow: /nl/search*ship_to=* Disallow: /es/search*ship_to=* Disallow: /mx/search*ship_to=* Disallow: /it/search*ship_to=* Disallow: /jp/search*ship_to=* Disallow: /pl/search*ship_to=* Disallow: /c/*ship_to=* Disallow: /uk/c/*ship_to=* Disallow: /au/c/*ship_to=* Disallow: /ca/c/*ship_to=* Disallow: /dk-en/c/*ship_to=* Disallow: /hk-en/c/*ship_to=* Disallow: /ie/c/*ship_to=* Disallow: /il-en/c/*ship_to=* Disallow: /in-en/c/*ship_to=* Disallow: /no-en/c/*ship_to=* Disallow: /nz/c/*ship_to=* Disallow: /se-en/c/*ship_to=* Disallow: /sg-en/c/*ship_to=* Disallow: /fr/c/*ship_to=* Disallow: /ca-fr/c/*ship_to=* Disallow: /de/c/*ship_to=* Disallow: /nl/c/*ship_to=* Disallow: /es/c/*ship_to=* Disallow: /mx/c/*ship_to=* Disallow: /it/c/*ship_to=* Disallow: /jp/c/*ship_to=* Disallow: /pl/c/*ship_to=* Disallow: /search*search_type=* Disallow: /uk/search*search_type=* Disallow: /au/search*search_type=* Disallow: /ca/search*search_type=* Disallow: /dk-en/search*search_type=* Disallow: /hk-en/search*search_type=* Disallow: /ie/search*search_type=* Disallow: /il-en/search*search_type=* Disallow: /in-en/search*search_type=* Disallow: /no-en/search*search_type=* Disallow: /nz/search*search_type=* Disallow: /se-en/search*search_type=* Disallow: /sg-en/search*search_type=* Disallow: /fr/search*search_type=* Disallow: /ca-fr/search*search_type=* Disallow: /de/search*search_type=* Disallow: /nl/search*search_type=* Disallow: /es/search*search_type=* Disallow: /mx/search*search_type=* Disallow: /it/search*search_type=* Disallow: /jp/search*search_type=* Disallow: /pl/search*search_type=* Disallow: /c/*search_type=* Disallow: /uk/c/*search_type=* Disallow: /au/c/*search_type=* Disallow: /ca/c/*search_type=* Disallow: /dk-en/c/*search_type=* Disallow: /hk-en/c/*search_type=* Disallow: /ie/c/*search_type=* Disallow: /il-en/c/*search_type=* Disallow: /in-en/c/*search_type=* Disallow: /no-en/c/*search_type=* Disallow: /nz/c/*search_type=* Disallow: /se-en/c/*search_type=* Disallow: /sg-en/c/*search_type=* Disallow: /fr/c/*search_type=* Disallow: /ca-fr/c/*search_type=* Disallow: /de/c/*search_type=* Disallow: /nl/c/*search_type=* Disallow: /es/c/*search_type=* Disallow: /mx/c/*search_type=* Disallow: /it/c/*search_type=* Disallow: /jp/c/*search_type=* Disallow: /pl/c/*search_type=* Disallow: /shop_name_search_service Disallow: /uk/shop_name_search_service Disallow: /au/shop_name_search_service Disallow: /ca/shop_name_search_service Disallow: /dk-en/shop_name_search_service Disallow: /hk-en/shop_name_search_service Disallow: /ie/shop_name_search_service Disallow: /il-en/shop_name_search_service Disallow: /in-en/shop_name_search_service Disallow: /no-en/shop_name_search_service Disallow: /nz/shop_name_search_service Disallow: /se-en/shop_name_search_service Disallow: /sg-en/shop_name_search_service Disallow: /fr/shop_name_search_service Disallow: /ca-fr/shop_name_search_service Disallow: /de/shop_name_search_service Disallow: /nl/shop_name_search_service Disallow: /es/shop_name_search_service Disallow: /mx/shop_name_search_service Disallow: /it/shop_name_search_service Disallow: /jp/shop_name_search_service Disallow: /pl/shop_name_search_service Allow: /mailinglist/email/ Allow: /uk/mailinglist/email/ Allow: /au/mailinglist/email/ Allow: /ca/mailinglist/email/ Allow: /dk-en/mailinglist/email/ Allow: /hk-en/mailinglist/email/ Allow: /ie/mailinglist/email/ Allow: /il-en/mailinglist/email/ Allow: /in-en/mailinglist/email/ Allow: /no-en/mailinglist/email/ Allow: /nz/mailinglist/email/ Allow: /se-en/mailinglist/email/ Allow: /sg-en/mailinglist/email/ Allow: /fr/mailinglist/email/ Allow: /ca-fr/mailinglist/email/ Allow: /de/mailinglist/email/ Allow: /nl/mailinglist/email/ Allow: /es/mailinglist/email/ Allow: /mx/mailinglist/email/ Allow: /it/mailinglist/email/ Allow: /jp/mailinglist/email/ Allow: /pl/mailinglist/email/ Sitemap: https://www.etsy.com/dynamic-sitemaps.xml?sitemap=promossitemap Sitemap: https://www.etsy.com/dynamic-sitemaps.xml?sitemap=taxonomyindex Sitemap: https://www.etsy.com/dynamic-sitemaps.xml?sitemap=shop_index_2 Sitemap: https://www.etsy.com/dynamic-sitemaps.xml?sitemap=marketindex Sitemap: https://www.etsy.com/dynamic-sitemaps.xml?sitemap=homesitemap Sitemap: https://www.etsy.com/dynamic-sitemaps.xml?sitemap=localsitemap Sitemap: https://www.etsy.com/dynamic-sitemaps.xml?sitemap=findssitemap Sitemap: https://www.etsy.com/listing_sitemaps/en/index.xml.gz # # What's all this then?# \ # # ----- # | . . | # ----- # \--|-|--/ # | | # |-------|
robots.txt versions
| From | Through | State | Version | |
|---|---|---|---|---|
| 2026-10-28 | 2026-10-28 | ok 200 | b31b7475fc59 | diff with previous |
| 2026-10-27 | 2026-10-27 | ok 200 | 6ebe2465427d | diff with previous |
| 2026-10-26 | 2026-10-26 | ok 200 | f03f4872b9f7 |
Timeline of changes
| Date | What changed |
|---|---|
| 2026-10-28 | robots.txt changed |
| 2026-10-27 | robots.txt changed |
Reconstructed history (Common Crawl)
From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.
| Crawl | Host | State | GPTBot | OAI-SearchBot | ChatGPT-User | ClaudeBot | Claude-SearchBot | Claude-User | PerplexityBot |
|---|---|---|---|---|---|---|---|---|---|
| 2026-09-21 CC-MAIN-2026-39 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2025-09-15 CC-MAIN-2025-38 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2024-09-16 CC-MAIN-2024-38 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2023-10-02 CC-MAIN-2023-40 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2022-10-03 CC-MAIN-2022-40 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2021-09-27 CC-MAIN-2021-39 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2020-09-28 CC-MAIN-2020-40 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2019-09-23 CC-MAIN-2019-39 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-09-24 CC-MAIN-2018-39 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-04-23 CC-MAIN-2018-17 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-01-29 CC-MAIN-2018-05 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-10-23 CC-MAIN-2017-43 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-09-25 CC-MAIN-2017-39 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-07-24 CC-MAIN-2017-30 | www.etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-04-24 CC-MAIN-2017-17 | etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-01-23 CC-MAIN-2017-04 | etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-10-03 CC-MAIN-2016-40 | etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-09-05 CC-MAIN-2016-36 | etsy.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
Evidence
Every daily observation of etsy.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).
API: https://infrarabbit.com/crawlers/api/v1/policy/etsy.com