disneyplus.com: what it tells AI crawlers

On 27 October 2026, disneyplus.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel
top 1k (Tranco)
robots.txt
ok (HTTP 200) · https://www.disneyplus.com/robots.txt
unchanged since 2026-10-26 · sha256 ea1fedfeaf8ce8df… · view as recorded
The * group
blocks applies to every agent without its own group

Per AI bot, 2026-10-27

User agentVerdictHow
GPTBotallowsnamed in its own group
OAI-SearchBotallowsnamed in its own group
ChatGPT-Userallowsnamed in its own group
ClaudeBotallowsnamed in its own group
Claude-SearchBotallowsnamed in its own group
Claude-Userallowsnamed in its own group
PerplexityBotallowsnamed in its own group
Perplexity-Userblocksthrough the * group
Googlebotallowsnamed in its own group
Google-Extendedallowsnamed in its own group
Applebot-Extendedallowsnamed in its own group
CCBotallowsnamed in its own group
Bytespiderallowsnamed in its own group
Meta-ExternalAgentallowsnamed in its own group
Amazonbotallowsnamed in its own group
Applebotallowsnamed in its own group
DeepSeekBotallowsnamed in its own group
DuckAssistBotallowsnamed in its own group
facebookexternalhitallowsnamed in its own group
Google-Agentallowsnamed in its own group
GrokBotallowsnamed in its own group
meta-externalfetcherallowsnamed in its own group
PetalBotallowsnamed in its own group
YouBotallowsnamed in its own group

Other signals

Publishes /llms.txtnot observed
Publishes /ai.txtnot observed
Publishes /.well-known/tdmrep.jsonnot observed
Reserves TDM rights (TDMRep file, header or meta)not observed
Declares Content Signals (robots.txt or header)not observed
Says ai-train=no in a Content Signalnot observed
Points to an RSL licence (robots.txt License line or Link header)not observed
Answers with an x402 payment offer (lower bound: only the homepage is probed)not observed
Answers HTTP 402 Payment Required (lower bound: only the homepage is probed)not observed
Sends noai / noimageai directivesnot observed
Refused our fetcher (403 / challenge)no

robots.txt as recorded sha256 ea1fedfeaf8ce8df…

#robots.txt for www.disneyplus.com/

User-agent: *
Disallow: /

# SEO Bots
User-agent: Googlebot
User-agent: Googlebot-Video
User-agent: Googlebot-Image
User-agent: Googlebot-News
User-agent: Storebot-Google
User-agent: Google-InspectionTool
User-agent: Applebot
User-agent: Bingbot
User-agent: BingPreview
User-agent: Slurp
User-agent: DuckDuckBot
User-agent: Baiduspider
User-agent: Baiduspider-mobile
User-agent: Baiduspider-video
User-agent: Baiduspider-image
User-agent: NaverBot
User-agent: Y!J-WSC
User-agent: YJ-WSC
User-agent: Yeti
User-agent: Yandex
User-agent: YandexBot
User-agent: YandexMobileBot
User-agent: YandexVideo
User-agent: YandexWebmaster
User-agent: YandexSitelinks
User-agent: SeznamBot
User-agent: PetalBot
User-agent: Sogou
User-agent: Qwantify
User-agent: Ecosia
User-agent: 360Spider
User-agent: Coc Coc
User-agent: Qwantbot
# SEO Tools
User-agent: Botify
User-agent: SemrushBot
User-agent: Screaming Frog SEO Spider
User-agent: DataForSeoBot
User-agent: ia_archiver
# Social
User-agent: Facebot
User-agent: Twitterbot
User-agent: facebookexternalhit
User-agent: instagram
User-agent: Threads
User-agent: TikTok
User-agent: Pinterest
User-agent: Pinterestbot
User-agent: redditbot
User-agent: WhatsApp
User-agent: Snapchat
User-agent: TelegramBot
User-agent: Viber
User-agent: KakaoTalk
User-agent: Line
# Ad Bots
User-agent: AdsBot-Google
User-agent: AdsBot-Google-Mobile
User-agent: AdsBot-Google-Mobile-Apps
User-agent: Mediapartners-Google
User-agent: Google-Ads-Creatives-Assistant
User-agent: FacebookAdPreview
User-agent: meta-externalads
User-agent: Twitterbot
User-agent: Adidxbot
User-agent: Bytespider
User-agent: IAS-Crawler
User-agent: DoubleVerifyBot
User-agent: Moatbot
User-agent: TheTradeDesk
User-agent: TTDCrawler
User-agent: CriteoBot
# AI Bots
User-agent: GPTBot
User-agent: ChatGPT-User
User-agent: OAI-SearchBot
User-agent: Google-Extended
User-agent: ClaudeBot
User-agent: Claude-User
User-agent: Claude-SearchBot
User-agent: PerplexityBot
User-agent: Applebot-Extended
User-agent: Meta-ExternalAgent
User-agent: Meta-ExternalFetcher
User-agent: Amazonbot
User-agent: CCBot
User-agent: Google-Agent
User-agent: GrokBot
User-agent: YouBot
User-agent: DuckAssistBot
User-agent: DeepSeekBot
Allow: /

# Auth Account Related Pages
Disallow: /home
Disallow: /*/home
Disallow: /episode
Disallow: /*/episode
Disallow: /watch
Disallow: /*/watch
Disallow: /billing
Disallow: /*/billing
Disallow: /search
Disallow: /*/search
Disallow: /account
Disallow: /*/account
Disallow: /app-settings
Disallow: /*/app-settings
Disallow: /enter-password
Disallow: /*/enter-password
Disallow: /create-password
Disallow: /*/create-password
Disallow: /welcome/responder

# Announce Sitemap
Sitemap: https://www.disneyplus.com/sitemap.xml

# Source: static-root-proxy

robots.txt versions

FromThroughStateVersion
2026-10-262026-10-28ok 200ea1fedfeaf8c

Timeline of changes

DateWhat changed
No changes since the first observation (2026-10-26).

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

CrawlHostStateGPTBotOAI-SearchBotChatGPT-UserClaudeBotClaude-SearchBotClaude-UserPerplexityBot
2026-09-21 CC-MAIN-2026-39www.disneyplus.comok viewallowsallowsunknownallowsallowsunknownallows
2025-09-15 CC-MAIN-2025-38www.disneyplus.comok viewblocksallowsunknownallowsallowsunknownallows
2024-09-16 CC-MAIN-2024-38www.disneyplus.comok viewblocksallowsunknownallowsallowsunknownallows
2023-10-02 CC-MAIN-2023-40www.disneyplus.comok viewblocksallowsunknownallowsallowsunknownallows
2022-10-03 CC-MAIN-2022-40www.disneyplus.comok viewallowsallowsunknownallowsallowsunknownallows
2021-09-27 CC-MAIN-2021-39www.disneyplus.comok viewallowsallowsunknownallowsallowsunknownallows
2020-09-28 CC-MAIN-2020-40www.disneyplus.comok viewallowsallowsunknownallowsallowsunknownallows
2019-09-23 CC-MAIN-2019-39disneyplus.comredirect-onlyunknownunknownunknownunknownunknownunknownunknown

Evidence

Every daily observation of disneyplus.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/disneyplus.com