flickr.com: what it tells AI crawlers

On 27 October 2026, flickr.com's robots.txt fully blocked GPTBot, ClaudeBot, Google-Extended and CCBot and partly blocked OAI-SearchBot, Claude-SearchBot and PerplexityBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://flickr.com/robots.txt
unchanged since 2026-10-26 · sha256 93b2b4f0def4c7f2… · view as recorded

The * group

blocks applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 93b2b4f0def4c7f2…

User-agent: Adsbot-Google
User-agent: Alexabot
User-agent: AppleBot
User-agent: archive.org_bot
User-agent: Assaybot
User-agent: baiduspider
User-agent: bitlybot
User-agent: bingbot
User-agent: BingPreview
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: coccocbot
User-agent: Discordbot
User-agent: DuckDuckBot
User-agent: facebookexternalhit
User-agent: Google-Site-Verification
User-agent: Google-Sitemaps
User-agent: Googlebot
User-agent: Googlebot-Image
User-agent: Googlebot-Mobile
User-agent: Googlebot-News
User-agent: Googlebot-Video
User-agent: gsa-crawler
User-Agent: HatenaBlog-bot
User-agent: ia_archiver
User-agent: Mediapartners-Google
User-agent: msnbot
User-agent: NAVER
User-agent: OAI-SearchBot
User-agent: PerplexityBot
User-agent: PetalBot
User-agent: Pingdom
User-agent: Pinterest
User-agent: redditbot
User-agent: Slackbot
User-agent: Slurp
User-agent: snapchat
User-agent: Twitterbot
User-agent: wp.com
User-agent: Yandex
User-agent: YandexImages
User-agent: YandexVideoParser
User-agent: Yeti
User-agent: ZoomBot
Disallow: /gp/
Disallow: /report_abuse.gne
Disallow: /abuse
Disallow: /images/*
Disallow: /apps/*
Disallow: /tools/demos/*
Disallow: /search
Disallow: /services/oauth
Disallow: /groups/10millionphotos/
Disallow: /photos/youpy/
Disallow: /photos/i_love_u_get_away_from_me/
Disallow: /faves-i_love_u_get_away_from_me/
Disallow: /photos/gbachelie/
Disallow: /photos/archivesact/6011019532/nearby/
Disallow: /yss_fragment.gne
Disallow: /photos/tags/*/page*
Disallow: /photos/*/lightbox/
Disallow: /groups/*/lightbox/
Disallow: /explore/*/lightbox/

User-agent: magpie-crawler
Disallow: /

User-agent: *
Disallow: /

Sitemap: https://www.flickr.com/sitemap/index/users/sitemap-index-users-00000000.xml.gz
Sitemap: https://www.flickr.com/sitemap/index/tags/sitemap-index-tags-00000000.xml.gz
Sitemap: https://www.flickr.com/sitemap/index/sets/sitemap-index-sets-00000000.xml.gz
Sitemap: https://www.flickr.com/sitemap/index/photos/sitemap-index-photos-00000000.xml.gz
Sitemap: https://www.flickr.com/sitemap/index/groups/sitemap-index-groups-00000000.xml.gz
Sitemap: https://www.flickr.com/sitemap/cameras/sitemap-cameras-00000000.xml.gz

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of flickr.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/flickr.com