unsplash.com: what it tells AI crawlers

On 27 October 2026, unsplash.com's robots.txt fully blocked ClaudeBot and Claude-SearchBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://unsplash.com/robots.txt
unchanged since 2026-10-26 · sha256 25bb1899e23f96b9… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 5f11e6e01691bfd9…

User-Agent: *

Sitemap: https://sitemaps.unsplash.com/sitemaps/sitemap.xml

Allow: /blog

Allow: /de/backgrounds

Allow: /backgrounds

Allow: /es/backgrounds

Allow: /fr/backgrounds

Allow: /it/backgrounds

Allow: /ja/backgrounds

Allow: /ko/backgrounds

Allow: /pt-br/backgrounds

Disallow: /photos/*/download

Disallow: /illustrations/*/download

Disallow: /ngetty

Disallow: /napi

Disallow: /nautocomplete

Disallow: /nabc

Disallow: /de/@*
Disallow: /de/account
Disallow: /de/account/*
Disallow: /de/b
Disallow: /de/b/*
Disallow: /de/community
Disallow: /de/einstellung
Disallow: /de/einstellung/job-posts/*
Disallow: /de/kollektionen
Disallow: /de/kollektionen/*
Disallow: /de/login
Disallow: /de/plus/r/*
Disallow: /de/presse
Disallow: /de/registrieren
Disallow: /de/s/benutzer/*
Disallow: /de/s/kollektionen/*
Disallow: /de/s/visuell/*
Disallow: /de/unsere-geschichte
Disallow: /de/werben

Disallow: /account
Disallow: /account/*
Disallow: /b
Disallow: /b/*
Disallow: /join
Disallow: /login
Disallow: /plus/r/*
Disallow: /s/collections/*
Disallow: /s/users/*
Disallow: /s/visual/*

Disallow: /es/%C3%BAnete
Disallow: /es/@*
Disallow: /es/account
Disallow: /es/account/*
Disallow: /es/anunciarse
Disallow: /es/b
Disallow: /es/b/*
Disallow: /es/colecciones
Disallow: /es/colecciones/*
Disallow: /es/comunidad
Disallow: /es/contrataci%C3%B3n
Disallow: /es/contrataci%C3%B3n/job-posts/*
Disallow: /es/historia
Disallow: /es/inicia-sesi%C3%B3n
Disallow: /es/plus/r/*
Disallow: /es/prensa
Disallow: /es/s/colecciones/*
Disallow: /es/s/perfecta/*
Disallow: /es/s/usuarios/*

Disallow: /fr/@*
Disallow: /fr/account
Disallow: /fr/account/*
Disallow: /fr/b
Disallow: /fr/b/*
Disallow: /fr/collections
Disallow: /fr/collections/*
Disallow: /fr/communaut%C3%A9
Disallow: /fr/connexion
Disallow: /fr/faire-votre-publicit%C3%A9
Disallow: /fr/histoire
Disallow: /fr/newsroom
Disallow: /fr/plus/r/*
Disallow: /fr/recrutement
Disallow: /fr/recrutement/job-posts/*
Disallow: /fr/s%E2%80%99inscrire
Disallow: /fr/s/collections/*
Disallow: /fr/s/utilisateurs/*
Disallow: /fr/s/visuel/*

Disallow: /it/@*
Disallow: /it/accedi
Disallow: /it/account
Disallow: /it/account/*
Disallow: /it/assunzioni
Disallow: /it/assunzioni/job-posts/*
Disallow: /it/b
Disallow: /it/b/*
Disallow: /it/collezioni
Disallow: /it/collezioni/*
Disallow: /it/community
Disallow: /it/diventa-inserzionista
Disallow: /it/la-nostra-storia
Disallow: /it/plus/r/*
Disallow: /it/press-room
Disallow: /it/registrati
Disallow: /it/s/collezioni/*
Disallow: /it/s/immagini/*
Disallow: /it/s/utenti/*

Disallow: /ja/%E3%82%B3%E3%83%9F%E3%83%A5%E3%83%8B%E3%83%86%E3%82%A3
Disallow: /ja/%E3%82%B3%E3%83%AC%E3%82%AF%E3%82%B7%E3%83%A7%E3%83%B3
Disallow: /ja/%E3%82%B3%E3%83%AC%E3%82%AF%E3%82%B7%E3%83%A7%E3%83%B3/*
Disallow: /ja/%E3%82%B5%E3%82%A4%E3%83%B3%E3%82%A4%E3%83%B3
Disallow: /ja/%E3%83%97%E3%83%AC%E3%82%B9
Disallow: /ja/%E5%B1%A5%E6%AD%B4
Disallow: /ja/%E5%BA%83%E5%91%8A
Disallow: /ja/%E6%A1%88%E4%BB%B6%E4%BE%9D%E9%A0%BC%E4%B8%AD
Disallow: /ja/%E6%A1%88%E4%BB%B6%E4%BE%9D%E9%A0%BC%E4%B8%AD/job-posts/*
Disallow: /ja/%E7%99%BB%E9%8C%B2
Disallow: /ja/@*
Disallow: /ja/account
Disallow: /ja/account/*
Disallow: /ja/b
Disallow: /ja/b/*
Disallow: /ja/plus/r/*
Disallow: /ja/s/%E3%82%B3%E3%83%AC%E3%82%AF%E3%82%B7%E3%83%A7%E3%83%B3/*
Disallow: /ja/s/%E3%83%93%E3%82%B8%E3%83%A5%E3%82%A2%E3%83%AB/*
Disallow: /ja/s/%E3%83%A6%E3%83%BC%E3%82%B6%E3%83%BC/*

Disallow: /ko/%EA%B0%80%EC%9E%85
Disallow: /ko/%EA%B3%A0%EC%9A%A9-%EC%A4%91
Disallow: /ko/%EA%B3%A0%EC%9A%A9-%EC%A4%91/job-posts/*
Disallow: /ko/%EA%B4%91%EA%B3%A0
Disallow: /ko/%EB%A1%9C%EA%B7%B8%EC%9D%B8
Disallow: /ko/%EB%B3%B4%EB%8F%84%EC%9E%90%EB%A3%8C
Disallow: /ko/%EC%97%AD%EC%82%AC
Disallow: /ko/%EC%BB%A4%EB%AE%A4%EB%8B%88%ED%8B%B0
Disallow: /ko/%EC%BB%AC%EB%A0%89%EC%85%98
Disallow: /ko/%EC%BB%AC%EB%A0%89%EC%85%98/*
Disallow: /ko/%ED%94%8C%EB%9F%AC%EC%8A%A4/r/*
Disallow: /ko/@*
Disallow: /ko/account
Disallow: /ko/account/*
Disallow: /ko/b
Disallow: /ko/b/*
Disallow: /ko/s/%EC%82%AC%EC%9A%A9%EC%9E%90/*
Disallow: /ko/s/%EC%8B%9C%EA%B0%81-%EC%9E%90%EB%A3%8C/*
Disallow: /ko/s/%EC%BB%AC%EB%A0%89%EC%85%98/*

Disallow: /pt-br/@*
Disallow: /pt-br/account
Disallow: /pt-br/account/*
Disallow: /pt-br/anunciar
Disallow: /pt-br/b
Disallow: /pt-br/b/*
Disallow: /pt-br/cadastre-se
Disallow: /pt-br/cole%C3%A7%C3%B5es
Disallow: /pt-br/cole%C3%A7%C3%B5es/*
Disallow: /pt-br/comunidade
Disallow: /pt-br/contratando
Disallow: /pt-br/contratando/job-posts/*
Disallow: /pt-br/imprensa
Disallow: /pt-br/login
Disallow: /pt-br/nossa-hist%C3%B3ria
Disallow: /pt-br/plus/r/*
Disallow: /pt-br/s/cole%C3%A7%C3%B5es/*
Disallow: /pt-br/s/usu%C3%A1rios/*
Disallow: /pt-br/s/visual/*

Disallow: /en-xa
Disallow: /en-xa/*

User-Agent: ImagesiftBot

Disallow: /

User-Agent: ClaudeBot

Disallow: /

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of unsplash.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/unsplash.com