Points to an RSL licence (robots.txt License line or Link header)
no
Answers with an x402 payment offer (lower bound: only the homepage is probed)
no
Answers HTTP 402 Payment Required (lower bound: only the homepage is probed)
no
Sends noai / noimageai directives
no
Refused our fetcher (403 / challenge)
no
robots.txt as recorded sha256 f81b70b7b2dc6af8…
User-agent: *
Disallow: /zoeken
Disallow: /search
Disallow: /block/
Disallow: /component/
Disallow: /components/
User-agent: GPTBot
User-agent: ChatGPT-User
User-agent: CCBot
User-agent: anthropic-ai
User-agent: PerplexityBot
User-agent: Amazonbot
User-agent: Applebot-Extended
User-agent: Bytespider
User-agent: ClaudeBot
User-agent: Claude-Web
User-agent: cohere-ai
User-agent: Diffbot
User-agent: Google-Extended
User-agent: Meta-ExternalFetcher
User-agent: AhrefsBot
User-agent: SemrushBot
User-agent: MJ12bot
User-agent: Exabot
User-agent: proximic
User-agent: SemrushBot-OCOB
User-agent: SemrushBot-SWA
User-agent: Yandex
User-agent: YandexAdditional
User-agent: YandexAdditionalBot
User-agent: YandexBot
User-agent: YandexImages
Disallow: /
User-agent: Googlebot
Disallow: /*redirect=1*
User-agent: Googlebot-News
Disallow: /advertorial/
Disallow: /uit-andere-media/
Sitemap: https://www.nu.nl/sitemap_index.xml
Sitemap: https://www.nu.nl/sitemap_news.xml
Sitemap: https://www.nu.nl/sitemap_sections.xml
Sitemap: https://www.nu.nl/sitemap_general.xml
# Alle auteurs-, naburige en databankrechten die op de inhoud en opmaak van de DPG Media websites
# en DPG Media apps rusten, worden door DPG Media BV uitdrukkelijk voorbehouden. De inhoud van de
# DPG Media websites en apps is uitsluitend voor persoonlijk, niet-commercieel gebruik en het is
# niet toegestaan om gegevens van de website of uit de apps door middel van screen scraping
# (of een andere geautomatiseerde werkwijze) te vergaren.
# Zie ook de Gebruikersvoorwaarden van DPG Media B.V. op www.dpgmedia.nl Gebruiksvoorwaarden - DPG Media
# All copyrights, neighbouring rights and database rights in the content and layout of the
# DPG Media websites and DPG Media apps are explicitly reserved by DPG Media BV. The content of the DPG Media
# websites and DPG Media apps is for personal, non-commercial use only and it is not allowed to
# collect data from the website or from the apps by means of screen scraping (or any other
# automated method).
# See also the terms of use of DPG Media B.V. at www.dpgmedia.nl Gebruiksvoorwaarden - DPG Media
No changes since the first observation (2026-10-26).
Reconstructed history (Common Crawl)
From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.
Every daily observation of nu.nl is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).