lefigaro.fr: what it tells AI crawlers

On 27 October 2026, lefigaro.fr's robots.txt fully blocked GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended and CCBot; it also reserved TDM rights.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel
top 10k (Tranco)
robots.txt
ok (HTTP 200) · https://www.lefigaro.fr/robots.txt
unchanged since 2026-10-26 · sha256 f6127a8af3818c9e… · view as recorded
The * group
allows applies to every agent without its own group

Per AI bot, 2026-10-27

User agentVerdictHow
GPTBotblocksnamed in its own group
OAI-SearchBotblocksnamed in its own group
ChatGPT-Userblocksnamed in its own group
ClaudeBotblocksnamed in its own group
Claude-SearchBotblocksnamed in its own group
Claude-Userblocksnamed in its own group
PerplexityBotallowsthrough the * group
Perplexity-Userallowsthrough the * group
Googlebotallowsthrough the * group
Google-Extendedblocksnamed in its own group
Applebot-Extendedallowsthrough the * group
CCBotblocksnamed in its own group
Bytespiderallowsthrough the * group
Meta-ExternalAgentallowsthrough the * group
Amazonbotblocksnamed in its own group
anthropic-aiblocksnamed in its own group
ApifyBotblocksnamed in its own group
ApifyWebsiteContentCrawlerblocksnamed in its own group
Brightbotblocksnamed in its own group
Claude-Webblocksnamed in its own group
CloudVertexBotblocksnamed in its own group
cohere-aiblocksnamed in its own group
CragCrawlerblocksnamed in its own group
Diffbotblocksnamed in its own group
Diffbot-Userblocksnamed in its own group
ExaBotblocksnamed in its own group
ExaSearchBotblocksnamed in its own group
FirecrawlAgentblocksnamed in its own group
GeistHaus-PageFetcherblocksnamed in its own group
Gemini-Deep-Researchblocksnamed in its own group
Google-CloudVertexBotblocksnamed in its own group
Google-NotebookLMblocksnamed in its own group
GoogleAgent-Marinerblocksnamed in its own group
Kimi-SearchBotblocksnamed in its own group
Kimi-Userblocksnamed in its own group
KimiBotblocksnamed in its own group
MistralAI-Userblocksnamed in its own group
omgiliblocksnamed in its own group
omgilibotblocksnamed in its own group
Querit-SearchBotblocksnamed in its own group
QueritBotblocksnamed in its own group
Scrapyblocksnamed in its own group
ShapBotblocksnamed in its own group
TavilyBotblocksnamed in its own group
UseAIblocksnamed in its own group
Webzio-Extendedblocksnamed in its own group
YouBotblocksnamed in its own group

Other signals

Publishes /llms.txtno
Publishes /ai.txtno
Publishes /.well-known/tdmrep.jsonyes
Reserves TDM rights (TDMRep file, header or meta)yes
Declares Content Signals (robots.txt or header)no
Says ai-train=no in a Content Signalno
Points to an RSL licence (robots.txt License line or Link header)no
Answers with an x402 payment offer (lower bound: only the homepage is probed)no
Answers HTTP 402 Payment Required (lower bound: only the homepage is probed)no
Sends noai / noimageai directivesno
Refused our fetcher (403 / challenge)no

TDMRep rules: / reservation=1

robots.txt as recorded sha256 7ab878e2423eac8c…

User-agent: *
Disallow: /synsearch/
Disallow: /brouillon/
Disallow: /async/
Disallow: /proxy/

Sitemap: https://www.lefigaro.fr/sitemap_news.xml
Sitemap: https://sitemaps.lefigaro.fr/lefigaro.fr/sections.xml
Sitemap: https://sitemaps.lefigaro.fr/lefigaro.fr/articles.xml
Sitemap: https://sitemaps.lefigaro.fr/lefigaro.fr/topics.xml
Sitemap: https://www.lefigaro.fr/elections/resultats/sitemap.xml

User-agent: Meltwater
Disallow: /

robots.txt versions

FromThroughStateVersion
2026-10-262026-10-27ok 200f6127a8af381

Timeline of changes

DateWhat changed
No changes since the first observation (2026-10-26).

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

CrawlHostStateGPTBotOAI-SearchBotChatGPT-UserClaudeBotClaude-SearchBotClaude-UserPerplexityBot
2025-09-15 CC-MAIN-2025-38www.lefigaro.frok viewblocksblocksunknownblocksblocksunknownblocks
2024-09-16 CC-MAIN-2024-38www.lefigaro.frok viewblocksallowsunknownallowsallowsunknownallows
2023-10-02 CC-MAIN-2023-40www.lefigaro.frok viewblocksallowsunknownallowsallowsunknownallows
2022-10-03 CC-MAIN-2022-40www.lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2021-09-27 CC-MAIN-2021-39www.lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2020-09-28 CC-MAIN-2020-40www.lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2019-09-23 CC-MAIN-2019-39www.lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2018-09-24 CC-MAIN-2018-39www.lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2017-09-25 CC-MAIN-2017-39www.lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2017-07-24 CC-MAIN-2017-30www.lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2017-04-24 CC-MAIN-2017-17lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2017-01-23 CC-MAIN-2017-04lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2016-10-03 CC-MAIN-2016-40www.lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows
2016-09-05 CC-MAIN-2016-36lefigaro.frok viewallowsallowsunknownallowsallowsunknownallows

Evidence

Every daily observation of lefigaro.fr is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/lefigaro.fr