leboncoin.fr: what it tells AI crawlers

On 27 October 2026, leboncoin.fr's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel: top 10k (Tranco)
robots.txt: ok (HTTP 200) · https://www.leboncoin.fr/robots.txt
unchanged since 2026-10-26 · sha256 bac2f1bf88edc5f0… · view as recorded

Per AI bot, 2026-10-27

User agent	Verdict	How
GPTBot	allows	named in its own group
OAI-SearchBot	allows	named in its own group
ChatGPT-User	allows	named in its own group
ClaudeBot	allows	named in its own group
Claude-SearchBot	allows	named in its own group
Claude-User	allows	named in its own group
PerplexityBot	allows	named in its own group
Perplexity-User	allows	named in its own group
Googlebot	allows	named in its own group
Google-Extended	allows	named in its own group
Applebot-Extended	allows	named in its own group
CCBot	allows	named in its own group
Bytespider	blocks	named in its own group
Meta-ExternalAgent	allows	no applicable rules
Amazonbot	allows	no applicable rules
anthropic-ai	allows	named in its own group
Applebot	allows	named in its own group
DuckAssistBot	allows	named in its own group
facebookexternalhit	blocks	named in its own group
MistralAI-Index	allows	named in its own group
MistralAI-User	allows	named in its own group

Other signals

Publishes /llms.txt	not observed
Publishes /ai.txt	not observed
Publishes /.well-known/tdmrep.json	not observed
Reserves TDM rights (TDMRep file, header or meta)	not observed
Declares Content Signals (robots.txt or header)	not observed
Says ai-train=no in a Content Signal	not observed
Points to an RSL licence (robots.txt License line or Link header)	not observed
Answers with an x402 payment offer (lower bound: only the homepage is probed)	not observed
Answers HTTP 402 Payment Required (lower bound: only the homepage is probed)	not observed
Sends noai / noimageai directives	not observed
Refused our fetcher (403 / challenge)	no

robots.txt as recorded sha256 9665448c6015ef9e…

## It's forbidden to use search robots or other automatic methods to access Leboncoin.fr.
## Access is only permitted with special permission from Leboncoin.fr.

## Google
User-agent: Googlebot
# friendly exceptions
Allow: /aide.htm
Allow: /aide_pro.htm
# www2 only
Allow: /livres
Allow: /linge_de_maison
Disallow: /ai
Disallow: /cp
Disallow: /support
Disallow: /account
Allow: /account/index.html
Disallow: /dc/decisions_de_justice
# Adviews
Disallow: /annonce/
Disallow: /vi/*.htm
Disallow: /li
# Bloquage des Demandes
Disallow: /*/demandes/
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
# Blocage des /Recherche (Rav)
Disallow: /recherche
Disallow: /boutique/*/*print=0
Disallow: /boutique/*.htm?*ca=*_k&*
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Disallow: /*_mediametrie
#on autorise boutiques (.htm)
Allow: /boutique*
#on autorise les html
Allow: /*/*.html
#on bloque les annonces avec et sans parametres
Disallow: /*/*.htm
Disallow: /*/bonnes_affaires/
Allow: /*

User-agent: Mediapartners-Google
# friendly exceptions
Allow: /aide.htm
Allow: /aide_pro.htm
# www2 only
Allow: /livres
Allow: /linge_de_maison
Disallow: /ai
Disallow: /cp
Disallow: /support
Disallow: /account
Allow: /account/index.html
Disallow: /dc/decisions_de_justice
# Adviews
Disallow: /annonce/
Disallow: /vi/*.htm
Disallow: /li
# Bloquage des Demandes
Disallow: /*/demandes/
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
# Blocage des /Recherche (Rav)
Disallow: /recherche
Disallow: /boutique/*/*print=0
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Disallow: /*_mediametrie
Allow: /*

## Yahoo
User-agent: slurp
# friendly exceptions
Allow: /aide.htm
Allow: /aide_pro.htm
# www2 only
Allow: /livres
Allow: /linge_de_maison
Disallow: /ai
Disallow: /cp
Disallow: /support
Disallow: /account
Allow: /account/index.html
Disallow: /dc/decisions_de_justice
# Adviews
Disallow: /annonce/
Disallow: /vi/*.htm
Disallow: /li
# Bloquage des Demandes
Disallow: /*/demandes/
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
# Blocage des /Recherche (Rav)
Disallow: /recherche
Disallow: /boutique/*/*print=0
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Disallow: /*_mediametrie
Allow: /*

# Googlebot Mobile
User-agent: Googlebot-Mobile
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
# Bloquage des Demandes
Disallow: /*/demandes/
Allow: /aide.htm
Allow: /aide_pro.htm
# www2 only
Allow: /livres
Allow: /linge_de_maison
Disallow: /ai
Disallow: /cp
Disallow: /support
Disallow: /account
Allow: /account/index.html
Disallow: /dc/decisions_de_justice
# Adviews
Disallow: /annonce/
Disallow: /vi/*.htm
Disallow: /li
# Bloquage des Demandes
Disallow: /*/demandes/
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
# Blocage des /Recherche (Rav)
Disallow: /recherche
Disallow: /boutique/*/*print=0
Disallow: /boutique/*.htm?*ca=*_k&*
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Disallow: /*_mediametrie
#on autorise boutiques (.htm)
Allow: /boutique*
#on autorise les html
Allow: /*/*.html
#on bloque les annonces avec et sans parametres
Disallow: /*/*.htm
Disallow: /*/bonnes_affaires/
Allow: /*

User-agent: msnbot
# friendly exceptions
Allow: /aide.htm
Allow: /aide_pro.htm
# www2 only
Allow: /livres
Allow: /linge_de_maison
Disallow: /ai
Disallow: /cp
Disallow: /support
Disallow: /account
Allow: /account/index.html
Disallow: /dc/decisions_de_justice
# Adviews
Disallow: /annonce/
Disallow: /vi/*.htm
Disallow: /li
# Bloquage des Demandes
Disallow: /*/demandes/
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
# Blocage des /Recherche (Rav)
Disallow: /recherche
Disallow: /boutique/*/*print=0
Disallow: /popup_siege_social.htm
Disallow: /*_mediametrie
Allow: /*

# allow archive.org to screenshot all our pages
User-agent: ia_archiver
Disallow:

# Allow proximic to access all our pages
User-agent: proximic
Disallow:

User-agent: Speobot
Disallow: /

robots.txt versions

From	Through	State	Version
2026-10-26	2026-10-27	ok 200	`bac2f1bf88ed`

Timeline of changes

Date	What changed
No changes since the first observation (2026-10-26).

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Crawl	Host	State	GPTBot	OAI-SearchBot	ChatGPT-User	ClaudeBot	Claude-SearchBot	Claude-User	PerplexityBot
2026-09-21 CC-MAIN-2026-39	www.leboncoin.fr	ok view	allows	allows	unknown	allows	allows	unknown	allows
2025-09-15 CC-MAIN-2025-38	www.leboncoin.fr	ok view	allows	allows	unknown	allows	allows	unknown	allows
2024-09-16 CC-MAIN-2024-38	www.leboncoin.fr	ok view	allows	allows	unknown	allows	allows	unknown	allows
2023-10-02 CC-MAIN-2023-40	www.leboncoin.fr	ok view	allows	allows	unknown	allows	allows	unknown	allows
2022-10-03 CC-MAIN-2022-40	www2.leboncoin.fr	redirect-only	unknown	unknown	unknown	unknown	unknown	unknown	unknown
2021-09-27 CC-MAIN-2021-39	www2.leboncoin.fr	ok view	allows	allows	unknown	allows	allows	unknown	allows
2020-09-28 CC-MAIN-2020-40	www2.leboncoin.fr	ok view	allows	allows	unknown	allows	allows	unknown	allows
2019-09-23 CC-MAIN-2019-39	www2.leboncoin.fr	ok view	allows	allows	unknown	allows	allows	unknown	allows
2018-09-24 CC-MAIN-2018-39	www2.leboncoin.fr	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-09-25 CC-MAIN-2017-39	www2.leboncoin.fr	ok view	blocks	blocks	unknown	blocks	blocks	unknown	blocks
2017-07-24 CC-MAIN-2017-30	www2.leboncoin.fr	ok view	blocks	blocks	unknown	blocks	blocks	unknown	blocks
2017-04-24 CC-MAIN-2017-17	leboncoin.fr	redirect-only	unknown	unknown	unknown	unknown	unknown	unknown	unknown
2017-01-23 CC-MAIN-2017-04	www2.leboncoin.fr	ok view	blocks	blocks	unknown	blocks	blocks	unknown	blocks
2016-10-03 CC-MAIN-2016-40	leboncoin.fr	redirect-only	unknown	unknown	unknown	unknown	unknown	unknown	unknown
2016-09-05 CC-MAIN-2016-36	leboncoin.fr	redirect-only	unknown	unknown	unknown	unknown	unknown	unknown	unknown

Evidence

Every daily observation of leboncoin.fr is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/leboncoin.fr