leboncoin.fr: what it tells AI crawlers

On 27 October 2026, leboncoin.fr's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 10k (Tranco)

robots.txt

ok (HTTP 200) · https://www.leboncoin.fr/robots.txt
unchanged since 2026-10-26 · sha256 bac2f1bf88edc5f0… · view as recorded

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 7a7e5ef144db4ca1…

## It's forbidden to use search robots or other automatic methods to access Leboncoin.fr.
## Access is only permitted with special permission from Leboncoin.fr.

## Google
User-agent: Googlebot
# friendly exceptions
Allow: /aide.htm
Allow: /aide_pro.htm
Allow: /arts_de_la_table
Allow: /materiel
Allow: /services
Allow: /sports_hobbies
# www2 only
Disallow: /ai
Disallow: /ar
Disallow: /cp
Disallow: /ma
Disallow: /rd
Disallow: /se
Disallow: /sp
Disallow: /support
Disallow: /dc/decisions_de_justice
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
Disallow: /boutique/*/*print=0
Disallow: /boutique/*.htm?*ca=*_k&*
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Disallow: /*_mediametrie
Allow: /*

User-agent: Mediapartners-Google
# friendly exceptions
Allow: /aide.htm
Allow: /aide_pro.htm
Allow: /arts_de_la_table
Allow: /materiel
Allow: /services
Allow: /sports_hobbies
# www2 only
Disallow: /ai
Disallow: /ar
Disallow: /cp
Disallow: /ma
Disallow: /rd
Disallow: /se
Disallow: /sp
Disallow: /support
Disallow: /dc/decisions_de_justice
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
Disallow: /boutique/*/*print=0
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Disallow: /*_mediametrie
Allow: /*

## Yahoo
User-agent: slurp
# friendly exceptions
Allow: /aide.htm
Allow: /aide_pro.htm
Allow: /arts_de_la_table
Allow: /materiel
Allow: /services
Allow: /sports_hobbies
# www2 only
Disallow: /ai
Disallow: /ar
Disallow: /cp
Disallow: /ma
Disallow: /rd
Disallow: /se
Disallow: /sp
Disallow: /support
Disallow: /dc/decisions_de_justice
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
Disallow: /boutique/*/*print=0
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Disallow: /*_mediametrie
Allow: /*

# Googlebot Mobile
User-agent: Googlebot-Mobile
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Allow: /*

User-agent: msnbot
# friendly exceptions
Allow: /aide.htm
Allow: /aide_pro.htm
Allow: /arts_de_la_table
Allow: /materiel
Allow: /services
Allow: /sports_hobbies
# www2 only
Disallow: /ai
Disallow: /ar
Disallow: /cp
Disallow: /ma
Disallow: /rd
Disallow: /se
Disallow: /sp
Disallow: /support
Disallow: /dc/decisions_de_justice
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
Disallow: /boutique/*/*print=0
Disallow: /popup_siege_social.htm
Disallow: /*_mediametrie
Allow: /*

# allow archive.org to screenshot all our pages
User-agent: ia_archiver
Disallow:

User-agent: Speobot
Disallow: /

User-agent: *
Disallow: /

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of leboncoin.fr is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/leboncoin.fr