leboncoin.fr: what it tells AI crawlers

On 27 October 2026, leboncoin.fr's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 10k (Tranco)

robots.txt

ok (HTTP 200) · https://www.leboncoin.fr/robots.txt
unchanged since 2026-10-26 · sha256 bac2f1bf88edc5f0… · view as recorded

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 6808ae2e129b3777…

## It's forbidden to use search robots or other automatic methods to access Leboncoin.fr.
## Access is only permitted with special permission from Leboncoin.fr.

## Bots friendly
User-agent: Googlebot
User-agent: Googlebot-Mobile
User-agent: Yahoo! Slurp
User-agent: Slurp
User-agent: slurp
User-agent: msnbot
User-agent: bingbot
User-agent: ia_archiver
# friendly exceptions
Allow: /aide.htm
Allow: /aide_pro.htm
# www2 only
Allow: /livres
Allow: /linge_de_maison
Disallow: /ai
Disallow: /cp
Disallow: /support
Disallow: /account
Disallow: /account/index.html
Disallow: /dc/decisions_de_justice
Disallow: /dc/accessibilite
# Adviews
Disallow: /annonce
Disallow: /classified/latest/
Allow: /annonces
Disallow: /vi/*
Disallow: /li
Disallow: /*/*.htm
Disallow: /ad/animaux/*
Disallow: /ad/animaux_perdus/*
Disallow: /ad/autres/*
# Holidays calendar pages
Disallow: /*/vacancier/
# Bloquage des Demandes
Disallow: /*/demandes
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
Disallow: /report-abuse
Disallow: /profile/
Disallow: /oauth2callback
Disallow: /discovery/*
# Blocage des /Recherche (Rav)
Disallow: /recherche
Disallow: /boutique/*/*print=0
Disallow: /boutique/*.htm?*ca=*_k&*
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Disallow: /*_mediametrie
#on autorise boutiques (.htm)
Allow: /boutique*
#on autorise les html
Allow: /*/*.html
Disallow: /*/bonnes_affaires
# Pages filtres
Allow: /f/*
# Gestion villes
Allow: /v/*
#Partners
Disallow: /partners/*
Allow: /partners/sofinco*
#Messaging
Disallow: /messages/*
Allow: /*

## AdsBot-Google and AdsBot-Google-Mobile
User-agent: AdsBot-Google
User-agent: AdsBot-Google-Mobile
# friendly exceptions
Allow: /recherche?category=2&*
Allow: /recherche?category=9&*
Allow: /recherche?category=10&*
Allow: /recherche?category=11&*
Allow: /recherche?category=13&*
Allow: /aide.htm
Allow: /aide_pro.htm
# www2 only
Allow: /livres
Allow: /linge_de_maison
Disallow: /ai
Disallow: /cp
Disallow: /support
Disallow: /account
Disallow: /account/index.html
Disallow: /dc/decisions_de_justice
Disallow: /dc/accessibilite
# Adviews
Disallow: /annonce
Disallow: /classified/latest/
Allow: /annonces
Disallow: /vi/*.htm
Disallow: /li
# Holidays calendar pages
Disallow: /*/vacancier/
# Bloquage des Demandes
Disallow: /*/demandes
# all
Disallow: /*th=0
Disallow: /aw
Disallow: /st
Disallow: /maintenance.html
Disallow: /pub
Disallow: /report-abuse
Disallow: /profile/
Disallow: /oauth2callback
Disallow: /boutique/*/*print=0
Disallow: /boutique/*.htm?*ca=*_k&*
Disallow: /popup_siege_social.htm
Disallow: /dc/popup_siege_social
Disallow: /*_mediametrie
Disallow: /discovery/*
#on autorise boutiques (.htm)
Allow: /boutique*
#on autorise les html
Allow: /*/*.html
#on bloque les annonces avec et sans parametres
Disallow: /*/*.htm
Disallow: /*/bonnes_affaires/
Disallow: /f/*
Allow: /*

# Allow specific Ads Googlebot to access all our pages
User-agent: Mediapartners-Google
Allow: /*

# Allow proximic to access all our pages
User-agent: proximic
Disallow:

# LLMs configuration
User-agent: Google-Extended
Disallow: /ad/
Allow: /*

User-agent: OAI-SearchBot
Disallow: /ad/
Allow: /*

User-agent: ChatGPT-User
Disallow: /ad/
Allow: /*

# Link to the sitemap index file
Sitemap: https://www.leboncoin.fr/auto-main.xml

Disallow: /app-router-poc
Disallow: /pages-router-poc

# Allow Meta's crawler for Instagram feature
User-agent: facebookexternalhit
Allow: /dc/cookies
Disallow: /*

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of leboncoin.fr is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/leboncoin.fr