allegro.pl: what it tells AI crawlers

On 27 October 2026, allegro.pl's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot; it also published llms.txt.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://allegro.pl/robots.txt
unchanged since 2026-10-26 · sha256 a1cd0a7283149064… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

llms.txt: "Allegro", 14,037 bytes, 55 links.

robots.txt as recorded sha256 a1cd0a7283149064…

user-agent: *
disallow: /showcase.php/
disallow: /SendMailToUser.php*
disallow: /report_item.php*
disallow: /events/clicks
disallow: /offer/
disallow: /ape
disallow: /payments
disallow: /recommendations/
disallow: /cart-aggregator/
disallow: /favourites/
disallow: /usertraits/
disallow: /mediation/
disallow: *snapshot=*
disallow: /language-dispatcher
disallow: *&source=allegro.com&welcome=1&shippingCountry=US*
disallow: /login/auth
disallow: /auth/oauth/authorize
disallow: /produkt/warianty/
disallow: /uzytkownik*&*
disallow: /opinie-produktowe/
disallow: /zglos-naruszenie/
disallow: /uzytkownik/*SHOP-TAG*
disallow: /magazyn-allegro/
disallow: /szukaj/multi?phrases*
disallow: /promowane/
disallow: /kampanie/*&*
disallow: *reco_id=*
disallow: *preco_id=*
disallow: /*?*=*&*=*&*=*&*=*&*=*&*=*&*=*&*=*&*=*&*=*&*
disallow: /kategoria/*?*=*&*=*&*=*&*=*&*=*&*=*&*
disallow: /moje-allegro/
disallow: /transakcja/
disallow: /z-drugiej-reki
allow: /uzytkownik/notfound?string*
allow: /uzytkownik/*bi_s=archiwum_allegro*
allow: /kategoria/*bi_s=archiwum_allegro*

user-agent: Mediapartners-Google
disallow: *origin_url=*
disallow: /transakcja/

User-agent: CCBot
Allow: /robots.txt$
Disallow: /*robots.txt
Disallow: /*bmatch=*
Disallow: /*gclid=*
Disallow: /*utm_campaign=*
Disallow: /*bi_s=*
Disallow: /logowanie*
Disallow: /auth/
Disallow: /uzytkownik/notfound*

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of allegro.pl is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/allegro.pl