avito.ru: what it tells AI crawlers

On 27 October 2026, avito.ru's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://www.avito.ru/robots.txt
unchanged since 2026-10-26 · sha256 ce0465ca24da75e3… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 469f4500f1997dd4…

Host: https://www.avito.ru

User-agent: Yandex
Disallow: /*?_openstat
Disallow: /*nomobile=
Disallow: /*utm_source=
Disallow: /*view=
Allow: /*view=default
Disallow: /*pmin=
Disallow: /*pmax=
Disallow: /*s=
Disallow: /*user=
Disallow: /*bt=
Disallow: /*i=
Disallow: /*gclid=
Disallow: /*noimg
Disallow: /*swf$
Disallow: /*price=
Disallow: /*p=
Disallow: /*/edit$
Disallow: /*/delete$
Disallow: /*/next$
Disallow: /*/*/*/do$
Disallow: /export/doc$
Disallow: /*/export/*
Disallow: /items/abuse
Disallow: /items/phone
Disallow: /items/send2friend
Disallow: /items/write
Disallow: /items/edit
Disallow: /items/delete
Disallow: /items/next
Disallow: /items/do
Disallow: /items/stat
Disallow: /restore
Disallow: /account/
Disallow: /packages/
Disallow: /s/
Allow: /s/cc/
Disallow: /stat/
Disallow: /api/
Disallow: /editemail
Disallow: /js/
Disallow: /searchUrl
Disallow: /events/add
Disallow: /clickstream/
Disallow: /delivery/common
Disallow: /k8s/
Disallow: /rpc/
Disallow: /performance/stats/1/load
Disallow: /stories
Disallow: /events/add/
Disallow: /clickstream/
Disallow: /api/
Disallow: /*/rabota/tag/
Disallow: /autocredit/calculator
Sitemap: https://www.avito.ru/sitemap/index.xml

User-agent: Mediapartners-Google
Disallow:

User-agent: YandexDirect
Allow: /
Sitemap: https://www.avito.ru/sitemap/index.xml


User-agent: Googlebot
Disallow: /*?_openstat
Disallow: /*nomobile=
Disallow: /*noimg
Disallow: /*swf$
Disallow: /*price=
Disallow: /*/edit$
Disallow: /*/delete$
Disallow: /*/next$
Disallow: /*/*/*/do$
Disallow: /export/doc$
Disallow: /*/export/*
Disallow: /items/abuse
Disallow: /items/phone
Disallow: /items/send2friend
Disallow: /items/write
Disallow: /items/edit
Disallow: /items/delete
Disallow: /items/next
Disallow: /items/do
Disallow: /items/stat
Disallow: /restore
Disallow: /account/
Disallow: /packages/
Disallow: /s/
Allow: /s/cc/
Disallow: /stat/
Disallow: /api/
Disallow: /editemail
Disallow: /js/
Disallow: /events/add
Disallow: /clickstream/
Disallow: /delivery/common
Disallow: /k8s/
Disallow: /rpc/
Disallow: /performance/stats/1/load
Disallow: /stories
Disallow: /*p=
Disallow: /events/add/
Disallow: /clickstream/
Disallow: /api/
Disallow: /*/rabota/tag/
Disallow: /autocredit/calculator
Sitemap: https://www.avito.ru/sitemap/index.xml


User-agent: *
Disallow: /*?_openstat
Disallow: /*nomobile=
Disallow: /*utm_source=
Disallow: /*view=
Allow: /*view=default
Disallow: /*pmin=
Disallow: /*pmax=
Disallow: /*s=
Disallow: /*user=
Disallow: /*bt=
Disallow: /*i=
Disallow: /*noimg
Disallow: /*swf$
Disallow: /*price=
Disallow: /*p=
Disallow: /*/edit$
Disallow: /*/delete$
Disallow: /*/next$
Disallow: /*/*/*/do$
Disallow: /export/doc$
Disallow: /*/export/*
Disallow: /items/abuse
Disallow: /items/phone
Disallow: /items/send2friend
Disallow: /items/write
Disallow: /items/edit
Disallow: /items/delete
Disallow: /items/next
Disallow: /items/do
Disallow: /items/stat
Disallow: /restore
Disallow: /account/
Disallow: /packages/
Disallow: /s/
Allow: /s/cc/
Disallow: /stat/
Disallow: /api/
Disallow: /editemail
Disallow: /js/
Disallow: /reklama/delivery
Disallow: /events/add
Disallow: /clickstream/
Disallow: /delivery/common
Disallow: /k8s/
Disallow: /rpc/
Disallow: /performance/stats/1/load
Disallow: /stories
Disallow: /events/add/
Disallow: /clickstream/
Disallow: /api/
Disallow: /*/rabota/tag/
Disallow: /autocredit/calculator
Sitemap: https://www.avito.ru/sitemap/index.xml

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of avito.ru is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/avito.ru