avito.ru: what it tells AI crawlers
On 27 October 2026, avito.ru's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.
Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.
- Panel
- top 1k (Tranco)
- robots.txt
- ok (HTTP 200) · https://www.avito.ru/robots.txt
unchanged since 2026-10-26 · sha256ce0465ca24da75e3…· view as recorded - The * group
- allows applies to every agent without its own group
Per AI bot, 2026-10-27
| User agent | Verdict | How |
|---|---|---|
| GPTBot | allows | through the * group |
| OAI-SearchBot | allows | through the * group |
| ChatGPT-User | allows | through the * group |
| ClaudeBot | allows | through the * group |
| Claude-SearchBot | allows | through the * group |
| Claude-User | allows | through the * group |
| PerplexityBot | allows | through the * group |
| Perplexity-User | allows | through the * group |
| Googlebot | allows | named in its own group |
| Google-Extended | allows | through the * group |
| Applebot-Extended | allows | through the * group |
| CCBot | allows | through the * group |
| Bytespider | allows | through the * group |
| Meta-ExternalAgent | allows | through the * group |
| Amazonbot | allows | through the * group |
Other signals
| Publishes /llms.txt | no |
| Publishes /ai.txt | no |
| Publishes /.well-known/tdmrep.json | no |
| Reserves TDM rights (TDMRep file, header or meta) | no |
| Declares Content Signals (robots.txt or header) | no |
| Says ai-train=no in a Content Signal | no |
| Points to an RSL licence (robots.txt License line or Link header) | no |
| Answers with an x402 payment offer (lower bound: only the homepage is probed) | no |
| Answers HTTP 402 Payment Required (lower bound: only the homepage is probed) | no |
| Sends noai / noimageai directives | no |
| Refused our fetcher (403 / challenge) | no |
robots.txt as recorded sha256 dea0300f7e9854b1…
User-agent: Yandex Disallow: /auto-logbook Allow: /pets/lost-and-found$ Allow: /pets/lost-and-found/$ Disallow: /pets/lost-and-found/ Disallow: /*?_openstat Disallow: /*nomobile= Disallow: /*utm_source= Disallow: /*view= Disallow: /*pmin= Disallow: /*pmax= Disallow: /*s= Disallow: /*user= Disallow: /*bt= Disallow: /*i= Disallow: /*gclid= Disallow: /*noimg Disallow: /*swf$ Disallow: /*price= Disallow: /*/edit$ Disallow: /*/edit/finish$ Disallow: /*/delete$ Disallow: /*/next$ Disallow: /*/*/*/do$ Disallow: /export/doc$ Disallow: /*/export/* Disallow: /items/abuse Disallow: /items/phone Disallow: /items/send2friend Disallow: /items/write Disallow: /items/edit Disallow: /items/delete Disallow: /items/next Disallow: /items/do Disallow: /items/stat Disallow: /restore Disallow: /account/ Disallow: /packages/ Disallow: /s/ Allow: /s/cc/ Disallow: /c/* Disallow: /stat/ Disallow: /api/ Disallow: /editemail Disallow: /js/ Disallow: /searchUrl Disallow: /events/add Disallow: /clickstream/ Disallow: /delivery/common Disallow: /k8s/ Disallow: /rpc/ Disallow: /performance/ Disallow: /bbl/ Disallow: /not-found Disallow: /orders/ Disallow: /stories Disallow: /events/add/ Disallow: /clickstream/ Disallow: /api/ Disallow: /*/rabota/tag/ Disallow: /autocredit/apply Disallow: /autocredit/calculator Disallow: /autocredit/sber_landing Disallow: /ukraina_*/ Disallow: /*/ukraina_*/ Disallow: /unsubscribe/ Disallow: /*/vakansii/catalog/ Disallow: /yclients-integrate-landing Disallow: */mall/ Allow: /general/mall/ Disallow: */business360/* Allow: */business360 Allow: */business360/oborudovanie_dlya_biznesa/ Allow: */business360/postavshchiki/oborudovanie_dlya_biznesa/ Disallow: /autosearch/ Disallow: /autosearch$ Disallow: /payment/ Disallow: /payment$ Disallow: /web/ Disallow: /au-imv-info/ Disallow: /au-imv-info$ Disallow: /search/filters/ Disallow: /ads-team$ Disallow: /recs/ Disallow: /recs$ Disallow: */interpretation/ Disallow: /profile/ Disallow: */autoload/ Disallow: /developments-catalog/download/document/ Disallow: /realty-agency-selection/* Disallow: /realty-agency-presentation/* Disallow: *3-k._kvartira_115_m_1153_et._4321427915* Disallow: *2-k._kvartira_82_m_1015_et._7745301411* Disallow: *4-k._kvartira_131_m_5968_et._4810347128* Disallow: *2-k._kvartira_82_m_1621_et._7738573941* Disallow: *2-k._kvartira_45_m_512_et._3911174194* Disallow: *2-k._kvartira_448_m_29_et._4412174907* Clean-param: smartleads-pid&0x419&20q&EsetProtoscanCtx&TPWlanWarnChecked&-360safeparam&-sm-au-&adj-adgroup&ca-adid&aq Clean-param: ca-agid&dim4&dim&ca-cid&categoryParentID&dim1&posting-id&password&adj-t&hjVerifyUUID&operation&userSearch&toWww&smartleads-src&hjAutogeneratedRecording&imagesAlt&kmmi7jrpjea5k6hveipiov7lou&lang&login&metaxua&msclkid&p47q&p59y0&parent-reqid&cat&city&conceptId&avito-ui-radio-group-247781&avito-ui-radio-group-42839&avito-ui-radio-group-5274142&avito-ui-radio-group-55414&avito-ui-radio-group-2187135&avito-ui-radio-group-388355&avito-ui-radio-group-471610 Clean-param: avito-ui-radio-group-625643&avito-ui-radio-group-1475384&avito-ui-radio-group-16418¶ms578-from-int&avito-ui-radio-group&avito-ui-radio-group-143876&avito-ui-radio-group-149&avito-ui-radio-group-1550&avito-ui-radio-group-1572&avito-ui-radio-group-16491&avito-ui-radio-group-176113&avito-ui-radio-group-179448&avito-ui-radio-group-18128&avito-ui-radio-group-18161&avito-ui-radio-group-18715&avito-ui-radio-group-23210&avito-ui-radio-group-23459&avito-ui-radio-group-26301 Clean-param: avito-ui-radio-group-27965&avito-ui-radio-group-28286&avito-ui-radio-group-31053&avito-ui-radio-group-32552&avito-ui-radio-group-325&avito-ui-radio-group-3694&avito-ui-radio-group-37693&avito-ui-radio-group-39893&avito-ui-radio-group-4491&avito-ui-radio-group-45678&avito-ui-radio-group-46647&avito-ui-radio-group-48549&avito-ui-radio-group-5142&avito-ui-radio-group-53857&avito-ui-radio-group-108260&avito-ui-radio-group-151692&avito-ui-radio-group-202135&avito-ui-radio-group-789604 Clean-param: avito-ui-radio-group-247782&avito-ui-radio-group-42840&avito-ui-radio-group-5274143&avito-ui-radio-group-55415&avito-ui-radio-group-16419&avito-ui-radio-group-110899&avito-ui-radio-group-103186&avito-ui-radio-group-103448&avito-ui-radio-group-108259&avito-ui-radio-group-151691&avito-ui-radio-group-202134&avito-ui-radio-group-48946&avito-ui-radio-group-7829&avito-ui-radio-group-789603 Clean-param: utm-med&verifyUser&context0¶ms110486¶ms1459¶ms48¶ms201¶ms498¶ms549¶ms162¶ms181¶ms716¶ms499¶ms578-from&avito-ui-radio-group-100769&avito-ui-radio-group-10896&avito-ui-radio-group-114397&avito-ui-radio-group-11678&avito-ui-radio-group-122494&avito-ui-radio-group-13259&avito-ui-radio-group-134032 Clean-param: avito-ui-radio-group-53858&avito-ui-radio-group-55254&avito-ui-radio-group-57052&avito-ui-radio-group-5839&avito-ui-radio-group-70489&avito-ui-radio-group-86166&avito-ui-radio-group-8803&avito-ui-radio-group-9072&avito-ui-radio-group-9341&avito-ui-radio-group-9415&avito-ui-radio-group-110900&countryID&stateId&cityID&langId&searchbox&search&log-sec&loc-hec&pcat&log-tec&-escaped-fragment-§ion&q12&q2&redig&rel-scan&search-author&search-query&sgProxy Clean-param: captcha-response&sub-id&cou&sa2&promocode&tl&e-captcha-response&captcha&h-captcha-response&vygoda-cb-active&cb-source&srsltid&source&context&sellerId&src&localPriority&proprofile&s_trg&category_id&propertyCost&token&next&category_id&showItems&backUrl&categoryId&cd®number&avito_campaign_id&reviewsPage&fromBlock Clean-param: loanPeriod&oldPrice&paymentID&platform&needShowGallery&srcp&str&fileFormat&locationId&fromPage&from_srf_offer&draftId&adrclid&mindbox-message-key&view&isVertical&chromehomescreenapp&targetPage&mango&drawId&state&orderId&avito-ui-radio-group-13260&avito-ui-radio-group-134033&avito-ui-radio-group-134620&avito-ui-radio-group-13532&avito-ui-radio-group-143877&avito-ui-radio-group-150&avito-ui-radio-group-1551&avito-ui-radio-group-1573 Clean-param: avito-ui-radio-group-134619&avito-ui-radio-group-13531&avito-ui-radio-group-23211&avito-ui-radio-group-23460&avito-ui-radio-group-26302&avito-ui-radio-group-27966&avito-ui-radio-group-28287&avito-ui-radio-group-31054&avito-ui-radio-group-32553&avito-ui-radio-group-326&avito-ui-radio-group-3695&avito-ui-radio-group-37694&avito-ui-radio-group-39894&avito-ui-radio-group-4492&avito-ui-radio-group-45679&avito-ui-radio-group-46648&avito-ui-radio-group-48550&avito-ui-radio-group-5143 Clean-param: avito-ui-radio-group-57051&avito-ui-radio-group-5838&avito-ui-radio-group-70488&avito-ui-radio-group-86165&mode&order&tags&code&a&lr&sub_id&iid&lr&text Clean-param: avito-ui-radio-group-877344&avito-ui-radio-group-107360&avito-ui-radio-group-14806&avito-ui-radio-group-153682&avito-ui-radio-group-107361&avito-ui-radio-group-14807&avito-ui-radio-group-153683&avito-ui-radio-group-23839&avito-ui-radio-group-23838&debug&editMode&gclit&spm&tduid&ulyanovsk&value&wbraid&serviceType&ouid&oauth&language&sellerId1&sellerId0&s1&actions&context1&s0&aslkdlk&avito-ui-radio-group-55252 Clean-param: query&f&s&radius&hist_back&i&sgtd&presentationType&sort&t&d&id&&adjust_t&adjust_campaign&adjust_creative&etext&gps_adid&idfa&oaid&filter&code&createApp&pointId&itemId&embedded&nonRefundableDiscountEnable Clean-param: slocation&%3Futm_medium&1745083422&569F3416-5542-4443-9460-210BF803FB10_kfa_cup_C6FA3ED5_6D17_47D1_B6E2_F4B02CC905E0_&_360safeparam&adjust_adgroup&advGeoRedirect&after_saving_draft&ca_adid&clid&ebq&erid>m_debug&hrbkey&ldrxq&olx&openStory&product&segment&smartleads_pid&userid&v&verifyUserLocation&writein&checkIn&rn&from_page&expanded&pmin Clean-param: pmax&guestsDetailed&back&addToCart&clckid&wd&srv-online-booking&delivery&from&cvPackageTypeApplied&tracker_limit&yadclid&utm_campai&ca_agid&Id&b&mt_click_id&refresh&searchForm&smartleads_sid&checkOut&vsrc&yadordid&ca_chid&win&g&smartleads_src&filter&mortgageProgram&paymentId&client_id&campaign&from_block&pageFrom&sharingKey&adjust_adgr&gigAction&scope Clean-param: searchEnable&ctime&aviclid&hideExampleReportButton&group Sitemap: https://www.avito.ru/sitemap/index.xml User-agent: Googlebot Disallow: /auto-logbook Allow: /pets/lost-and-found$ Allow: /pets/lost-and-found/$ Disallow: /pets/lost-and-found/ Disallow: /*?_openstat Disallow: /*nomobile= Disallow: /*noimg Disallow: /*swf$ Disallow: /*price= Disallow: /*/edit$ Disallow: /*/edit/finish$ Disallow: /*/delete$ Disallow: /*/next$ Disallow: /*/*/*/do$ Disallow: /export/doc$ Disallow: /*/export/* Disallow: /items/abuse Disallow: /items/phone Disallow: /items/send2friend Disallow: /items/write Disallow: /items/edit Disallow: /items/delete Disallow: /items/next Disallow: /items/do Disallow: /items/stat Disallow: /restore Disallow: /account/ Disallow: /packages/ Disallow: /s/ Allow: /s/cc/ Disallow: /c/* Disallow: /stat/ Disallow: /api/ Disallow: /editemail Disallow: /js/ Disallow: /events/add Disallow: /clickstream/ Disallow: /delivery/common Disallow: /k8s/ Disallow: /rpc/ Disallow: /performance/ Disallow: /bbl/ Disallow: /not-found Disallow: /orders/ Disallow: /stories Disallow: /events/add/ Disallow: /clickstream/ Disallow: /api/ Disallow: /*/rabota/tag/ Allow: /*/predlozheniya_uslug/*-ASg*/tag/ Disallow: /*-ASg*/tag/ Disallow: /autocredit/apply Disallow: /autocredit/calculator Disallow: /autocredit/sber_landing Disallow: /ukraina_*/ Disallow: /*/ukraina_*/ Disallow: /unsubscribe/ Disallow: /*regnumber= Disallow: /*avito_campaign_id= Disallow: /*reviewsPage= Disallow: /*/vakansii/catalog/ Disallow: /yclients-integrate-landing Disallow: */mall/ Allow: /general/mall/ Disallow: */business360/* Allow: */business360 Allow: */business360/oborudovanie_dlya_biznesa/ Allow: */business360/postavshchiki/oborudovanie_dlya_biznesa/ Disallow: /autosearch/ Disallow: /autosearch$ Disallow: /payment/ Disallow: /payment$ Disallow: /web/ Disallow: /au-imv-info/ Disallow: /au-imv-info$ Disallow: /search/filters/ Disallow: /ads-team$ Disallow: /recs/ Disallow: /recs$ Disallow: */interpretation/ Disallow: /profile/ Disallow: */autoload/ Disallow: /developments-catalog/download/document/ Disallow: /realty-agency-selection/* Disallow: /realty-agency-presentation/* Disallow: /*?f= Disallow: /*&f= Disallow: /*?radius= Disallow: /*&radius= Disallow: /*?searchRadius= Disallow: /*&searchRadius= Disallow: /*?s= Disallow: /*&s= Disallow: /*?sort= Disallow: /*&sort= Disallow: /*?view= Disallow: /*&view= Allow: /*view=default Disallow: /*?expanded= Disallow: /*&expanded= Disallow: /*?rn= Disallow: /*&rn= Disallow: /*?source= Disallow: /*&source= Disallow: /*?page_from= Disallow: /*&page_from= Disallow: /*?from_page= Disallow: /*&from_page= Disallow: /*?fromPage= Disallow: /*&fromPage= Disallow: /additem Allow: /autoteka-products Disallow: /autoteka-products/ Allow: /autoteka-report-example$ Disallow: /autoteka-report-example? Allow: /autoteka-report-by-identifier Disallow: /autoteka-report-by-identifier/ Disallow: /user/review Disallow: /order/ Disallow: /rewards/ Disallow: /avcalls-window Disallow: /booking/ Disallow: /promotions/ Disallow: /catalog/*/compare-equipment Sitemap: https://www.avito.ru/sitemap/index.xml Sitemap: https://m.avito.ru/sitemap/mobile_index.xml User-agent: * Disallow: /auto-logbook Allow: /pets/lost-and-found$ Allow: /pets/lost-and-found/$ Disallow: /pets/lost-and-found/ Disallow: /*?_openstat Disallow: /*nomobile= Disallow: /*utm_source= Disallow: /*view= Allow: /*view=default Disallow: /*pmin= Disallow: /*pmax= Disallow: /*s= Disallow: /*user= Disallow: /*bt= Disallow: /*i= Disallow: /*noimg Disallow: /*swf$ Disallow: /*price= Disallow: /*/edit$ Disallow: /*/edit/finish$ Disallow: /*/delete$ Disallow: /*/next$ Disallow: /*/*/*/do$ Disallow: /export/doc$ Disallow: /*/export/* Disallow: /items/abuse Disallow: /items/phone Disallow: /items/send2friend Disallow: /items/write Disallow: /items/edit Disallow: /items/delete Disallow: /items/next Disallow: /items/do Disallow: /items/stat Disallow: /restore Disallow: /account/ Disallow: /packages/ Disallow: /s/ Allow: /s/cc/ Disallow: /c/* Disallow: /stat/ Disallow: /api/ Disallow: /editemail Disallow: /js/ Disallow: /reklama/delivery Disallow: /events/add Disallow: /clickstream/ Disallow: /delivery/common Disallow: /k8s/ Disallow: /rpc/ Disallow: /performance/ Disallow: /bbl/ Disallow: /not-found Disallow: /orders/ Disallow: /stories Disallow: /events/add/ Disallow: /clickstream/ Disallow: /api/ Disallow: /*/rabota/tag/ Allow: /*/predlozheniya_uslug/*-ASg*/tag/ Disallow: /*-ASg*/tag/ Disallow: /autocredit/apply Disallow: /autocredit/calculator Disallow: /autocredit/sber_landing Disallow: /job-payments Disallow: /ukraina_*/ Disallow: /*/ukraina_*/ Disallow: /unsubscribe/ Disallow: /*regnumber= Disallow: /*avito_campaign_id= Disallow: /*reviewsPage= Disallow: /q/* Disallow: /*/vakansii/catalog/ Disallow: /yclients-integrate-landing Disallow: */mall/ Allow: /general/mall/ Disallow: */business360/* Allow: */business360 Allow: */business360/oborudovanie_dlya_biznesa/ Allow: */business360/postavshchiki/oborudovanie_dlya_biznesa/ Disallow: /autosearch/ Disallow: /autosearch$ Disallow: /payment/ Disallow: /payment$ Disallow: /web/ Disallow: /au-imv-info/ Disallow: /au-imv-info$ Disallow: /search/filters/ Disallow: /ads-team$ Disallow: /recs/ Disallow: /recs$ Disallow: */interpretation/ Disallow: /profile/ Disallow: */autoload/ Disallow: /developments-catalog/download/document/ Disallow: /order/checkout/ Disallow: /add* Disallow: /registration/ Disallow: /realty-agency-selection/* Disallow: /realty-agency-presentation/* Disallow: /*?f= Disallow: /*&f= Disallow: /*?radius= Disallow: /*&radius= Disallow: /*?searchRadius= Disallow: /*&searchRadius= Disallow: /*?s= Disallow: /*&s= Disallow: /*?sort= Disallow: /*&sort= Disallow: /*?view= Disallow: /*&view= Allow: /*view=default Disallow: /*?expanded= Disallow: /*&expanded= Disallow: /*?rn= Disallow: /*&rn= Disallow: /*?source= Disallow: /*&source= Disallow: /*?page_from= Disallow: /*&page_from= Disallow: /*?from_page= Disallow: /*&from_page= Disallow: /*?fromPage= Disallow: /*&fromPage= Disallow: /additem Allow: /autoteka-products Disallow: /autoteka-products/ Allow: /autoteka-report-example$ Disallow: /autoteka-report-example? Allow: /autoteka-report-by-identifier Disallow: /autoteka-report-by-identifier/ Disallow: /user/review Disallow: /order/ Disallow: /rewards/ Disallow: /avcalls-window Disallow: /booking/ Disallow: /promotions/ Disallow: /catalog/*/compare-equipment Sitemap: https://www.avito.ru/sitemap/index.xml Sitemap: https://m.avito.ru/sitemap/mobile_index.xml User-agent: Mediapartners-Google Disallow: User-agent: YandexDirect Allow: / Sitemap: https://www.avito.ru/sitemap/index.xml
robots.txt versions
| From | Through | State | Version | |
|---|---|---|---|---|
| 2026-10-26 | 2026-10-27 | ok 200 | ce0465ca24da |
Timeline of changes
| Date | What changed |
|---|---|
| No changes since the first observation (2026-10-26). | |
Reconstructed history (Common Crawl)
From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.
| Crawl | Host | State | GPTBot | OAI-SearchBot | ChatGPT-User | ClaudeBot | Claude-SearchBot | Claude-User | PerplexityBot |
|---|---|---|---|---|---|---|---|---|---|
| 2026-09-21 CC-MAIN-2026-39 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2025-09-15 CC-MAIN-2025-38 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2024-09-16 CC-MAIN-2024-38 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2023-10-02 CC-MAIN-2023-40 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2022-10-03 CC-MAIN-2022-40 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2021-09-27 CC-MAIN-2021-39 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2020-09-28 CC-MAIN-2020-40 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2019-09-23 CC-MAIN-2019-39 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-09-24 CC-MAIN-2018-39 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-01-29 CC-MAIN-2018-05 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-10-23 CC-MAIN-2017-43 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-09-25 CC-MAIN-2017-39 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-07-24 CC-MAIN-2017-30 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-04-24 CC-MAIN-2017-17 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-01-23 CC-MAIN-2017-04 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-10-03 CC-MAIN-2016-40 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-09-05 CC-MAIN-2016-36 | www.avito.ru | ok view | allows | allows | unknown | allows | allows | unknown | allows |
Evidence
Every daily observation of avito.ru is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).
API: https://infrarabbit.com/crawlers/api/v1/policy/avito.ru