vtb.ru: what it tells AI crawlers

On 28 October 2026, vtb.ru's robots.txt fully blocked GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-29 03:59 UTC (daily run of 2026-10-28). Proof for that day.

Panel: top 10k (Tranco)
robots.txt: ok (HTTP 200) · https://xn--90ab2c.xn--p1ai/robots.txt
unchanged since 2026-10-26 · sha256 6856255165e0c5f2… · view as recorded
The * group: blocks applies to every agent without its own group

Per AI bot, 2026-10-28

User agent	Verdict	How
GPTBot	blocks	through the * group
OAI-SearchBot	blocks	through the * group
ChatGPT-User	blocks	through the * group
ClaudeBot	blocks	through the * group
Claude-SearchBot	blocks	through the * group
Claude-User	blocks	through the * group
PerplexityBot	blocks	through the * group
Perplexity-User	blocks	through the * group
Googlebot	blocks	through the * group
Google-Extended	blocks	through the * group
Applebot-Extended	blocks	through the * group
CCBot	blocks	through the * group
Bytespider	blocks	through the * group
Meta-ExternalAgent	blocks	through the * group
Amazonbot	blocks	through the * group

Other signals

Publishes /llms.txt	not observed
Publishes /ai.txt	not observed
Publishes /.well-known/tdmrep.json	not observed
Reserves TDM rights (TDMRep file, header or meta)	not observed
Declares Content Signals (robots.txt or header)	not observed
Says ai-train=no in a Content Signal	not observed
Points to an RSL licence (robots.txt License line or Link header)	not observed
Answers with an x402 payment offer (lower bound: only the homepage is probed)	not observed
Answers HTTP 402 Payment Required (lower bound: only the homepage is probed)	not observed
Sends noai / noimageai directives	not observed
Refused our fetcher (403 / challenge)	no

robots.txt as recorded sha256 7a489ce247762952…

User-Agent: *
Disallow: /malyj-biznes/promo/tarif
Disallow: /search
Disallow: /o-banke/?
Disallow: /*sphrase_id=
Disallow: /o-banke/press-centr/novosti-i-press-relizy/?*&page=
Disallow: /o-banke/*?feed=
Disallow: /*utm_
Disallow: /*from=
Disallow: /*gclid=
Disallow: /*yclid=
Disallow: /404
Disallow: /*query=
Disallow: /*code=
Disallow: /*fromdomain=
Disallow: /*?$
Disallow: /*?%
Disallow: /*pid=
Disallow: /*site_switcher=
Disallow: /*ssl
Disallow: /*register=
Disallow: /*bid=
Disallow: /*s=
Disallow: /*_ga=
Disallow: /*nomobile=
Disallow: /*nomobileredirect=
Disallow: /*dialog=
Disallow: /*cc=
Disallow: /*print=
Disallow: /*/svoboda-vybora/?package
Disallow: /*homeadv=
Disallow: /*geo=
Disallow: /*ts=
Disallow: /*rb_clickid=
Disallow: /*sX=
Disallow: /*sY=
Disallow: /*s1=
Disallow: /*l=
Disallow: /*sub_id1=
Disallow: /*sub_id2=
Disallow: /*device_type=
Disallow: /*PAGEN_1=
Disallow: /*page=1$
Disallow: /*?*/
Disallow: /*city=
Disallow: /*year=
Disallow: /*month=
Disallow: /*region_code=
Disallow: /*from102=
Disallow: /*id102=
Disallow: /*ClickHash=
Disallow: /*AffiliateID=
Disallow: /vtblkl7yg3hs2vvuf*
Disallow: /vtbgjrfturehjkjkyifz*
Disallow: /malyj-biznes/promo/tarif2
Disallow: /malyj-biznes/promo/tarif1
Disallow: /malyj-biznes/promo/tarifip1
Disallow: /malyj-biznes/promo/tarifip2
Disallow: /malyj-biznes/promo/tarifllc1
Disallow: /malyj-biznes/promo/tarifllc2
Disallow: /malyj-biznes/promo/delo
Disallow: /personal/vklady-i-scheta/maksimum-380/?name=maksimum-380
Disallow: /o-prisoedinenii-banka-vtb24 
Disallow: /akcionery-i-investory/raskrytie-informacii/soobshcheniya-o-sushchestvennyh-faktah-i-raskrytie-insajderskoj-informacii
Disallow: /landing/nachnem-s-luchshego-net
Disallow: /*?id_office=
Disallow: /*?cltopic=2
Disallow: /*?year=
Disallow: /*?_escaped_fragment_=
Disallow: /*?te=
Disallow: /*?id26=
Disallow: /personal/kredit-nalichnymi/kn/?mi2=true
Disallow: /personal/kredit-nalichnymi/ref/?mi2=true
Disallow: /personal/kredit-nalichnymi/kn/
Disallow: /personal/kredit-nalichnymi/ref/

User-agent: Yandex
Disallow: /search
Disallow: /o-banke/?
Disallow: /*sphrase_id=
Disallow: /o-banke/press-centr/novosti-i-press-relizy/?*&page=
Disallow: /vtblkl7yg3hs2vvuf*
Disallow: /vtbgjrfturehjkjkyifz*
Disallow: /o-banke/*?feed=
Disallow: /404
Disallow: /*query=
Disallow: /*?$
Disallow: /*?%
Disallow: /*site_switcher=
Disallow: /*ssl
Disallow: /*register=
Disallow: /*nomobile=
Disallow: /*nomobileredirect=
Disallow: /*dialog=
Disallow: /*cc=
Disallow: /*print=
Disallow: /*/svoboda-vybora/?package
Disallow: /*geo=
Disallow: /*ts=
Disallow: /*PAGEN_1=
Disallow: /*page=1$
Disallow: /*?*/
Disallow: /malyj-biznes/promo/tarif2
Disallow: /malyj-biznes/promo/tarif1
Disallow: /malyj-biznes/promo/tarifip1
Disallow: /malyj-biznes/promo/tarifip2
Disallow: /malyj-biznes/promo/tarifllc1
Disallow: /malyj-biznes/promo/tarifllc2
Disallow: /malyj-biznes/promo/delo
Disallow: /personal/vklady-i-scheta/maksimum-380/?name=maksimum-380
Disallow: /o-prisoedinenii-banka-vtb24 
Disallow: /akcionery-i-investory/raskrytie-informacii/soobshcheniya-o-sushchestvennyh-faktah-i-raskrytie-insajderskoj-informacii
Disallow: /landing/nachnem-s-luchshego-net 
Disallow: /*?id_office=
Disallow: /*?cltopic=2
Disallow: /*?year=
Disallow: /*?_escaped_fragment_=
Disallow: /*?te=
Disallow: /*?id26=
Disallow: /personal/kredit-nalichnymi/kn/?mi2=true
Disallow: /personal/kredit-nalichnymi/ref/?mi2=true
Disallow: /personal/kredit-nalichnymi/kn/
Disallow: /personal/kredit-nalichnymi/ref/

Clean-param: ref&utm_campaign&utm_medium&utm_source&utm_term&_openstat&gclid&yclid&from&FROM&code&fromdomain&pid&bid&utm_referrer&s&_ga&homeadv&rb_clickid /
Clean-param: sX&sY&s1&s2&l&sub_id1&sub_id2&device_type&city&region_code&from102&id102&ClickHash&AffiliateID /
Host: https://www.vtb.ru
Sitemap: https://www.vtb.ru/sitemap.xml
 
User-agent: Googlebot
Disallow: /search
Disallow: /o-banke/?
Disallow: /*sphrase_id=
Disallow: /o-banke/press-centr/novosti-i-press-relizy/?*&page=
Disallow: /o-banke/*?feed=
Disallow: /*utm_
Disallow: /*from=
Disallow: /*gclid=
Disallow: /*yclid=
Disallow: /404
Disallow: /*query=
Disallow: /*code=
Disallow: /*fromdomain=
Disallow: /*?$
Disallow: /*?%
Disallow: /*pid=
Disallow: /*site_switcher=
Disallow: /*ssl
Disallow: /*register=
Disallow: /*bid=
Disallow: /*s=
Disallow: /*_ga=
Disallow: /*nomobile=
Disallow: /*nomobileredirect=
Disallow: /*dialog=
Disallow: /*cc=
Disallow: /*print=
Disallow: /*/svoboda-vybora/?package
Disallow: /*homeadv=
Disallow: /*geo=
Disallow: /*ts=
Disallow: /*rb_clickid=
Disallow: /*sX=
Disallow: /*sY=
Disallow: /*s1=
Disallow: /*l=
Disallow: /*sub_id1=
Disallow: /*sub_id2=
Disallow: /*device_type=
Disallow: /*PAGEN_1=
Disallow: /*page=1$
Disallow: /*?*/
Disallow: /*city=
Disallow: /*year=
Disallow: /*month=
Disallow: /*region_code=
Disallow: /*ClickHash=
Disallow: /*AffiliateID=
Disallow: /vtblkl7yg3hs2vvuf*
Disallow: /vtbgjrfturehjkjkyifz*
Disallow: /malyj-biznes/promo/tarif2
Disallow: /malyj-biznes/promo/tarif1
Disallow: /malyj-biznes/promo/tarifip1
Disallow: /malyj-biznes/promo/tarifip2
Disallow: /malyj-biznes/promo/tarifllc1
Disallow: /malyj-biznes/promo/tarifllc2
Disallow: /malyj-biznes/promo/delo
Disallow: /personal/vklady-i-scheta/maksimum-380/?name=maksimum-380
Disallow: /o-prisoedinenii-banka-vtb24 
Disallow: /akcionery-i-investory/raskrytie-informacii/soobshcheniya-o-sushchestvennyh-faktah-i-raskrytie-insajderskoj-informacii
Disallow: /landing/nachnem-s-luchshego-net
Disallow: /*?id_office=
Disallow: /*?cltopic=2
Disallow: /*?year=
Disallow: /*?_escaped_fragment_=
Disallow: /*?te=
Disallow: /*?id26=
Disallow: /personal/kredit-nalichnymi/kn/?mi2=true
Disallow: /personal/kredit-nalichnymi/ref/?mi2=true
Disallow: /personal/kredit-nalichnymi/kn/
Disallow: /personal/kredit-nalichnymi/ref/

User-agent: AdsBot-Google
Allow: /malyj-biznes/promo/tarif
 
User-agent: YaDirectFetcher
Allow: /malyj-biznes/promo/tarif
 
User-agent: GoogleBot
Disallow: /malyj-biznes/promo/tarif
 
User-agent: YandexBot
Disallow: /malyj-biznes/promo/tarif


Sitemap: https://www.vtb.ru/sitemap.xml

robots.txt versions

From	Through	State	Version
2026-10-26	2026-10-28	ok 200	`6856255165e0`

Timeline of changes

Date	What changed
No changes since the first observation (2026-10-26).

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Crawl	Host	State	GPTBot	OAI-SearchBot	ChatGPT-User	ClaudeBot	Claude-SearchBot	Claude-User	PerplexityBot
2026-09-21 CC-MAIN-2026-39	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2025-09-15 CC-MAIN-2025-38	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2024-09-16 CC-MAIN-2024-38	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2023-10-02 CC-MAIN-2023-40	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2021-09-27 CC-MAIN-2021-39	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2020-09-28 CC-MAIN-2020-40	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2019-09-23 CC-MAIN-2019-39	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2018-09-24 CC-MAIN-2018-39	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2018-01-29 CC-MAIN-2018-05	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-10-23 CC-MAIN-2017-43	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-09-25 CC-MAIN-2017-39	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-07-24 CC-MAIN-2017-30	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-04-24 CC-MAIN-2017-17	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-01-23 CC-MAIN-2017-04	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2016-10-03 CC-MAIN-2016-40	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2016-09-05 CC-MAIN-2016-36	www.vtb.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows

Evidence

Every daily observation of vtb.ru is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-28: audit path (JSON), day root and OpenTimestamps proof (pending (4 calendars)).

API: https://infrarabbit.com/crawlers/api/v1/policy/vtb.ru