ozon.ru: what it tells AI crawlers

On 27 October 2026, ozon.ru's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel: top 1k (Tranco)
robots.txt: ok (HTTP 200) · https://www.ozon.ru/robots.txt?__rr=1
unchanged since 2026-10-26 · sha256 661b8425037042a6… · view as recorded
The * group: partial applies to every agent without its own group

Per AI bot, 2026-10-27

User agent	Verdict	How
GPTBot	partial	through the * group
OAI-SearchBot	partial	through the * group
ChatGPT-User	partial	through the * group
ClaudeBot	partial	through the * group
Claude-SearchBot	partial	through the * group
Claude-User	partial	through the * group
PerplexityBot	partial	through the * group
Perplexity-User	partial	through the * group
Googlebot	partial	named in its own group
Google-Extended	partial	through the * group
Applebot-Extended	partial	through the * group
CCBot	partial	through the * group
Bytespider	partial	through the * group
Meta-ExternalAgent	partial	through the * group
Amazonbot	partial	through the * group

Other signals

Publishes /llms.txt	not observed
Publishes /ai.txt	not observed
Publishes /.well-known/tdmrep.json	not observed
Reserves TDM rights (TDMRep file, header or meta)	not observed
Declares Content Signals (robots.txt or header)	not observed
Says ai-train=no in a Content Signal	not observed
Points to an RSL licence (robots.txt License line or Link header)	not observed
Answers with an x402 payment offer (lower bound: only the homepage is probed)	not observed
Answers HTTP 402 Payment Required (lower bound: only the homepage is probed)	not observed
Sends noai / noimageai directives	not observed
Refused our fetcher (403 / challenge)	no

robots.txt as recorded sha256 70ec216c575894db…

User-agent: Yandex
Disallow: /*?appointmentmouse
Disallow: /*?apsource
Disallow: /*?assignmentcover
Disallow: /*?decortype
Disallow: /*?diameterfilter
Disallow: /*?fileformats
Disallow: /*?foreignlanguage
Disallow: /*?formfactor
Disallow: /*?headphoneout
Disallow: /*?images
Disallow: /*?inductionhob
Disallow: /*?movie
Disallow: /*?opticalpower
Disallow: /*?packing
Disallow: /*?presenceautoanswer
Disallow: /*?presencewatch
Disallow: /*?radius
Disallow: /*?remotecontrol
Disallow: /*?size
Disallow: /*?typetransform
#Custom
Disallow: /action/*/?
Disallow: /*_catalog/
Disallow: /*?catalog_select
Disallow: /context/*/catalog
Disallow: /context/*/?catalog=
Disallow: /context/catalog/id/*/*?
Disallow: /context/detail/id/*/_
Disallow: /*add2navigator=
Disallow: /*.aspx
Disallow: /*.cfm
Disallow: /*context=search
Disallow: /*context=catalog
Disallow: /*context=info
Disallow: /*context/info
Disallow: /*context=mailmaster
Disallow: /*context=compare
Disallow: /context/*/?action=addform
Disallow: /?context=tag_list&TagName=
Disallow: /?context=demand
Disallow: /?context=list_add
Disallow: /?context=merchant_offer
Disallow: /?context=show&id=
Disallow: /?context=showdate_theater&eventday=
Disallow: /?context=book_search_list&searchstring=
Disallow: /?context=video_search_list&searchstring=
Disallow: /?context=ticket_ozon&performance=
Disallow: /?context=client_wish_list
Disallow: /context/login
Disallow: /context/newuser
Disallow: /context/registrationle/
Disallow: /*merchantinfo2
Disallow: /context/client_wish
Disallow: /context/myadvice
Disallow: /context/*_auction/
Disallow: /*is_discounted=1
Disallow: /context/detail/id/$
Disallow: /person/*catalog=
Disallow: /person/*brand=
Disallow: /person/*group=
Disallow: /person/*page=
Disallow: /catalog/1180334/?sporttype=
#SinglePages
Disallow: /context/div_travel/
Disallow: /graphics/
Disallow: /context/paritylist/
Disallow: /context/boutique/
Disallow: /context/div_import/
Disallow: /context/mailmaster/
Disallow: /context/partner_contract/
Disallow: /context/techauction/
Disallow: /multimedia/
Disallow: /webservice/
Disallow: /webservices/
Disallow: /download/
Disallow: /context/goods-sale/
Disallow: /context/discount-goods/
Disallow: /lego/
Disallow: /context/happy_birthday/
Disallow: /context/client/
Disallow: /context/client_opinion/
Disallow: /context/detail/id/1687456/
Disallow: /context/detail/id/2404375/
Disallow: /context/detail/id/4116707/
Disallow: /context/detail/id/1816074/
Disallow: /context/detail/id/3625907/
Disallow: /context/detail/id/4492894/
Disallow: /context/detail/id/2996092/
Disallow: /context/detail/id/2445930/
Disallow: /context/detail/id/2787791/
Disallow: /context/partner_xml/
Disallow: /context/compare/
Disallow: /context/detail/id/3845181/
Disallow: /context/detail/id/200430/
Disallow: /context/detail/id/2928145/
Disallow: /context/detail/id/1687637/
Disallow: /context/registrationle/
Disallow: /context/detail/id/1687456/
Disallow: /context/detail/id/4792641/
#Params
Disallow: /*_gs_
Disallow: /*aspectratio
Disallow: /*bests=
Disallow: /*?group=
Allow: /brand/*?group=
Disallow: /*discount=
Disallow: /*item_seria_id=
Disallow: /*isspecialprice=
Disallow: /*new=
Disallow: /*price=
Disallow: /*rating=
Disallow: /*sale=
Disallow: /*store=
Disallow: /*subscribe=
Disallow: /*text=
#Markers
Disallow: /*from=
Disallow: /*frommarket=
Disallow: /*gclid=
Disallow: /*marketingcode=
Disallow: /*merchant=
Disallow: /*openstat=
Disallow: /*partneragentid=
Disallow: /*perehod=
Disallow: /*PHPSESSID=
Disallow: /*?ref=
Clean-param: utm&ref /catalog/
Clean-param: utm&ref /context/detail/
Clean-param: partner&rating&sort
Host: www.ozon.ru
Sitemap: http://www.ozon.ru/SitemapIndex.xml

User-agent: Googlebot
Disallow: /*?appointmentmouse
Disallow: /*?apsource
Disallow: /*?assignmentcover
Disallow: /*?decortype
Disallow: /*?diameterfilter
Disallow: /*?fileformats
Disallow: /*?foreignlanguage
Disallow: /*?formfactor
Disallow: /*?headphoneout
Disallow: /*?images
Disallow: /*?inductionhob
Disallow: /*?movie
Disallow: /*?opticalpower
Disallow: /*?packing
Disallow: /*?presenceautoanswer
Disallow: /*?presencewatch
Disallow: /*?radius
Disallow: /*?remotecontrol
Disallow: /*?size
Disallow: /*?typetransform
#Custom
Disallow: /action/*/?
Disallow: /*_catalog/
Disallow: /*?catalog_select
Disallow: /context/*/catalog
Disallow: /context/*/?catalog=
Disallow: /context/catalog/id/*/*?
Disallow: /context/detail/id/*/_
Disallow: /*add2navigator=
Disallow: /*.aspx
Disallow: /*.cfm
Disallow: /*.html
Disallow: /*context=search
Disallow: /*context=catalog
Disallow: /*context=info
Disallow: /*context/info
Disallow: /*context=mailmaster
Disallow: /*context=compare
Disallow: /context/*/?action=addform
Disallow: /?context=tag_list&TagName=
Disallow: /?context=demand
Disallow: /?context=list_add
Disallow: /?context=merchant_offer
Disallow: /?context=show&id=
Disallow: /?context=showdate_theater&eventday=
Disallow: /?context=book_search_list&searchstring=
Disallow: /?context=video_search_list&searchstring=
Disallow: /?context=ticket_ozon&performance=
Disallow: /?context=client_wish_list
Disallow: /context/login
Disallow: /context/newuser
Disallow: /context/registrationle/
Disallow: /*merchantinfo2
Disallow: /context/client_wish
Disallow: /context/myadvice
Disallow: /context/*_auction/
Disallow: /*is_discounted=1
Disallow: /context/detail/id/$
Disallow: /person/*catalog=
Disallow: /person/*brand=
Disallow: /person/*group=
Disallow: /person/*page=
Disallow: /catalog/1180334/?sporttype=
#SinglePages
Disallow: /context/div_travel/
Disallow: /graphics/
Disallow: /context/paritylist/
Disallow: /context/boutique/
Disallow: /context/div_import/
Disallow: /context/mailmaster/
Disallow: /context/partner_contract/
Disallow: /context/techauction/
Disallow: /multimedia/
Disallow: /webservice/
Disallow: /webservices/
Disallow: /download/
Disallow: /context/goods-sale/
Disallow: /context/discount-goods/
Disallow: /lego/
Disallow: /context/happy_birthday/
Disallow: /context/client/
Disallow: /context/client_opinion/
Disallow: /context/detail/id/1687456/
Disallow: /context/detail/id/2404375/
Disallow: /context/detail/id/4116707/
Disallow: /context/detail/id/1816074/
Disallow: /context/detail/id/3625907/
Disallow: /context/detail/id/4492894/
Disallow: /context/detail/id/2996092/
Disallow: /context/detail/id/2445930/
Disallow: /context/detail/id/2787791/
Disallow: /context/partner_xml/
Disallow: /context/compare/
Disallow: /context/detail/id/3845181/
Disallow: /context/detail/id/200430/
Disallow: /context/detail/id/2928145/
Disallow: /context/detail/id/1687637/
Disallow: /context/registrationle/
Disallow: /context/detail/id/1687456/
Disallow: /context/detail/id/4792641/
#Params
Disallow: /*_gs_
Disallow: /*aspectratio
Disallow: /*bests=
Disallow: /*?group=
Allow: /brand/*?group=
Disallow: /*isspecialprice=
Disallow: /*new=
Disallow: /*price=
Disallow: /*rating=
Disallow: /*sale=
Disallow: /*sort=
Disallow: /*store=
Disallow: /*subscribe=
Disallow: /*text=
#Markers
Disallow: /*from=
Disallow: /*frommarket=
Disallow: /*gclid=
Disallow: /*marketingcode=
Disallow: /*merchant=
Disallow: /*openstat=
Disallow: /*perehod=
Disallow: /*PHPSESSID=
Disallow: /*?ref=
Sitemap: http://www.ozon.ru/SitemapIndex.xml

User-agent: *
Disallow: /*?appointmentmouse
Disallow: /*?apsource
Disallow: /*?assignmentcover
Disallow: /*?decortype
Disallow: /*?diameterfilter
Disallow: /*?fileformats
Disallow: /*?foreignlanguage
Disallow: /*?formfactor
Disallow: /*?headphoneout
Disallow: /*?images
Disallow: /*?inductionhob
Disallow: /*?movie
Disallow: /*?opticalpower
Disallow: /*?packing
Disallow: /*?presenceautoanswer
Disallow: /*?presencewatch
Disallow: /*?radius
Disallow: /*?remotecontrol
Disallow: /*?size
Disallow: /*?typetransform
#Custom
Disallow: /action/*/?
Disallow: /*_catalog/
Disallow: /*?catalog_select
Disallow: /context/*/catalog
Disallow: /context/*/?catalog=
Disallow: /context/catalog/id/*/*?
Disallow: /context/detail/id/*/_
Disallow: /*add2navigator=
Disallow: /*.aspx
Disallow: /*.cfm
Disallow: /*context=search
Disallow: /*context=catalog
Disallow: /*context=info
Disallow: /*context/info
Disallow: /*context=mailmaster
Disallow: /*context=compare
Disallow: /context/*/?action=addform
Disallow: /?context=tag_list&TagName=
Disallow: /?context=demand
Disallow: /?context=list_add
Disallow: /?context=merchant_offer
Disallow: /?context=show&id=
Disallow: /?context=showdate_theater&eventday=
Disallow: /?context=book_search_list&searchstring=
Disallow: /?context=video_search_list&searchstring=
Disallow: /?context=ticket_ozon&performance=
Disallow: /?context=client_wish_list
Disallow: /context/login
Disallow: /context/newuser
Disallow: /context/registrationle/
Disallow: /*merchantinfo2
Disallow: /context/client_wish
Disallow: /context/myadvice
Disallow: /context/*_auction/
Disallow: /*is_discounted=1
Disallow: /context/detail/id/$
Disallow: /person/*catalog=
Disallow: /person/*brand=
Disallow: /person/*group=
Disallow: /person/*page=
#SinglePages
Disallow: /context/div_travel/
Disallow: /graphics/
Disallow: /context/paritylist/
Disallow: /context/boutique/
Disallow: /context/div_import/
Disallow: /context/mailmaster/
Disallow: /context/partner_contract/
Disallow: /context/techauction/
Disallow: /multimedia/
Disallow: /webservice/
Disallow: /webservices/
Disallow: /download/
Disallow: /context/goods-sale/
Disallow: /context/discount-goods/
Disallow: /lego/
Disallow: /context/happy_birthday/
Disallow: /context/client/
Disallow: /context/client_opinion/
Disallow: /context/detail/id/1687456/
Disallow: /context/detail/id/2404375/
Disallow: /context/detail/id/4116707/
Disallow: /context/detail/id/1816074/
Disallow: /context/detail/id/3625907/
Disallow: /context/detail/id/4492894/
Disallow: /context/detail/id/2996092/
Disallow: /context/detail/id/2445930/
Disallow: /context/detail/id/2787791/
Disallow: /context/partner_xml/
Disallow: /context/compare/
Disallow: /context/detail/id/3845181/
Disallow: /context/detail/id/200430/
Disallow: /context/detail/id/2928145/
Disallow: /context/detail/id/1687637/
Disallow: /context/registrationle/
Disallow: /context/detail/id/1687456/
Disallow: /context/detail/id/4792641/
#Params
Disallow: /*_gs_
Disallow: /*aspectratio
Disallow: /*bests=
Disallow: /*?group=
Allow: /brand/*?group=
Disallow: /*isspecialprice=
Disallow: /*new=
Disallow: /*price=
Disallow: /*rating=
Disallow: /*sale=
Disallow: /*sort=
Disallow: /*store=
Disallow: /*subscribe=
Disallow: /*text=
#Markers
Disallow: /*from=
Disallow: /*frommarket=
Disallow: /*marketingcode=
Disallow: /*merchant=
Disallow: /*openstat=
Disallow: /*perehod=
Disallow: /*PHPSESSID=
Disallow: /*?ref=

robots.txt versions

From	Through	State	Version
2026-10-26	2026-10-28	ok 200	`661b84250370`

Timeline of changes

Date	What changed
No changes since the first observation (2026-10-26).

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Crawl	Host	State	GPTBot	OAI-SearchBot	ChatGPT-User	ClaudeBot	Claude-SearchBot	Claude-User	PerplexityBot
2026-09-21 CC-MAIN-2026-39	ozon.ru	redirect-only	unknown	unknown	unknown	unknown	unknown	unknown	unknown
2025-09-15 CC-MAIN-2025-38	ozon.ru	redirect-only	unknown	unknown	unknown	unknown	unknown	unknown	unknown
2024-09-16 CC-MAIN-2024-38	ozon.ru	redirect-only	unknown	unknown	unknown	unknown	unknown	unknown	unknown
2023-10-02 CC-MAIN-2023-40	www.ozon.ru	ok view	partial	partial	unknown	partial	partial	unknown	partial
2022-10-03 CC-MAIN-2022-40	www.ozon.ru	ok view	partial	partial	unknown	partial	partial	unknown	partial
2021-09-27 CC-MAIN-2021-39	www.ozon.ru	ok view	partial	partial	unknown	partial	partial	unknown	partial
2020-09-28 CC-MAIN-2020-40	www.ozon.ru	ok view	partial	partial	unknown	partial	partial	unknown	partial
2019-09-23 CC-MAIN-2019-39	www.ozon.ru	ok view	partial	partial	unknown	partial	partial	unknown	partial
2018-09-24 CC-MAIN-2018-39	www.ozon.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2018-01-29 CC-MAIN-2018-05	www.ozon.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-10-23 CC-MAIN-2017-43	www.ozon.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-09-25 CC-MAIN-2017-39	www.ozon.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-07-24 CC-MAIN-2017-30	www.ozon.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-04-24 CC-MAIN-2017-17	ozon.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2017-01-23 CC-MAIN-2017-04	ozon.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2016-10-03 CC-MAIN-2016-40	ozon.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows
2016-09-05 CC-MAIN-2016-36	ozon.ru	ok view	allows	allows	unknown	allows	allows	unknown	allows

Evidence

Every daily observation of ozon.ru is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/ozon.ru