samsung.com: what it tells AI crawlers

On 27 October 2026, samsung.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot; it also published llms.txt.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel
top 1k (Tranco)
robots.txt
ok (HTTP 200) · https://www.samsung.com/robots.txt
unchanged since 2026-10-26 · sha256 e07ce06f3220d049… · view as recorded
The * group
allows applies to every agent without its own group

Per AI bot, 2026-10-27

User agentVerdictHow
GPTBotallowsnamed in its own group
OAI-SearchBotallowsnamed in its own group
ChatGPT-Userallowsnamed in its own group
ClaudeBotallowsthrough the * group
Claude-SearchBotallowsthrough the * group
Claude-Userallowsthrough the * group
PerplexityBotallowsnamed in its own group
Perplexity-Userallowsthrough the * group
Googlebotallowsnamed in its own group
Google-Extendedallowsnamed in its own group
Applebot-Extendedallowsthrough the * group
CCBotallowsthrough the * group
Bytespiderallowsthrough the * group
Meta-ExternalAgentallowsthrough the * group
Amazonbotallowsthrough the * group

Other signals

Publishes /llms.txtyes
Publishes /ai.txtno
Publishes /.well-known/tdmrep.jsonno
Reserves TDM rights (TDMRep file, header or meta)no
Declares Content Signals (robots.txt or header)no
Says ai-train=no in a Content Signalno
Points to an RSL licence (robots.txt License line or Link header)no
Answers with an x402 payment offer (lower bound: only the homepage is probed)no
Answers HTTP 402 Payment Required (lower bound: only the homepage is probed)no
Sends noai / noimageai directivesno
Refused our fetcher (403 / challenge)no

llms.txt: "Samsung Electronics", 4,597 bytes, 31 links.

robots.txt as recorded sha256 14df4935f7841e5b…

# Allow web crolling by search engine for all directory
# target server : www

User-agent: *
Disallow: /*.notfound.html
Disallow: /*.redirect.html
Disallow: /ru/*printPage.sic
Disallow: /ru/*share-with-colleagues-form.do?
Disallow: /ru/*productcompare
Disallow: /ru/*search
Disallow: /ru/*?group=*
Disallow: /uk/pdf/
Disallow: /ie/pdf/
Disallow: /uk/aprimoemail/
Disallow: /uk/*productcompare
Disallow: /us/dealer/*
Disallow: /de/api/*
Disallow: /de/business/api/*
Disallow: /de/system/*
Disallow: /*/c/p/*
Disallow: /*/api/*
Disallow: /*/system/*
Disallow: /es/api/*
Disallow: /9hours
Disallow: /*/search/
Disallow: /*/business/search/
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-EDGEPACK1
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-EDGEPACK2
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-EDGEPACK3
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL1
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL2
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL3
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL4
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL5
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL6
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL7
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL8
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL9
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL10
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL11
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-MARVEL12
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-THOR
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-IRONMAN
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-CA
Disallow: /uk/consumer/mobile-devices/store-accessories/licensed-accessories/F-HULK
Disallow: /uk/*.pdf$
Disallow: /fr/*.pdf$
Disallow: /fr/*.doc$
Disallow: /fr/*.docx$
Disallow: /fr/*.ppt$
Disallow: /fr/*.pptx$
Disallow: /uk/consumer/mobile-devices/uk-shop/edge/SM-G925FZDEXEU
Disallow: /uk/consumer/mobile-devices/uk-shop/edge/SM-G925FZKEXEU
Disallow: /uk/consumer/mobile-devices/uk-shop/edge/SM-G925FZWEXEU
Disallow: /smartphone/galaxys/video/*.swf$ 
Disallow: /smartphone/galaxy-s-2/video/*.swf$ 
Disallow: /smartphone/galaxy-tab/video/*.swf$ 
Disallow: /smartphone/omnia/icon/*.swf$ 
Disallow: /smartphone/omnia/pro/*.swf$ 
Disallow: /smartphone/smartphone/wave/video/*.swf$ 
Disallow: /smartphone/technology/video/*.swf$ 
Disallow: /smartphone/navigon/XML_Content/*.xml 
Disallow: /uk/consumer/pc-peripherals/store-displays/hospitality-displays/HG55EC690EBXXU
Disallow: /uk/consumer/pc-peripherals/store-displays/hospitality-displays/HG40EC670CWXXU
Disallow: /uk/consumer/pc-peripherals/store-displays/hospitality-displays/HG32EC470GWXXU
Disallow: /ae/news/null/
Disallow: /ae_ar/news/null/
Disallow: /ae/*.do
Disallow: /ae_ar/*.do
Disallow: /ae/*.swf
Disallow: /ae_ar/*.swf
Disallow: /us/shop/837
Disallow: /us/shop/837/*

User-agent: Yandex
Disallow: /us* 
Disallow: /uk* 
Disallow: /ca* 
Disallow: /sg* 

Sitemap: http://www.samsung.com/sitemap.xml

robots.txt versions

FromThroughStateVersion
2026-10-262026-10-28ok 200e07ce06f3220

Timeline of changes

DateWhat changed
No changes since the first observation (2026-10-26).

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

CrawlHostStateGPTBotOAI-SearchBotChatGPT-UserClaudeBotClaude-SearchBotClaude-UserPerplexityBot
2026-09-21 CC-MAIN-2026-39www.samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2025-09-15 CC-MAIN-2025-38www.samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2024-09-16 CC-MAIN-2024-38www.samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2023-10-02 CC-MAIN-2023-40www.samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2022-10-03 CC-MAIN-2022-40www.samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2021-09-27 CC-MAIN-2021-39www.samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2020-09-28 CC-MAIN-2020-40www.samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2019-09-23 CC-MAIN-2019-39www.samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2018-09-24 CC-MAIN-2018-39www.samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2018-04-23 CC-MAIN-2018-17www.samsung.comok viewblocksblocksunknownblocksblocksunknownblocks
2018-01-29 CC-MAIN-2018-05www.samsung.comok viewblocksblocksunknownblocksblocksunknownblocks
2017-10-23 CC-MAIN-2017-43www.samsung.comok viewblocksblocksunknownblocksblocksunknownblocks
2017-09-25 CC-MAIN-2017-39www.samsung.comok viewblocksblocksunknownblocksblocksunknownblocks
2017-07-24 CC-MAIN-2017-30www.samsung.comok viewblocksblocksunknownblocksblocksunknownblocks
2017-04-24 CC-MAIN-2017-17samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2017-01-23 CC-MAIN-2017-04samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2016-10-03 CC-MAIN-2016-40samsung.comok viewallowsallowsunknownallowsallowsunknownallows
2016-09-05 CC-MAIN-2016-36samsung.comok viewallowsallowsunknownallowsallowsunknownallows

Evidence

Every daily observation of samsung.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/samsung.com