lg.com: what it tells AI crawlers
On 27 October 2026, lg.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot; it also published llms.txt.
Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.
- Panel
- top 1k (Tranco)
- robots.txt
- ok (HTTP 200) · https://www.lg.com:443/robots.txt
unchanged since 2026-10-26 · sha25646035ac91b468df4…· view as recorded - The * group
- allows applies to every agent without its own group
Per AI bot, 2026-10-27
| User agent | Verdict | How |
|---|---|---|
| GPTBot | allows | through the * group |
| OAI-SearchBot | allows | through the * group |
| ChatGPT-User | allows | through the * group |
| ClaudeBot | allows | through the * group |
| Claude-SearchBot | allows | through the * group |
| Claude-User | allows | through the * group |
| PerplexityBot | allows | through the * group |
| Perplexity-User | allows | through the * group |
| Googlebot | allows | through the * group |
| Google-Extended | allows | through the * group |
| Applebot-Extended | allows | through the * group |
| CCBot | allows | through the * group |
| Bytespider | allows | through the * group |
| Meta-ExternalAgent | allows | through the * group |
| Amazonbot | allows | through the * group |
Other signals
| Publishes /llms.txt | yes |
| Publishes /ai.txt | no |
| Publishes /.well-known/tdmrep.json | no |
| Reserves TDM rights (TDMRep file, header or meta) | no |
| Declares Content Signals (robots.txt or header) | no |
| Says ai-train=no in a Content Signal | no |
| Points to an RSL licence (robots.txt License line or Link header) | no |
| Answers with an x402 payment offer (lower bound: only the homepage is probed) | no |
| Answers HTTP 402 Payment Required (lower bound: only the homepage is probed) | no |
| Sends noai / noimageai directives | no |
| Refused our fetcher (403 / challenge) | no |
llms.txt: "LG Electronics (lg.com) – Official Content Guide for AI (Global Hub)", 2,253 bytes, 11 links.
robots.txt as recorded sha256 94c4903123571a69…
User-agent: * Allow: / Disallow: Disallow: /*/*.ldwf* Disallow: /*/support/product-help/doc* Disallow: /*/support/videos/videos-tutorials-layer* Disallow: /*/support/error-pages Disallow: /error Disallow: /common/bug Disallow: /common/content_* Disallow: /common/error Disallow: /common/search/ Disallow: /common/support Disallow: /templatedata Disallow: /war Disallow: /download/support/contactus Disallow: /download/support Disallow: /*.ajax$ Disallow: /us/about-lg/investor-relations/ Disallow: /*gclid Disallow: /us/includes/ Disallow: /us/cms-templates/ Disallow: /us/common/ Disallow: /us/compare/ Disallow: /us/sys/ Disallow: /us/application-data Disallow: /us/secure Disallow: /*/products/wtb* Disallow: /compare.lg?* Disallow: /*/*.swf$ Disallow: /mylg/ Disallow: /flash/ Disallow: /*/*/reviews/product/* Disallow: /lgecs.downloadFile.ldwf?* Disallow: /es/HEProductExperiencePage Disallow: /es/HEProductExperiencePage/ Disallow: /es/HEProductExperiencePage/tv/main Disallow: /*/srch* Disallow: /*?search=* Disallow: /*?bvrrp=* Disallow: /*_escaped_fragment* Disallow: /*?lgeurl=* Disallow: /*/response/* Disallow: /*/popup/* Disallow: /*/search.lg* Disallow: /au/pdfs/* Disallow: /nz/pdfs/* Disallow: /*.jsp?subCatId Disallow: /*.jsp?value Disallow: /*.jsp?contextPath Disallow: /*.jsp?ref_no Disallow: /*.jsp?s_kwcid Disallow: /*.jsp?cmpid Disallow: /jp/support/repair-warranty/locate-repair-center-detail/* Disallow: /jp/mobile-phones/download-page/* User-agent: Baiduspider Disallow: /*/wtb?modelName=* Disallow: /*/my-support Disallow: /*/mylg-login Disallow: /*?cmpid Disallow: /*/player?videoId= Disallow: /*/embedded-player?videoId= User-agent: Yandex Allow: / Disallow: /error Disallow: /common/bug Disallow: /common/content_* Disallow: /common/error Disallow: /common/support Disallow: /templatedata Disallow: /war Disallow: /download/support/contactus Disallow: /adm Disallow: /Admin Disallow: /download/support Disallow: /*.ajax$ Disallow: /us/about-lg/investor-relations/ Disallow: /es/HEProductExperiencePage Disallow: /es/HEProductExperiencePage/ Disallow: /es/HEProductExperiencePage/tv/main Disallow: /*cmpid* Disallow: /*product-help/CT Disallow: /*locate-repair-center-detail/ Disallow: /*video-tutorials/CT Disallow: /*utm_ Disallow: /*/search.lg* Disallow: /*/response/* Disallow: /*?search=* Disallow: /*/popup/* Disallow: /*?bvrrp=* Disallow: /*/srch* Clean-param: utm_source&utm_medium&utm_term&utm_campaign&utm_content / Clean-param: gclid / Clean-param:countryCd /*.jsp Clean-param:subCatId /*.jsp Clean-param:value /*.jsp Clean-param:contextPath /*.jsp Clean-param:ref_no /*.jspA Clean-param:s_kwcid /*.jsp Clean-param:cmpid /*.jsp # LGEUS-744 User-agent: Googlebot Disallow: User-agent: Googlebot-image Disallow: # LGEUS-744 # Sitemap files Sitemap: https://www.lg.com/sitemap.xml Host: www.lg.com User-agent: Mediapartners-Google Allow:/*.jsp?cmpid User-agent: AdsBot-Google Allow: /*.jsp?cmpid User-agent: AdsBot-Google-Mobile Allow: /*.jsp?cmpid
robots.txt versions
| From | Through | State | Version | |
|---|---|---|---|---|
| 2026-10-26 | 2026-10-28 | ok 200 | 46035ac91b46 |
Timeline of changes
| Date | What changed |
|---|---|
| No changes since the first observation (2026-10-26). | |
Reconstructed history (Common Crawl)
From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.
| Crawl | Host | State | GPTBot | OAI-SearchBot | ChatGPT-User | ClaudeBot | Claude-SearchBot | Claude-User | PerplexityBot |
|---|---|---|---|---|---|---|---|---|---|
| 2026-09-21 CC-MAIN-2026-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2025-09-15 CC-MAIN-2025-38 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2024-09-16 CC-MAIN-2024-38 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2023-10-02 CC-MAIN-2023-40 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2022-10-03 CC-MAIN-2022-40 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2021-09-27 CC-MAIN-2021-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2020-09-28 CC-MAIN-2020-40 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2019-09-23 CC-MAIN-2019-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-09-24 CC-MAIN-2018-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-04-23 CC-MAIN-2018-17 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-01-29 CC-MAIN-2018-05 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-10-23 CC-MAIN-2017-43 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-09-25 CC-MAIN-2017-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-07-24 CC-MAIN-2017-30 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-04-24 CC-MAIN-2017-17 | lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-01-23 CC-MAIN-2017-04 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-10-03 CC-MAIN-2016-40 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-09-05 CC-MAIN-2016-36 | lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
Evidence
Every daily observation of lg.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).