lg.com: what it tells AI crawlers
On 27 October 2026, lg.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot; it also published llms.txt.
Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.
- Panel
- top 1k (Tranco)
- robots.txt
- ok (HTTP 200) · https://www.lg.com:443/robots.txt
unchanged since 2026-10-26 · sha25646035ac91b468df4…· view as recorded - The * group
- allows applies to every agent without its own group
Per AI bot, 2026-10-27
| User agent | Verdict | How |
|---|---|---|
| GPTBot | allows | through the * group |
| OAI-SearchBot | allows | through the * group |
| ChatGPT-User | allows | through the * group |
| ClaudeBot | allows | through the * group |
| Claude-SearchBot | allows | through the * group |
| Claude-User | allows | through the * group |
| PerplexityBot | allows | through the * group |
| Perplexity-User | allows | through the * group |
| Googlebot | allows | through the * group |
| Google-Extended | allows | through the * group |
| Applebot-Extended | allows | through the * group |
| CCBot | allows | through the * group |
| Bytespider | allows | through the * group |
| Meta-ExternalAgent | allows | through the * group |
| Amazonbot | allows | through the * group |
Other signals
| Publishes /llms.txt | yes |
| Publishes /ai.txt | no |
| Publishes /.well-known/tdmrep.json | no |
| Reserves TDM rights (TDMRep file, header or meta) | no |
| Declares Content Signals (robots.txt or header) | no |
| Says ai-train=no in a Content Signal | no |
| Points to an RSL licence (robots.txt License line or Link header) | no |
| Answers with an x402 payment offer (lower bound: only the homepage is probed) | no |
| Answers HTTP 402 Payment Required (lower bound: only the homepage is probed) | no |
| Sends noai / noimageai directives | no |
| Refused our fetcher (403 / challenge) | no |
llms.txt: "LG Electronics (lg.com) – Official Content Guide for AI (Global Hub)", 2,253 bytes, 11 links.
robots.txt as recorded sha256 0c157911c36cbb3c…
User-agent: * Allow: / Disallow: /*/support/feedback?* Disallow: /*/search*?* Disallow: /*/*.ldwf* Disallow: /*/support/product-help/doc* Disallow: /*/support/videos/videos-tutorials-layer* Disallow: /*/support/error-pages Disallow: /error Disallow: /common/bug Disallow: /common/content_* Disallow: /common/error Disallow: /common/search/ Disallow: /common/support Disallow: /templatedata Disallow: /war Disallow: /download/support/contactus Disallow: /download/support Disallow: /*.ajax$ Disallow: /us/about-lg/investor-relations/ Disallow: /*gclid Disallow: /us/includes/ Disallow: /us/cms-templates/ Disallow: /us/common/ Disallow: /us/compare/ Disallow: /us/sys/ Disallow: /us/application-data Disallow: /us/secure Disallow: /*/products/wtb* Disallow: /compare.lg?* Disallow: /*/*.swf$ Disallow: /mylg/ Disallow: /flash/ Disallow: /*/*/reviews/product/* Disallow: /lgecs.downloadFile.ldwf?* Disallow: /es/HEProductExperiencePage Disallow: /es/HEProductExperiencePage/ Disallow: /es/HEProductExperiencePage/tv/main Disallow: /*/srch* Disallow: /*?search=* Disallow: /*?bvrrp=* Disallow: /*_escaped_fragment* Disallow: /*?lgeurl=* Disallow: /*/response/* Disallow: /*/popup/* Disallow: /*/search.lg* Disallow: /au/pdfs/* Disallow: /nz/pdfs/* Disallow: /*.jsp?subCatId Disallow: /*.jsp?value Disallow: /*.jsp?contextPath Disallow: /*.jsp?ref_no Disallow: /*.jsp?s_kwcid Disallow: /*.jsp?cmpid Disallow: /jp/support/repair-warranty/locate-repair-center-detail/* Disallow: /jp/mobile-phones/download-page/* Disallow: /it/convenzioni-aziendali/corporate-benefit* Disallow: /lg5-common-gp/capcha/botdetectcaptcha Disallow: /WEB-INF/jsp/common/error/error404.jsp Disallow:/*?from=* Disallow:/*?awc=* Disallow: /*?category=* Disallow: /*?amp=* Disallow: /*?code=* Disallow: /*?cmpid=* Disallow: /*?utm_ Disallow: /*?sub_ Disallow: /*?sa=* Disallow: /*?element=* Disallow: /*?model=* Disallow: /*?bizType=* Disallow: /*?cjevent=* Disallow: /*?csSalesCode=* Disallow: /*?emptyModel=* User-agent: Baiduspider Disallow: /*/wtb?modelName=* Disallow: /*/my-support Disallow: /*/mylg-login Disallow: /*?cmpid Disallow: /*/player?videoId= Disallow: /*/embedded-player?videoId= User-agent: Yandex Allow: / Disallow: /error Disallow: /common/bug Disallow: /common/content_* Disallow: /common/error Disallow: /common/support Disallow: /templatedata Disallow: /war Disallow: /download/support/contactus Disallow: /adm Disallow: /Admin Disallow: /download/support Disallow: /*.ajax$ Disallow: /us/about-lg/investor-relations/ Disallow: /es/HEProductExperiencePage Disallow: /es/HEProductExperiencePage/ Disallow: /es/HEProductExperiencePage/tv/main Disallow: /*locate-repair-center-detail/ Disallow: /*video-tutorials/CT Disallow: /*utm_ Disallow: /*/search.lg* Disallow: /*/response/* Disallow: /*?search=* Disallow: /*/popup/* Disallow: /*?bvrrp=* Disallow: /*/srch* Disallow: /search-all?* Clean-param: utm_source&utm_medium&utm_term&utm_campaign&utm_content / Clean-param: gclid / Clean-param:countryCd /*.jsp Clean-param:subCatId /*.jsp Clean-param:value /*.jsp Clean-param:contextPath /*.jsp Clean-param:ref_no /*.jspA Clean-param:s_kwcid /*.jsp Clean-param:cmpid /*.jsp Clean-param:element&model&bizType&cjevent&csSalesCode Clean-param:modelid&ref&code&&awc&from&category&cmpid&emptyModel&id / Clean-param:elqat Clean-param:_gl Clean-param:_lang Clean-param:action Clean-param:adobeSearchType Clean-param:bannerLinkUrl Clean-param:bannerOmniture Clean-param:blogger Clean-param:bvdata Clean-param:bvroute Clean-param:bvstate Clean-param:c Clean-param:categoryId Clean-param:categoryName Clean-param:componentID Clean-param:countryCode Clean-param:countryName Clean-param:csB2bFlag Clean-param:customerModelCode Clean-param:disableGlobalInfoCollect Clean-param:downTime Clean-param:ds_rl Clean-param:elqah Clean-param:elqaid Clean-param:elqTrackId Clean-param:emptyModel Clean-param:etext Clean-param:feed Clean-param:fromApp Clean-param:gbraid Clean-param:gclsrc Clean-param:gtm_debug Clean-param:hashTag Clean-param:headerTitle Clean-param:hl Clean-param:initialTab Clean-param:keyword Clean-param:lgcom_source Clean-param:limit Clean-param:linkId Clean-param:m Clean-param:menu Clean-param:modelCode Clean-param:modelName Clean-param:modelNum Clean-param:modelNumber Clean-param:msclkid Clean-param:name Clean-param:nm Clean-param:nofire Clean-param:nx Clean-param:obsBuynowFlag Clean-param:obsOnly Clean-param:oldTerm Clean-param:omniture Clean-param:ort Clean-param:p Clean-param:q Clean-param:quickMenuSuperCategoryId Clean-param:r Clean-param:realSearch Clean-param:rearr-factors Clean-param:receiptNumber Clean-param:redirectCheckUrl Clean-param:reSearch Clean-param:returnFlag Clean-param:rev Clean-param:s Clean-param:salesModel Clean-param:search Clean-param:searchChck Clean-param:searchResultFlag Clean-param:selectItem Clean-param:sid Clean-param:siteType Clean-param:sort Clean-param:source Clean-param:spm Clean-param:subcategory Clean-param:subCateId Clean-param:submit Clean-param:superCategoryId Clean-param:superCateId Clean-param:surveyPageChangeYn Clean-param:target Clean-param:tpclid Clean-param:twclid Clean-param:type Clean-param:u Clean-param:url Clean-param:viewSalesCode Clean-param:visit_id Clean-param:wbraid Clean-param:yandex_ad_client_id # Sitemap files Sitemap: https://www.lg.com/sitemap.xml Host: www.lg.com User-agent: Mediapartners-Google Allow:/*.jsp?cmpid User-agent: AdsBot-Google Allow: /*.jsp?cmpid User-agent: AdsBot-Google-Mobile Allow: /*.jsp?cmpid
robots.txt versions
| From | Through | State | Version | |
|---|---|---|---|---|
| 2026-10-26 | 2026-10-28 | ok 200 | 46035ac91b46 |
Timeline of changes
| Date | What changed |
|---|---|
| No changes since the first observation (2026-10-26). | |
Reconstructed history (Common Crawl)
From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.
| Crawl | Host | State | GPTBot | OAI-SearchBot | ChatGPT-User | ClaudeBot | Claude-SearchBot | Claude-User | PerplexityBot |
|---|---|---|---|---|---|---|---|---|---|
| 2026-09-21 CC-MAIN-2026-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2025-09-15 CC-MAIN-2025-38 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2024-09-16 CC-MAIN-2024-38 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2023-10-02 CC-MAIN-2023-40 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2022-10-03 CC-MAIN-2022-40 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2021-09-27 CC-MAIN-2021-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2020-09-28 CC-MAIN-2020-40 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2019-09-23 CC-MAIN-2019-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-09-24 CC-MAIN-2018-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-04-23 CC-MAIN-2018-17 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-01-29 CC-MAIN-2018-05 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-10-23 CC-MAIN-2017-43 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-09-25 CC-MAIN-2017-39 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-07-24 CC-MAIN-2017-30 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-04-24 CC-MAIN-2017-17 | lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-01-23 CC-MAIN-2017-04 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-10-03 CC-MAIN-2016-40 | www.lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-09-05 CC-MAIN-2016-36 | lg.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
Evidence
Every daily observation of lg.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).