time.com: what it tells AI crawlers

On 28 October 2026, time.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-29 03:59 UTC (daily run of 2026-10-28). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://time.com/robots.txt
unchanged since 2026-10-26 · sha256 9ae14e98633c12d8… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-28

Other signals

robots.txt as recorded sha256 0a19a48b7c60a8da…

Sitemap: https://time.com/news-sitemap.xml

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
# exclude urls of format time.com/page/7456/?search
User-agent: Googlebot
Disallow: /?search*
Disallow: */munich/index_html*
Disallow: /*?__rmid___get___page
Disallow: /*?*__hsfp
Disallow: /*?*__hstc
Disallow: /*?*__rmid
Disallow: /*?*__rmidpage
Disallow: /*?*/*ref
Disallow: /*?*002/*0902
Disallow: /*?*2&hubs_content
Disallow: /*?*ajs_event
Disallow: /*?*app
Disallow: /*?*attachment_id
Disallow: /*?*author
Disallow: /*?*bcpid
Disallow: /*?*bcpidpage
Disallow: /*?*cat
Disallow: /*?*controlsVisibleOnLoad
Disallow: /*?*country
Disallow: /*?*curator
Disallow: /*?*edition
Disallow: /*?*egotecpage
Disallow: /*?*fbclid
Disallow: /*?*hl
Disallow: /*?*hsCtaTracking
Disallow: /*?*hubs_signup-cta
Disallow: /*?*iframe
Disallow: /*?*iid
Disallow: /*?*industry
Disallow: /*?*inline-read-more
Disallow: /*?*kwp_4
Disallow: /*?*library
Disallow: /*?*limit
Disallow: /*?*linkId
Disallow: /*?*luicode
Disallow: /*?*ntv_a
Disallow: /*?*offset
Disallow: /*?*omnil
Disallow: /*?*onwardjourney
Disallow: /*?*origin
Disallow: /*?*page
Disallow: /*?*PageSpeed
Disallow: /*?*plan
Disallow: /*?*poy-poll
Disallow: /*?*prx_t
Disallow: /*?*ref
Disallow: /*?*reviews_page
Disallow: /*?*scrape
Disallow: /*?*scriptPath
Disallow: /*?*source
Disallow: /*?*sr_source
Disallow: /*?*subscription
Disallow: /*?*ua
Disallow: /*?*utm
Disallow: /*?*utm_campaign
Disallow: /*?*utm_medium
Disallow: /*?*utm_source
Disallow: /*?*xid
Disallow: /*?*yyyy
Disallow: /*?bclid=
Disallow: /*?bcpid=
Disallow: /*?bctid=
Disallow: /*?EGOTECpage
Disallow: /*?hc_location
Disallow: /*?pagespeed
Disallow: /*?ref=
Disallow: /*?ref=binfind.com
Disallow: /*?sa=
Disallow: /*?search
Disallow: /feed
Disallow: /nextadvisor/search/*
Disallow: /page/*/?a_aid=
Disallow: /page/*/?egotecpage
Disallow: /page/*/?hc_location=
Disallow: /page/*/?iq_id=
Disallow: /page/*/?pcd=
Disallow: /page/*/?search=
Disallow: /search/?q=*
Disallow: /time_tout/*
Disallow: /wp-admin/*
Disallow: /wp-login/*
Disallow: /uncategorized

# TIME stamped Sitemap
Sitemap: https://time.com/personal-finance/sitemap.xml

Sitemap: https://time.com/shopping/sitemap.xml

# Sitemap archive
Sitemap: https://time.com/sitemap.xml

# Video Sitemap archive
Sitemap: https://time.com/video-sitemap.xml

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of time.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-28: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973640)).

API: https://infrarabbit.com/crawlers/api/v1/policy/time.com