cisco.com: what it tells AI crawlers

On 27 October 2026, cisco.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://www.cisco.com/robots.txt
unchanged since 2026-10-26 · sha256 898698b0f990c8e1… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 c1bf180c609d78a8…

#--------------------------------
User-agent: cisco-googlebot-enterprise
Disallow: /bug-navigator # Bug Data
# Disallow: /cgi-bin # allow test crawls for TAC support content
Disallow: /pcgi-bin # no programs
Disallow: /univ-src/ccden # will get content through /univercd
Disallow: /cpropub/univercd # obsolete
Disallow: /jobs     # temporary entry per performance team
#--------------------------------

#--------------------------------
User-agent: *
Disallow: /bug-navigator # Bug Data
Disallow: /web/telepresence/pro/
Disallow: /cgi-bin # no programs
Disallow: /pcgi-bin # no programs
Disallow: /univ-src/ccden # will get content through /univercd
Disallow: /cpropub/univercd # obsolete
Disallow: /jobs     # temporary entry per performance team
Disallow: /go/
Disallow: /en/US/swassets/
Disallow: /univercd/cc/td/doc
Disallow: /univercd/
Disallow: /tacpage
Disallow: /survey/
Disallow: /web/login
Disallow: /ciscodotcom/
Disallow: /web/help/index.html
Disallow: /en/US/swassets/sw274/sitewide_country_language_selector.html
Disallow: /public/library/iosplanner/reldesignation.html
Disallow: /web/DE/telepresence/pro/
Disallow: /web/ES/telepresence/pro/
Disallow: /web/IT/telepresence/pro/
Disallow: /web/PO/telepresence/pro/
Disallow: /web/RU/telepresence/pro/
Disallow: /JP/support/public/mt/tac/100/1002347/techtip_conventions.shtml
Disallow: /web/AT/telepresence/pro/
Disallow: /web/BE/telepresence/pro/
Disallow: /web/BE/fr/telepresence/pro/
Disallow: /web/BE/nl/telepresence/pro/
Disallow: /web/CH/de/telepresence/pro/
Disallow: /web/CH/fr/telepresence/pro/
Disallow: /web/CZ/telepresence/pro/
Disallow: /web/DK/telepresence/pro/
Disallow: /web/FI/telepresence/pro/
Disallow: /web/FR/telepresence/pro/
Disallow: /web/HU/telepresence/pro/
Disallow: /web/ME/ar/telepresence/pro/
Disallow: /web/ME/telepresence/pro/
Disallow: /web/NL/telepresence/pro/
Disallow: /web/NO/telepresence/pro/
Disallow: /web/PL/telepresence/pro/
Disallow: /web/PT/telepresence/pro/
Disallow: /web/RO/telepresence/pro/
Disallow: /web/SE/telepresence/pro/
Disallow: /web/TR/telepresence/pro/
Disallow: /cisco/web/UK/telepresence/pro/
Disallow: /web/ZA/telepresence/pro/
Disallow: /web/go/
Disallow: /web/offers/pdfs/
Disallow: /c/en/us/applicat/camp/
Disallow: /c/en/us/applicat/non-camp/
Disallow: /en/us/products/products-sanity-test/ 	
Disallow: /en/us/solutions/solutions-sanity-test/
Disallow: /content/en/us/td/docs/Sanity

#--------------------------------

#--------------------------------
User-agent: Yandex
Disallow: /global/RU/
#--------------------------------
# All changes to robots.txt need to be approved by Jennifer Barnhill(jbarnhil) 

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of cisco.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/cisco.com