cisco.com: what it tells AI crawlers
On 27 October 2026, cisco.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.
Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.
- Panel
- top 1k (Tranco)
- robots.txt
- ok (HTTP 200) · https://www.cisco.com/robots.txt
unchanged since 2026-10-26 · sha256898698b0f990c8e1…· view as recorded - The * group
- allows applies to every agent without its own group
Per AI bot, 2026-10-27
| User agent | Verdict | How |
|---|---|---|
| GPTBot | allows | through the * group |
| OAI-SearchBot | allows | through the * group |
| ChatGPT-User | allows | through the * group |
| ClaudeBot | allows | through the * group |
| Claude-SearchBot | allows | through the * group |
| Claude-User | allows | through the * group |
| PerplexityBot | allows | through the * group |
| Perplexity-User | allows | through the * group |
| Googlebot | allows | through the * group |
| Google-Extended | allows | through the * group |
| Applebot-Extended | allows | through the * group |
| CCBot | allows | through the * group |
| Bytespider | allows | through the * group |
| Meta-ExternalAgent | allows | through the * group |
| Amazonbot | allows | through the * group |
Other signals
| Publishes /llms.txt | no |
| Publishes /ai.txt | no |
| Publishes /.well-known/tdmrep.json | no |
| Reserves TDM rights (TDMRep file, header or meta) | no |
| Declares Content Signals (robots.txt or header) | no |
| Says ai-train=no in a Content Signal | no |
| Points to an RSL licence (robots.txt License line or Link header) | no |
| Answers with an x402 payment offer (lower bound: only the homepage is probed) | no |
| Answers HTTP 402 Payment Required (lower bound: only the homepage is probed) | no |
| Sends noai / noimageai directives | no |
| Refused our fetcher (403 / challenge) | no |
robots.txt as recorded sha256 04809e44ebe9ec97…
#-------------------------------- User-agent: cisco-googlebot-enterprise Disallow: /bug-navigator # Bug Data # Disallow: /cgi-bin # allow test crawls for TAC support content Disallow: /pcgi-bin # no programs Disallow: /univ-src/ccden # will get content through /univercd Disallow: /cpropub/univercd # obsolete Disallow: /jobs # temporary entry per performance team #-------------------------------- #-------------------------------- User-agent: * Disallow: /bug-navigator # Bug Data Disallow: /web/telepresence/pro/ Disallow: /cgi-bin # no programs Disallow: /pcgi-bin # no programs Disallow: /univ-src/ccden # will get content through /univercd Disallow: /cpropub/univercd # obsolete Disallow: /jobs # temporary entry per performance team Disallow: /go/ Disallow: /en/US/swassets/ Disallow: /univercd/cc/td/doc Disallow: /univercd/ Disallow: /tacpage Disallow: /survey/ Disallow: /web/login Disallow: /ciscodotcom/ Disallow: /web/help/index.html Disallow: /en/US/swassets/sw274/sitewide_country_language_selector.html Disallow: /public/library/iosplanner/reldesignation.html Disallow: /web/DE/telepresence/pro/ Disallow: /web/ES/telepresence/pro/ Disallow: /web/IT/telepresence/pro/ Disallow: /web/PO/telepresence/pro/ Disallow: /web/RU/telepresence/pro/ Disallow: /JP/support/public/mt/tac/100/1002347/techtip_conventions.shtml Disallow: /web/AT/telepresence/pro/ Disallow: /web/BE/telepresence/pro/ Disallow: /web/BE/fr/telepresence/pro/ Disallow: /web/BE/nl/telepresence/pro/ Disallow: /web/CH/de/telepresence/pro/ Disallow: /web/CH/fr/telepresence/pro/ Disallow: /web/CZ/telepresence/pro/ Disallow: /web/DK/telepresence/pro/ Disallow: /web/FI/telepresence/pro/ Disallow: /web/FR/telepresence/pro/ Disallow: /web/HU/telepresence/pro/ Disallow: /web/ME/ar/telepresence/pro/ Disallow: /web/ME/telepresence/pro/ Disallow: /web/NL/telepresence/pro/ Disallow: /web/NO/telepresence/pro/ Disallow: /web/PL/telepresence/pro/ Disallow: /web/PT/telepresence/pro/ Disallow: /web/RO/telepresence/pro/ Disallow: /web/SE/telepresence/pro/ Disallow: /web/TR/telepresence/pro/ Disallow: /cisco/web/UK/telepresence/pro/ Disallow: /web/ZA/telepresence/pro/ Disallow: /web/go/ Disallow: /web/offers/pdfs/ Disallow: /c/en/us/applicat/camp/ Disallow: /c/en/us/applicat/non-camp/ Disallow: /en/us/products/products-sanity-test/ Disallow: /en/us/solutions/solutions-sanity-test/ Disallow: /content/en/us/td/docs/Sanity Disallow: /c/dam/assets/offers/pdfs/ Disallow: /c/dam/m/da_dk/offers/assets/images/ Disallow: /c/dam/m/da_dk/offers/assets/pdfs/ Disallow: /c/dam/m/de_de/offers/assets/images/ Disallow: /c/dam/m/de_de/offers/assets/pdfs/ Disallow: /c/dam/m/en_au/offers/assets/images/ Disallow: /c/dam/m/en_au/offers/assets/pdfs/ Disallow: /c/dam/m/en_au/offers/images/ Disallow: /c/dam/m/en_au/offers/video/ Disallow: /c/dam/m/en_emear/offers/assets/images/ Disallow: /c/dam/m/en_emear/offers/assets/pdfs/ Disallow: /c/dam/m/en_hk/offers/assets/images/ Disallow: /c/dam/m/en_hk/offers/assets/pdf/ Disallow: /c/dam/m/en_id/offers/assets/images/ Disallow: /c/dam/m/en_id/offers/assets/pdfs/ Disallow: /c/dam/m/en_id/offers/images/ Disallow: /c/dam/m/en_in/offers/assets/images/ Disallow: /c/dam/m/en_in/offers/assets/pdfs/ Disallow: /c/dam/m/en_in/offers/images Disallow: /c/dam/m/en_in/offers/video/ Disallow: /c/dam/m/en_my/offers/assets/images/ Disallow: /c/dam/m/en_my/offers/assets/pdfs/ Disallow: /c/dam/m/en_my/offers/images/ Disallow: /c/dam/m/en_ph/offers/assets/images/ Disallow: /c/dam/m/en_ph/offers/assets/pdfs/ Disallow: /c/dam/m/en_ph/offers/images/ Disallow: /c/dam/m/en_sg/offers/assets/images/ Disallow: /c/dam/m/en_sg/offers/assets/pdfs/ Disallow: /c/dam/m/en_sg/offers/images/ Disallow: /c/dam/m/en_sg/offers/video/ Disallow: /c/dam/m/en_th/offers/assets/images/ Disallow: /c/dam/m/en_th/offers/assets/pdfs/ Disallow: /c/dam/m/en_th/offers/images/ Disallow: /c/dam/m/en_uk/offers/assets/images/ Disallow: /c/dam/m/en_uk/offers/assets/pdfs/ Disallow: /c/dam/m/en_vn/offers/assets/images/ Disallow: /c/dam/m/en_vn/offers/assets/pdfs/ Disallow: /c/dam/m/es_es/offers/assets/images/ Disallow: /c/dam/m/es_es/offers/assets/pdfs/ Disallow: /c/dam/m/es_la/offers/assets/images/ Disallow: /c/dam/m/es_la/offers/assets/pdfs/ Disallow: /c/dam/m/es_la/offers/video/ Disallow: /c/dam/m/es_mx/offers/assets/images/ Disallow: /c/dam/m/es_mx/offers/assets/pdfs/ Disallow: /c/dam/m/fi_fi/offers/assets/images/ Disallow: /c/dam/m/fi_fi/offers/assets/pdfs/ Disallow: /c/dam/m/fr_ca/offers/assets/images/ Disallow: /c/dam/m/fr_ca/offers/assets/pdfs/ Disallow: /c/dam/m/fr_ca/offers/video/ Disallow: /c/dam/m/fr_fr/offers/assets/images Disallow: /c/dam/m/fr_fr/offers/assets/pdfs/ Disallow: /c/dam/m/it_it/offers/assets/images/ Disallow: /c/dam/m/it_it/offers/assets/pdfs/ Disallow: /c/dam/m/ja_jp/offers/161/security/images/ Disallow: /c/dam/m/ja_jp/offers/161/security/pdf/ Disallow: /c/dam/m/ja_jp/offers/162/network/images Disallow: /c/dam/m/ja_jp/offers/162/network/pdf/ Disallow: /c/dam/m/ja_jp/offers/162/sc10/images/ Disallow: /c/dam/m/ja_jp/offers/162/sc10/pdf/ Disallow: /c/dam/m/ja_jp/offers/162/security/pdf/ Disallow: /c/dam/m/ko_kr/offers/assets/images/ Disallow: /c/dam/m/ko_kr/offers/assets/pdfs/ Disallow: /c/dam/m/ko_kr/offers/images/ Disallow: /c/dam/m/nl_be/offers/assets/images/ Disallow: /c/dam/m/nl_be/offers/assets/pdfs/ Disallow: /c/dam/m/nl_nl/offers/assets/images/ Disallow: /c/dam/m/nl_nl/offers/assets/pdfs/ Disallow: /c/dam/m/no_no/offers/assets/images/ Disallow: /c/dam/m/no_no/offers/assets/pdfs/ Disallow: /c/dam/m/pl_pl/offers/assets/images/ Disallow: /c/dam/m/pl_pl/offers/assets/pdfs/ Disallow: /c/dam/m/pt_br/offers/assets/images Disallow: /c/dam/m/pt_br/offers/assets/pdfs Disallow: /c/dam/m/pt_br/offers/img/ Disallow: /c/dam/m/pt_br/offers/pdf/ Disallow: /c/dam/m/pt_br/offers/video/ Disallow: /c/dam/m/ru_ru/offers/assets/images/ Disallow: /c/dam/m/ru_ru/offers/assets/pdfs/ Disallow: /c/dam/m/sv_se/offers/assets/images/ Disallow: /c/dam/m/sv_se/offers/assets/pdfs/ Disallow: /c/dam/m/tr_tr/offers/assets/images/ Disallow: /c/dam/m/tr_tr/offers/assets/pdfs/ Disallow: /c/dam/m/zh_cn/offers/assets/images/ Disallow: /c/dam/m/zh_cn/offers/assets/pdfs/ Disallow: /c/dam/m/zh_tw/offers/assets/images/ Disallow: /c/es_mx/support/ Disallow: /c/es_mx/tech/ Disallow: /c/es_mx/td/ Disallow: /c/ru_ru/support/ Disallow: /c/ru_ru/tech/ Disallow: /c/ru_ru/td/ Disallow: /c/fr_ca/support/ Disallow: /c/fr_ca/tech/ Disallow: /c/fr_ca/td/ Disallow: /c/ja_jp/support/ Disallow: /c/ja_jp/tech/ Disallow: /c/ja_jp/td/ Disallow: /c/dam/m/zh_tw/offers/assets/pdfs/ Disallow: /c/dam/en/us/td/docs/Sanity/datasheet/ Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ar_ae_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ar_eg_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ar_qa_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ar_sa_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_az_az_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_bg_bg_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_bs_ba_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_cs_cz_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_da_dk_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_de_at_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_de_ch_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_de_de_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_el_gr_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_ae_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_au_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_be_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_ca_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_cy_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_dz_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_eg_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_hk_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_id_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_il_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_in_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_mt_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_my_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_ng_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_ph_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_qa_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_sa_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_sg_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_th_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_uk_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_us_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_vn_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_en_za_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_ar_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_bo_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_bz_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_cl_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_co_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_cr_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_ec_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_es_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_gt_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_hn_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_mx_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_ni_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_pa_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_pe_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_pr_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_py_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_sv_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_uy_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_es_ve_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_et_ee_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_fi_fi_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_fr_be_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_fr_ca_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_fr_ch_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_fr_dz_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_fr_fr_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_hr_hr_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_hu_hu_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_it_it_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ja_jp_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ko_kr_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_lt_lt_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_lv_lv_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_mk_mk_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_nl_be_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_nl_nl_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_no_no_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_pl_pl_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_pt_br_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_pt_pt_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ro_ro_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ru_kz_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ru_ru_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_ru_ua_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_sk_sk_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_sl_si_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_sr_rs_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_sv_se_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_th_th_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_tr_tr_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_uk_ua_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_vi_vn_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_zh_cn_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_zh_hk_index.xml Sitemap: http://www.cisco.com/web/sitemap/www_cisco_com_zh_tw_index.xml #-------------------------------- #-------------------------------- User-agent: Yandex Disallow: /global/RU/ #-------------------------------- # All changes to robots.txt need to be approved by Jennifer Barnhill(jbarnhil)
robots.txt versions
| From | Through | State | Version | |
|---|---|---|---|---|
| 2026-10-26 | 2026-10-27 | ok 200 | 898698b0f990 |
Timeline of changes
| Date | What changed |
|---|---|
| No changes since the first observation (2026-10-26). | |
Reconstructed history (Common Crawl)
From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.
| Crawl | Host | State | GPTBot | OAI-SearchBot | ChatGPT-User | ClaudeBot | Claude-SearchBot | Claude-User | PerplexityBot |
|---|---|---|---|---|---|---|---|---|---|
| 2026-09-21 CC-MAIN-2026-39 | www.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2025-09-15 CC-MAIN-2025-38 | www.cisco.com | ok view | blocks | allows | unknown | allows | allows | unknown | allows |
| 2024-09-16 CC-MAIN-2024-38 | www.cisco.com | ok view | blocks | allows | unknown | allows | allows | unknown | allows |
| 2023-10-02 CC-MAIN-2023-40 | www.cisco.com | ok view | blocks | allows | unknown | allows | allows | unknown | allows |
| 2022-10-03 CC-MAIN-2022-40 | www.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2021-09-27 CC-MAIN-2021-39 | www.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2020-09-28 CC-MAIN-2020-40 | www.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2019-09-23 CC-MAIN-2019-39 | www.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-09-24 CC-MAIN-2018-39 | www3.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2018-01-29 CC-MAIN-2018-05 | www3.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-10-23 CC-MAIN-2017-43 | www3.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-09-25 CC-MAIN-2017-39 | www3.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-07-24 CC-MAIN-2017-30 | www3.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2017-04-24 CC-MAIN-2017-17 | www9.cisco.com | soft404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| 2017-01-23 CC-MAIN-2017-04 | cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-10-03 CC-MAIN-2016-40 | www9.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
| 2016-09-05 CC-MAIN-2016-36 | www9.cisco.com | ok view | allows | allows | unknown | allows | allows | unknown | allows |
Evidence
Every daily observation of cisco.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).
API: https://infrarabbit.com/crawlers/api/v1/policy/cisco.com