ibm.com: what it tells AI crawlers

On 27 October 2026, ibm.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://www.ibm.com/robots.txt
unchanged since 2026-10-26 · sha256 8e29d2cf6f21b8d8… · view as recorded

The * group

partial applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 8e29d2cf6f21b8d8…

User-agent: *
Disallow: /account/registration
Disallow: /account/mypro
Disallow: /account/myint
Disallow: /Admin
Disallow: /cgi-
Disallow: /contact/employees/servlets
Disallow: /data/
Disallow: /db2s
Disallow: /developerworks/*-pdf.pdf$
Disallow: /developerworks/forums/servlet
Disallow: /developerworks/forums/abuse
Disallow: /developerworks/forums/post
Disallow: /docs/api
Disallow: /fcgi-
Disallow: /fscripts
# Disallow: /homepage
Disallow: /image
Disallow: /mashupmaker
Disallow: /mysupport/s/forumshome
Disallow: /mysupport/s/forumsproduct
Disallow: /mysupport/s/forumsquestion
Disallow: /partnerworld/portal
Disallow: /portal/home
Disallow: /products/finder
Disallow: /profiles
Disallow: /scripts
Disallow: /Scripts
Disallow: /search/?
Disallow: /Search
Disallow: /smartmarket/*/productListInitial.action?
Disallow: /standards
Disallow: /support/knowledgecenter/api
Disallow: /support/knowledgecenter/v1
Disallow: /survey
Disallow: /testing
Disallow: /tmp
Disallow: /vrm/bin/sendThisPage
Disallow: /webmaster
Disallow: /zx
Disallow: /zz
Disallow: /*.ssi$
Disallow: /account/myibm/InterestsEdit.do
Disallow: /wcs
Disallow: /wcsstore
Disallow: /webapp
Disallow: /web/portal/software/websphere
Disallow: /common/austin-summit
Disallow: /link
Disallow: /links
Disallow: /web/portal/commerce 
Disallow: /industries/clients
Disallow: /standards
# Disallow: /internal
Disallow: /test
Disallow: /test-kick-tires
Disallow: /test-v1-page-template
Disallow: /product-detail
Disallow: /watson/assets/pdfs/reg
Disallow: /zprod-ease
Disallow: /zProd-ease
Disallow: /zshowcase
Disallow: /widgets-admin
Disallow: /w3
Disallow: /mintlify-assets
Disallow: /industries/federal/fedcat/ibm-automation-services-for-government
# Disallow: /multimedia

sitemap: https://www.ibm.com/content/sitemap/href-sitemap-index.xml
sitemap: https://www.ibm.com/content/dam/adobe-cms/IBM_Adobe_Sitemap.xml
sitemap: https://www.ibm.com/mysupport/s/sitemap.xml
sitemap: https://www.ibm.com/us-en.sitemap.news-sitemap.xml
sitemap: https://www.ibm.com/docs/en/sitemap.xml

User-agent: Fast corporate crawler
Disallow: //
Disallow: /account/registration
Disallow: /account/mypro
Disallow: /account/myint
Disallow: /Admin
Disallow: /cgi-
Disallow: /common
Disallow: /contact/employees/servlets
Disallow: /data/
Disallow: /db2s
Disallow: /developerworks
Disallow: /fcgi-
Disallow: /fscripts
Disallow: /homepage
Disallow: /i/
Disallow: /image
Disallow: /investor
Disallow: /link
Disallow: /mashupmaker
Disallow: /portal/home
Disallow: /products/finder
Disallow: /profiles
Disallow: /SametimeWebApp
Disallow: /scripts
Disallow: /Scripts
Disallow: /search/?
Disallow: /Search
Disallow: /survey
Disallow: /tmp
Disallow: /vrm/bin/sendThisPage
Disallow: /webmaster
Disallow: /zx
Disallow: /zz

User-agent: 008
Disallow: /smartmarket

# Added for IN4173782 on 7Aug2013
User-agent: magpie-crawler
Disallow: /connections/

# Added for IN4177562 on 8Aug2013
User-agent: magpie-crawler/1.1
Disallow: /connections/

# Added for IN4177562 on 8Aug2013
User-agent: MagpieRSS
Disallow: /connections/   

User-agent: psnsearch
Disallow: /developerworks/wikis/login.action

# Added to block site mirroring
User-agent: HTTrack
Disallow: /


User-agent: Acrolinx-bot
# Disallow: //
Disallow: /account/registration
Disallow: /account/mypro
Disallow: /account/myint
Disallow: /Admin
Disallow: /cgi-
Disallow: /contact/employees/servlets
Disallow: /data/
Disallow: /db2s
Disallow: /developerworks/*-pdf.pdf$
Disallow: /developerworks/forums/servlet
Disallow: /developerworks/forums/abuse
Disallow: /developerworks/forums/post
Disallow: /docs/api
Disallow: /fcgi-
Disallow: /fscripts
# Disallow: /homepage
Disallow: /image
Disallow: /mashupmaker
Disallow: /mysupport/s/forumshome
Disallow: /mysupport/s/forumsproduct
Disallow: /mysupport/s/forumsquestion
Disallow: /partnerworld/portal
Disallow: /portal/home
Disallow: /products/finder
Disallow: /profiles
Disallow: /scripts
Disallow: /Scripts
Disallow: /search/?
Disallow: /Search
Disallow: /smartmarket/*/productListInitial.action?
Disallow: /standards
Disallow: /support/knowledgecenter/api
Disallow: /support/knowledgecenter/v1
Disallow: /survey
Disallow: /testing
Disallow: /tmp
Disallow: /vrm/bin/sendThisPage
Disallow: /webmaster
Disallow: /zx
Disallow: /zz
Disallow: /*.ssi$
Disallow: /account/myibm/InterestsEdit.do
Disallow: /wcs
Disallow: /wcsstore
Disallow: /webapp
Disallow: /web/portal/software/websphere
Disallow: /common/austin-summit
Disallow: /link
Disallow: /links
Disallow: /web/portal/commerce 
Disallow: /industries/clients
Disallow: /standards

User-agent: amazon-Quick-on-behalf-of
Disallow: /mysupport

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of ibm.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/ibm.com