forbes.com: what it tells AI crawlers

On 27 October 2026, forbes.com's robots.txt fully blocked GPTBot, ClaudeBot, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://www.forbes.com/robots.txt
unchanged since 2026-10-26 · sha256 29e98ab89e88236d… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 865a08dc70d7475f…

User-agent: *
Allow: /ajax/follow_info*
Allow: /ajax/getUniregUserData*

Disallow: /following/
Disallow: /search/
Disallow: /follow/
Disallow: /ajax/
Disallow: /author/following/
Disallow: /blog/following/
Disallow: /json/
Disallow: /header/channels/
Disallow: /find-more/
Disallow: /preview/
Disallow: *wp-admin*
Disallow: /typeahead/
Disallow: /pepe/
Disallow: /media-manager/
Disallow: /coupons/visit/*
Disallow: /test/
Disallow: /advisor/*zx1fa*

User-agent: googlebot-news
Disallow: /sites/adobe/
Disallow: /sites/arizonacommerce/
Disallow: /sites/bmoharrisbank/
Disallow: /sites/businessaviation/
Disallow: /sites/capitalonespark/
Disallow: /sites/cdw/
Disallow: /sites/centurylink/
Disallow: /sites/dell/
Disallow: /sites/emc/
Disallow: /sites/ey/
Disallow: /sites/fedex/
Disallow: /sites/fidelity/
Disallow: /sites/gyro/
Disallow: /sites/hsbc/
Disallow: /sites/houlihanlawrence/
Disallow: /sites/hiscox/
Disallow: /sites/ibm/
Disallow: /sites/infosys/
Disallow: /sites/mdanderson/
Disallow: /sites/medidata/
Disallow: /sites/microsoft/
Disallow: /sites/microsoftdynamics/
Disallow: /sites/netapp/
Disallow: /sites/northwesternmutual/
Disallow: /sites/oppenheimerfunds/
Disallow: /sites/oracle/
Disallow: /sites/pace/
Disallow: /sites/raymondjames/
Disallow: /sites/sage/
Disallow: /sites/samsung/
Disallow: /sites/sap/
Disallow: /sites/sungardas/
Disallow: /sites/taboola/
Disallow: /sites/ups/
Disallow: /sites/delta/
Disallow: /sites/airbus/
Disallow: /sites/symantec/
Disallow: /sites/paypal/
Disallow: /sites/olddominion/
Disallow: /sites/salesforce/
Disallow: /sites/gapinternational/
Disallow: /sites/sprintbusiness/
Disallow: /sites/transamerica/
Disallow: /sites/constantcontact/
Disallow: /sites/robertocoin/
Disallow: /sites/canon/
Disallow: /sites/samsungbusiness/
Disallow: /sites/att/
Disallow: /sites/lasvegas/
Disallow: /sites/iowa/
Disallow: /sites/teradata/
Disallow: /sites/thehartford/
Disallow: /sites/statoil/
Disallow: /sites/cit/
Disallow: /sites/deloitte/
Disallow: /sites/huawei/
Disallow: /sites/relateiq/
Disallow: /sites/cftc/
Disallow: /sites/celebritycruises/
Disallow: /sites/universityofcalifornia/
Disallow: /sites/coxbusiness/
Disallow: /sites/pega/
Disallow: /sites/athenahealth/ 
Disallow: /sites/etrade/
Disallow: /sites/comcastbusiness/
Disallow: /sites/optum/
Disallow: /sites/seagate/
Disallow: /sites/rbcwealthmanagement/
Disallow: /sites/avigilon/
Disallow: /sites/citi/
Disallow: /sites/castlight/
Disallow: /sites/hyatt/
Disallow: /sites/johnhancock/
Disallow: /sites/koreanair/
Disallow: /sites/merrilllynch/
Disallow: /sites/ptc/
Disallow: /sites/toyota/
Disallow: /sites/unify/
Disallow: /sites/xerox/
Disallow: /sites/united/
Disallow: /sites/zenithwatches/
Disallow: /sites/jpmorganchase/
Disallow: /sites/ramcommercial/
Disallow: /sites/puremichigan/
Disallow: /sites/tmfgroup/
Disallow: /sites/usaa/
Disallow: /sites/emerson/
Disallow: /sites/aberdeen/
Disallow: /sites/kellogg/
Disallow: /sites/zurich/
Disallow: /sites/freeenterprise/
Disallow: /sites/allclearid/
Disallow: /sites/accenture/
Disallow: /sites/mercedesbenz/
Disallow: /sites/nice/
Disallow: /sites/riverbed/
Disallow: /sites/pimco/
Disallow: /sites/netjets/
Disallow: /sites/vmware/
Disallow: /sites/visa/
Disallow: /sites/statefarm/
Disallow: /sites/philips/
Disallow: /sites/adp/
Disallow: /sites/zenithwatches/
Disallow: /sites/citrix/ 
Disallow: /sites/gwcic/
Disallow: /sites/cargill/
Disallow: /sites/bankofsingapore/
Disallow: /sites/dbsbank/
Disallow: /sites/gradsoflife/
Disallow: /sites/internationaliotcouncil/
Disallow: /pega/customers-for-life
Disallow: /celebritycruises/guide-to-the-globe
Disallow: /sites/disneyinstitute/
Disallow: /sites/concertocloud/
Disallow: /sites/capitalonefutureedge/
Disallow: /sites/mahindracomviva/
Disallow: /sites/workday/
Disallow: /sites/colorado/
Disallow: /sites/charlesschwab/
Disallow: /sites/chicagobooth/
Disallow: /sites/morganstanley/
Disallow: /sites/opentext/
Disallow: /sites/icims/
Disallow: /sites/thriventmutualfunds/
Disallow: /sites/cbre/
Disallow: /sites/universityofphoenix/#57934e047260
Disallow: /sites/ppg/
Disallow: /sites/americanfunds/
Disallow: /sites/lilly/
Disallow: /sites/norton/
Disallow: /sites/qualityassurance/
Disallow: /sites/qlik/
Disallow: /whats-next/planes-trains-and-analytics
Disallow: /top-50-wealth-managers
Disallow: /toyota/fueled-by-skepticism
Disallow: /toyota/coming-soon
Disallow: /dell/cloud-future-of-medicine
Disallow: /cit/renewable-energy
Disallow: /retail
Disallow: /mobile/cities-transformed 
Disallow: /cit/small-business-mergers-2 
Disallow: /cit/small-business-mergers 
Disallow: /sites/kpmg/
Disallow: /dell/cities-transformed
Disallow: /sites/neustar
Disallow: /sites/ubs
Disallow: /sites/northerntrust
Disallow: /sites/infiniti 
Disallow: /sites/honeywell
Disallow: /sites/charlesschwabadvisor 
Disallow: /sites/frisco 
Disallow: /sites/colehaan 
Disallow: /sites/gapincthiswayahead   
Disallow: /sites/discoverpersonalloans   
Disallow: /sites/delltechnologies
Disallow: /colehaan/daring-to-step-forward
Disallow: /johnhancock/
Disallow: /johnhancock/in-a-lifetime-healthcare
Disallow: /johnhancock/in-a-lifetime-work
Disallow: /oppenheimerfunds/the-upside
Disallow: /sites/hpeenterprisenxt 
Disallow: /sites/rackspace 
Disallow: /sites/themacallan 
Disallow: /themacallan/worlds-together
Disallow: /sites/inteliotpartnership
Disallow: /oppenheimerfunds/the-upside
Disallow: /partners/
Disallow: /johnhancock/in-a-lifetime-sustainability
Disallow: /sites/domperignon
Disallow: /sites/shinola
Disallow: /sites/hpinc
Disallow: /sites/hp
Disallow: /sites/alfresco
Disallow: /cbre/built-for-performance
Disallow: /sites/mmedric
Disallow: /sites/indiana
Disallow: /sites/siemenshealthineers
Disallow: /sites/rackspace
Disallow: /rackspace/elevating-performance
Disallow: /sites/suntrust 
Disallow: /sites/unicefusa
Disallow: /sites/mitsubishiheavyindustries
Disallow: /best-countries-for-business/list
Disallow: /sites/paloaltonetworks
Disallow: /sites/f5
Disallow: /sites/samsungtv 
Disallow: /sites/qa
Disallow: /sites/paycom
Disallow: /sites/testbrandvoice
Disallow: /business/blahblah
Disallow: /sites/spdr
Disallow: /sites/lincoln
Disallow: /sites/abb
Disallow: /sites/eycybersecurity
Disallow: /sites/civicnation
Disallow: /sites/bcg 
Disallow: /sites/vanguard 
Disallow: /sites/akamai
Disallow: /sites/kyocera
Disallow: /sites/socialimpact 
Disallow: /sites/emirates 
Disallow: /sites/hpincemea 
Disallow: /sites/ibmperformancecomputing 
Disallow: /sites/pfizer   
Disallow: /sites/ibmpowersystems 
Disallow: /sites/vanguardcharitable 
Disallow: /sites/suntrustprivatewealth 
Disallow: /sites/capitalone
Disallow: /sites/juniper
Disallow: /sites/wellsfargo 
Disallow: /sites/bnymellon 
Disallow: /sites/braintree 
Disallow: /sites/bnymellonwealthmanagement
Disallow: /sites/epson 
Disallow: /sites/thomsonreuters 
Disallow: /sites/paysafe 
Disallow: /sites/josabank 
Disallow: /sites/hennessyparadisimperial
Disallow: /sites/cartier
Disallow: /sites/anabluewing
Disallow: /sites/eidoo 
Disallow: /sites/cmstesting 
Disallow: /sites/adobepartnership
Disallow: /sites/cornelljohnson
Disallow: /sites/quantcast
Disallow: /sites/japan
Disallow: /sites/impactpartners
Disallow: /sites/nvidia
Disallow: /sites/personalcapital
Disallow: /sites/microsoft365
Disallow: /sites/providencedigitalinnovation
Disallow: /sites/whittiertrust
Disallow: /sites/kinetica
Disallow: /sites/paulevans 
Disallow: /sites/ritz-carltonleadershipcenter 
Disallow: /sites/icoengine
Disallow: /sites/tesaro 
Disallow: /sites/theglenlivet 
Disallow: /feature/sf-test 
Disallow: /sites/courtyardmarriott
Disallow: /sites/nuveen
Disallow: /sites/intelai
Disallow: /sites/secret
Disallow: /sites/hec 
Disallow: /retail/old 
Disallow: /sites/alliantgroup 
Disallow: /sites/intuitivesurgical 
Disallow: /sites/sunovion
Disallow: /sites/ofx 
Disallow: /sites/firstrepublicbank 
Disallow: /sites/1850byfolgers 
Disallow: /sites/audi 
Disallow: /sites/nuveencef 
Disallow: /sites/1850brandcoffee  
Disallow: /sites/cisco  
Disallow: /sites/blackstarcapitalltd 
Disallow: /sites/usbankaltitudereserve
Disallow: /sites/dassaultsystemes
Disallow: /sites/urogen
Disallow: /sites/smartsheet
Disallow: /sites/iwc
Disallow: /sites/datingcom
Disallow: /sites/intuitquickbooks
Disallow: /sites/oticon
Disallow: /sites/servicenow
Disallow: /sites/deloitteriskandfinancialadvisory
Disallow: /sites/dbsprivatebank
Disallow: /sites/americanexpress
Disallow: /sites/tanotofoundation
Disallow: /sites/citizensaccess
Disallow: /sites/salesforceessentials
Disallow: /sites/drexeluniversity 
Disallow: /sites/microsoftinnovation
Disallow: /sites/mypos
Disallow: /sites/harvardpilgrimhealthcare
Disallow: /sites/appdynamics
Disallow: /sites/ministryoftourismindia
Disallow: /sites/orlando
Disallow: /sites/extrahop
Disallow: /sites/ascm
Disallow: /sites/crowe
Disallow: /sites/peerfit
Disallow: /sites/theleukemialymphomasociety 
Disallow: /sites/hewlettpackardenterprise
Disallow: /sites/montblanc
Disallow: /sites/meero
Disallow: /sites/braze
Disallow: /sites/publicissapient
Disallow: /sites/flexport

User-agent: Amazonbot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: cohere-ai
Disallow: /

User-agent: Diffbot
Disallow: /

User-agent: FacebookBot
Disallow: /

User-agent: FriendlyCrawler
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: ImagesiftBot
Disallow: /

User-agent: meta-externalagent
User-agent: Meta-ExternalAgent
Disallow: /

User-Agent: OAI-SearchBot 
Disallow: / 

User-agent: omgili
Disallow: /

User-agent: omgilibot
Disallow: /

User-Agent: PerplexityBot
Disallow: /

Sitemap: https://www.forbes.com/news_sitemap.xml
Sitemap: https://www.forbes.com/video_sitemap.xml
Sitemap: https://www.forbes.com/companies_sitemap.xml
Sitemap: https://www.forbes.com/health/sitemap.xml
Sitemap: https://www.forbes.com/connect/events-sitemap.xml

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of forbes.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/forbes.com