jetbrains.com: what it tells AI crawlers

On 28 October 2026, jetbrains.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-29 03:59 UTC (daily run of 2026-10-28). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://www.jetbrains.com/robots.txt
unchanged since 2026-10-26 · sha256 9372e147d5d59b25… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-28

Other signals

robots.txt as recorded sha256 ac4d7f2386dd11e4…

User-Agent: *
disallow: */search*
disallow: */youtrack/help*
disallow: */shop*
disallow: */eshop*
disallow: */estore*
disallow: */form*
disallow: */unitrun*
disallow: */specials/
disallow: */resharper20*
disallow: */resharper/documentation/help20/
disallow: */resharper/webhelp45/
disallow: */resharper/webhelp/help41/
disallow: */resharper/webhelp50/
disallow: */resharper/webhelp60/
disallow: */resharper/webhelp70/
disallow: */resharper/webhelp80/
disallow: */help/resharper/5.0/
disallow: */help/resharper/6.0/
disallow: */help/resharper/7.0/
disallow: */help/resharper/8.0/
disallow: */help/resharper/9.2/
disallow: */resharper/webhelp/
disallow: */dotcover/webhelp27/
disallow: */help/dotcover/2.7/
disallow: */help/dotcover/3.0/
disallow: */help/dotcover/3.2/
disallow: */dotmemory/webhelp35/
disallow: */help/dotmemory/3.5/
disallow: */help/dotmemory/4.1/
disallow: */help/dotmemory/4.4/
disallow: */decompiler/webhelp/
disallow: */decompiler/webhelp10/
disallow: */help/decompiler/1.0/
disallow: */help/decompiler/1.3/
disallow: */help/decompiler/1.5/
disallow: */profiler/webhelp55/
disallow: */help/profiler/5.5/
disallow: */help/profiler/6.0/
disallow: */help/profiler/6.2/
disallow: */upsource/help/1.0/
disallow: */help/upsource/1.0/
disallow: */idea/webhelp10.5/
disallow: */idea/webhelp11/
disallow: */idea/webhelp13/
disallow: */help/idea/13.0/
disallow: */help/idea/14.0/
disallow: */download/download_thanks.jsp
disallow: */clion/help1.0
disallow: */help/clion/1.0
disallow: */help/clion/1.1
disallow: */help/clion/1.5
disallow: */help/hub/1.0
disallow: */download-thanks
sitemap: https://www.jetbrains.com/sitemap_index.xml
sitemap: https://www.jetbrains.com/sitemap_support_index.xml

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of jetbrains.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-28: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973640)).

API: https://infrarabbit.com/crawlers/api/v1/policy/jetbrains.com