netangels.ru: what it tells AI crawlers

On 27 October 2026, netangels.ru's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://www.netangels.ru/robots.txt
unchanged since 2026-10-27 · sha256 d96259f1abb73ab6… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 d96259f1abb73ab6…

User-agent: *
Disallow: */register/
Disallow: */order/
Disallow: */auth/
Disallow: /out/
Disallow: /static/
Disallow: /support/hosting-old/
Disallow: *?what=*
Disallow: */?p_ref=*
Disallow: *?q=*
Disallow: *?search=*
Disallow: *?modal=*
Disallow: *?__proto__*
Disallow: *?constructor*
Disallow: *?rest_route=*
Disallow: *?_action=*
Disallow: *?email=*
Disallow: *?ignorethis=*
Disallow: *?KEY1*
Disallow: *?K1*
Disallow: *?%D0%B5%D1%83=*
Disallow: *?_id=*
Disallow: *?bx_sender_conversion_id=*
Disallow: *?2VSb5wRQEs3*
Disallow: *?2VSb5wUNDRj*
Disallow: *?2VSb5wdG97o*
Disallow: *?yprqee=*
Allow: /cloud/?utm_
Allow: /cloud/?gclid
Allow: /cloud/?yclid
Allow: /cloud/?from
Allow: /cloud/?openstat
Disallow: /cloud/?*

User-agent: Yandex
Disallow: */register/
Disallow: */order/
Disallow: */auth/
Disallow: /out/
Disallow: /static/
Disallow: /support/hosting-old/
Clean-param: utm_referer&utm_ya_campaign&utm_candidate&yabizcmpgn&ybaip&yadiscount&yadclid&yadordid&utm_source&utm_medium&utm_campaign&utm_content&utm_term&utm_caqrryew&openstat&from&gclid&yclid /
Clean-param: keywords&scid&_escaped_fragment_&_task&_err&external-test-ids&term&product&_currency&_lang&fromApp&p&city&lr&p_ref&pref&Enpass6AutoFill&_token&etext&_mbox&adv&section_id&amp&yandex-source&modal&ctime /
Clean-param: constructor&_rq_delayed&clickX&pcode-active-testids&clickY&click-time&pcodever&height&rendered-direct-assets&banner-&test-tags&pcode-test-ids-from-count&adsdk-bundle-version&constructor-rendered-assets&order-banners-options&spm&NH4G&r&banner-sizes&pipage&frm&send&format-type&actual-format&dummy&ref&wbraid&media-test-tag&src_pof&bbb-&target&disableGlobalInfoCollect&erid&source&gtm_debug&ignorethistest-tag&bbb-room&gtm_latency&pifm_chp&locale&_x_tr_hist&_source /
Clean-param: __proto__&prototype&q&rest_route&_action&_id&email&ignorethis&yprqee&KEY1&K1&_ref&_ym_debug&search&what&%D0%B5%D1%83&bx_sender_conversion_id&2VSb5wRQEs3&2VSb5wUNDRj&2VSb5wdG97o /

Sitemap: https://www.netangels.ru/sitemap.xml

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of netangels.ru is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/netangels.ru