immobiliare.it: what it tells AI crawlers

On 27 October 2026, immobiliare.it's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 10k (Tranco)

robots.txt

ok (HTTP 200) · https://www.immobiliare.it/robots.txt
unchanged since 2026-10-27 · sha256 214fcadc3da06a20… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 3993c66e55ce6351…

User-Agent: *
Allow: /news/wp-admin/admin-ajax.php
Disallow: *idmacroarea=
Disallow: *slau=
Disallow: /ricerca-mappa/
Disallow: /search-map
Disallow: /search-list
Disallow: /ricerca.php
Disallow: /ricerca_agenzie.php
Disallow: /track-events/
Disallow: /nav-ads/
Disallow: /mutui/calcolatore?prezzo=
Disallow: /foto_stampa_beez.php
Disallow: /user/
Disallow: /utente/
Disallow: /dettaglio.php
Disallow: /dettaglio_progetto.php
Disallow: *stampa=s
Disallow: /news/newsletter/
Disallow: /news/?s=
Disallow: /news/nl-unsubscribe/
Disallow: /news/wp-admin/
Disallow: /news/search/
Disallow: /valutazione-immobiliare/valuta-
Disallow: /vendi-immobile/vendi-
Disallow: /agenzie-immobiliari/*/progedil-
Disallow: /imprese-edili/*/progedil-
Disallow: /mutui/riepilogo
Disallow: /mutui/mailing-list
Disallow: /mutui/opt-out
Disallow: /mutui/prenota
Disallow: /mutui/richiedi/finanziamento
Disallow: /mutui/riepilogo/finanziamento
Disallow: /demo-home/
Disallow: /demo-detail/

Sitemap: https://www.immobiliare.it/sitemap.xml
Sitemap: https://www.immobiliare.it/news/sitemap_index.xml

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of immobiliare.it is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/immobiliare.it