meta.com: what it tells AI crawlers

On 27 October 2026, meta.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://www.meta.com/robots.txt
unchanged since 2026-10-26 · sha256 5be2af84fc90337b… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 47074e6e5ac27c41…

User-agent: *
Allow: /*help/support/$
Allow: /*account/$
Allow: /*order/find/$
Allow: /*return/find/$
Disallow: *?_ga=
Disallow: /*help_app/*
Disallow: /*help/support/*
Disallow: /*account/*
Disallow: /order/*
Disallow: /*help/search/*
Disallow: /*rma/*
Disallow: /return/*
Disallow: /meta-employee-store/
Disallow: /bv/upload/*
Disallow: *utm_source
Disallow: *utm_content
Disallow: *utm_campaign
Disallow: *utm_offering
Disallow: *utm_product
Disallow: *utm_medium
Disallow: *?fb_comment_id=
Disallow: *?id=
Disallow: *?cursor=
Disallow: *?ref=
Disallow: *?intern_source
Disallow: *?intern_content
Disallow: */?uid

Sitemap: https://www.meta.com/sitemap.xml
Sitemap: https://www.meta.com/help/sitemap.xml
Sitemap: https://www.meta.com/blog/quest/sitemap.xml
Sitemap: https://www.meta.com/experiences/sitemap.xml

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of meta.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/meta.com