uber.com: what it tells AI crawlers

On 27 October 2026, uber.com's robots.txt allowed all of GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot and CCBot.

Last verified 2026-10-28 03:59 UTC (daily run of 2026-10-27). Proof for that day.

Panel

top 1k (Tranco)

robots.txt

ok (HTTP 200) · https://www.uber.com/robots.txt
unchanged since 2026-10-26 · sha256 40a0335cf0146087… · view as recorded

The * group

allows applies to every agent without its own group

Per AI bot, 2026-10-27

Other signals

robots.txt as recorded sha256 bb1e0addc13147f1…

# robotstxt.org/
User-agent: *
allow: *.js
allow: *.css
disallow: /invite
disallow: /report-issue
disallow: */?_ga
disallow: *?gh_jid
disallow: */?client_id
disallow: */?realestate_id
disallow: */go/
disallow: */cmlntest-access
disallow: */api/
disallow: /app/
disallow: /local/search
disallow: /local/*?*sort
disallow: /local/*/*/*/*/
disallow: /global/es-es/airports/
disallow: /global/fr-ca/airports/
disallow: /in/hi/e/*
disallow: /in/bn/e/*
disallow: /in/kn/e/*
disallow: /in/mr/e/*
disallow: /in/ta/e/*
disallow: /in/te/e/*
disallow: /in/ur/e/*

User-agent: AdsBot-Google
Disallow:

User-agent: Googlebot-Image
Disallow:

sitemap: https://www.uber.com/sitemap.xml

disallow: /ch/en/
disallow: /in/ur/
disallow: /jp/zh/
disallow: /kr/zh/
disallow: /ma/fr/
disallow: /ma/en/

robots.txt versions

Timeline of changes

Reconstructed history (Common Crawl)

From Common Crawl's robots.txt captures: what CCBot was served, one crawl per quarter. A separate series from the daily observations above; a 403 here means Common Crawl was refused, not this ledger.

Evidence

Every daily observation of uber.com is a leaf in that day's Merkle tree. Proof of the robots.txt line for 2026-10-27: audit path (JSON), day root and OpenTimestamps proof (complete (bitcoin block 973493)).

API: https://infrarabbit.com/crawlers/api/v1/policy/uber.com