Applebot-Extended: share of top websites blocking it

On 26 October 2026, 12.1% of the readable top 10k websites fully blocked Applebot-Extended in robots.txt (bounds 9.6% to 29.8% over all 8,220 websites, 1,659 of them unreadable); 3.8% blocked it partly and 10.2% named it.

Last verified 2026-10-26 21:13 UTC (daily run of 2026-10-26). Proof for that day.

Operator
Apple
What it is
Powers features in Siri, Spotlight, Safari, Apple Intelligence, and others.
Description
Apple has a secondary user agent, Applebot-Extended ... [that is] used to train Apple's foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools.
robots.txt token
User-agent: Applebot-Extended
Listed since
2026-10-26 (in this ledger's registry)

Fully blocks

12.1%

791 sites; bounds 9.6% to 29.8%

Partly blocks

3.8%

250 sites

Name it explicitly

10.2%

670 sites; unreadable: 3,439

Over time

Share of panel sites blocking Applebot-Extended4 lines, 2016-09-05 to 2026-10-26; values between 2.52 and 12.06 %; latest 12.06 %.1.764.527.2910.0512.82Sep 2016Mar 2019Sep 2021Apr 2024Oct 2026blocks, reconstructed (Common Crawl): 2016-09-05 UTC: 3.14 %blocks, reconstructed (Common Crawl): 2017-09-25 UTC: 3.51 %blocks, reconstructed (Common Crawl): 2018-09-24 UTC: 3.61 %blocks, reconstructed (Common Crawl): 2019-09-23 UTC: 2.80 %blocks, reconstructed (Common Crawl): 2020-09-28 UTC: 2.52 %blocks, reconstructed (Common Crawl): 2021-09-27 UTC: 2.61 %blocks, reconstructed (Common Crawl): 2022-10-03 UTC: 2.55 %blocks, reconstructed (Common Crawl): 2024-09-16 UTC: 4.63 %blocks, reconstructed (Common Crawl): 2026-09-21 UTC: 9.95 %partly blocks, reconstructed (Common Crawl): 2016-09-05 UTC: 5.04 %partly blocks, reconstructed (Common Crawl): 2017-09-25 UTC: 4.91 %partly blocks, reconstructed (Common Crawl): 2018-09-24 UTC: 4.82 %partly blocks, reconstructed (Common Crawl): 2019-09-23 UTC: 4.76 %partly blocks, reconstructed (Common Crawl): 2020-09-28 UTC: 4.82 %partly blocks, reconstructed (Common Crawl): 2021-09-27 UTC: 4.98 %partly blocks, reconstructed (Common Crawl): 2022-10-03 UTC: 4.91 %partly blocks, reconstructed (Common Crawl): 2024-09-16 UTC: 5.07 %partly blocks, reconstructed (Common Crawl): 2026-09-21 UTC: 5.53 %blocks, daily observation: 2026-10-26 UTC: 12.06 %partly blocks, daily observation: 2026-10-26 UTC: 3.81 %

"Partly blocks" counts every robots.txt whose rules for Applebot-Extended disallow some paths, including generic rules for all crawlers (for example /search); it is not a measure of targeted AI policy. Shares are of the readable websites (W − U: 8,220 websites, 1,659 unreadable); the bounds assume none / all of the unreadable sites block. Solid: daily first-party observations (top 10k of the Tranco list W3ZV9, one fixed cohort). Dashed: reconstructed from Common Crawl (top 10k, sites readable in every crawl), a separate series.

Reconstructed values per crawl
CrawlDateBlocksSites read
CC-MAIN-2016-362016-09-053.1%3,216
CC-MAIN-2017-392017-09-253.5%3,216
CC-MAIN-2018-392018-09-243.6%3,216
CC-MAIN-2019-392019-09-232.8%3,216
CC-MAIN-2020-402020-09-282.5%3,216
CC-MAIN-2021-392021-09-272.6%3,216
CC-MAIN-2022-402022-10-032.5%3,216
CC-MAIN-2024-382024-09-164.6%3,216
CC-MAIN-2026-392026-09-2110.0%3,216

Recent blockers

Sites that started fully blocking Applebot-Extended since 2026-07-28, highest-ranked first.

DateSiteWhat changed
n-tv.denow blocks Applebot-Extended (was allow)
ptt.ccnow blocks Applebot-Extended (was allow)
xnxx.healthnow blocks Applebot-Extended (was allow)
manhwaweb.comnow blocks Applebot-Extended (was allow)
xn--12c1ezaww.comnow blocks Applebot-Extended (was allow)
loteriadehoy.comnow blocks Applebot-Extended (was allow)
elmundo.esnow blocks Applebot-Extended (was allow)
kasserver.comnow blocks Applebot-Extended (was allow)
filmyzilla56.comnow blocks Applebot-Extended (was allow)
filmyzilla62.comnow blocks Applebot-Extended (was allow)
filmyzilla63.comnow blocks Applebot-Extended (was allow)
mastodon.socialnow blocks Applebot-Extended (was allow)
bollyflix.afnow blocks Applebot-Extended (was allow)
vaticannews.vanow blocks Applebot-Extended (was allow)
swwoittlv.ccnow blocks Applebot-Extended (was allow)
rae.esnow blocks Applebot-Extended (was allow)
fragrantica.comnow blocks Applebot-Extended (was allow)

Recent unblockers

DateSiteWhat changed
vnexpress.netnow allows Applebot-Extended (was block)
zimbra.comnow allows Applebot-Extended (was block)
nova-app.eunow allows Applebot-Extended (was block)
resetera.comnow allows Applebot-Extended (was block)
olhardigital.com.brnow partly blocks Applebot-Extended (was block)

Sites blocking Applebot-Extended now

facebook.com · fbcdn.net · instagram.com · twitter.com (*) · googlevideo.com (*) · whatsapp.net · netflix.com (*) · x.com (*) · pinterest.com (*) · tiktok.com · whatsapp.com · baidu.com (*) · msn.com · reddit.com (*) · skybridge.click (*) · registrar-servers.com (*) · t-online.de · snapchat.com · tumblr.com · nytimes.com · flickr.com (*) · ebay.com · theguardian.com · cnn.com · meraki.com (*) · bbc.com · forbes.com · bbc.co.uk · t.co (*) · imdb.com (*) · canva.com · launchpad.net · reuters.com · nature.com · demdex.net (*) · weibo.com (*) · naver.com (*) · trustpilot.com · netflix.net (*) · nflxvideo.net (*) · washingtonpost.com · bloomberg.com · wsj.com (*) · cookiedatabase.org (*) · gcdn.co (*) · pixabay.com · kwai.net (*) · quora.com · statista.com · behance.net · edgecdn.ru (*) · uol.com.br · ft.com · wired.com · tripadvisor.com · threads.com · netcraze.io (*) · usatoday.com · unesco.org · dailymail.com

(*) blocked through the * group, not by name. Full list in current.csv.

Cite this

InfraRabbit AI Crawl Ledger, "Share of top websites blocking Applebot-Extended", https://infrarabbit.com/crawlers/bot/Applebot-Extended, retrieved 2026-10-26. CC BY 4.0.