BotView research archive
Crawler diagnostics for 30 websites: archived scan results
This table preserves the results of an earlier BotView scan of 30 SaaS, developer, publishing and commerce websites. The mean site score was 60/100. The scores describe that scanner’s observations at the time; they do not measure how often ChatGPT or Google cited these sites.
Originally published February 26, 2026. Explanation revised September 6, 2026; sites have not been rescanned for this revision.
How the original scan worked
The original run sampled homepages and selected pricing, blog, feature or documentation pages. The scanner compared initial HTML with content rendered in Chromium and recorded diagnostics such as failed resources and rendering gaps. Requests came from the scanner’s network, using browser and crawler user agents.
The table contains site-level scores and the recorded average JavaScript dependency percentage. That percentage describes content added during rendering under the earlier analyzer. It is not the percentage of a site missing from a search index.
This was a selected sample of familiar websites, not a random sample of the web. A low score could reflect the response to our scanner, the pages selected or a rendering failure. It cannot establish the cause of a site’s search performance. Linked reports may no longer be available if their sharing permissions have changed.
The recorded results
| Site | Category | Score / 100 | JS dependency | Source |
|---|---|---|---|---|
| Hugging Face | AI | 87 | 2% | Report |
| Framer | Website Builder | 82 | 0% | Report |
| Moz | SEO Tools | 81 | 10% | Report |
| Zendesk | SaaS | 78 | 7% | Report |
| Shopify | E-commerce | 78 | 9% | Report |
| Notion | Productivity | 76 | 7% | Report |
| Semrush | SEO Tools | 74 | 22% | Report |
| Replicate | AI | 74 | 2% | Report |
| Stripe | Developer | 70 | 0% | Report |
| Vercel | Developer | 67 | 0% | Report |
| BigCommerce | E-commerce | 65 | 9% | Report |
| WordPress.com | Content | 65 | 0% | Report |
| Ghost | Content | 64 | 0% | Report |
| Medium | Content | 63 | 1% | Report |
| Anthropic | AI | 63 | 2% | Report |
| Webflow | Website Builder | 60 | 3% | Report |
| Zapier | Productivity | 59 | 15% | Report |
| Intercom | SaaS | 57 | 18% | Report |
| Twilio | Developer | 57 | 0% | Report |
| Wix | Website Builder | 53 | 2% | Report |
| Squarespace | Website Builder | 52 | 12% | Report |
| Substack | Content | 52 | 40% | Report |
| Supabase | Developer | 50 | 43% | Report |
| OpenAI | AI | 50 | 9% | Report |
| HubSpot | SaaS | 49 | 0% | Report |
| Mailchimp | SaaS | 48 | 0% | Report |
| Monday.com | SaaS | 47 | 7% | Report |
| Calendly | Productivity | 35 | 34% | Report |
| Ahrefs | SEO Tools | 30 | 61% | Report |
| Canva | Productivity | 20 | 20% | Report |
What to check on your own site
Compare an important page’s initial HTML with its rendered content. If key text is added only by JavaScript, inspect whether it renders successfully. If a scanner receives a challenge, check CDN logs and verify crawler identity before changing access controls.
Keep training and search policies separate. Blocking a training crawler is not evidence that a site has opted out of search. A diagnostic score is also not a count of citations or referred visitors.
For a small example you can reproduce yourself, see our JavaScript rendering experiment. It includes both page fixtures, browser screenshots and the measurement script.