AI Visibility Benchmark · Methodology
Follow the finding back to its source.
A useful recommendation should survive a simple question: what did you observe, and how did that lead to this advice?
Benchmark record
- Edition
- 2026.06
- Sample
- 50 reviewed service-business websites
- Method
- Automated crawl, then manual semantic review, then scoring
Benchmark corpus
This page explains the June 2026 study and what you can check in its retained records. Our interpretation connects those observations to possible next questions. The recommendation for an individual business still needs its audience, goal and current evidence.
How the sample was selected
The published June protocol required a service business with a service page, a conversion route and indexable public HTML that could be reviewed without administrative access. The final set contains 50 records with distinct site IDs across eleven recorded business categories. All original country fields were coded US; there is no UK sample in this edition.
Consulting, accounting and agency sites account for 35 records. This is a selected public-page sample, not a random sample of a market or a representative US/UK comparison group.
What selection leaves out
The protocol excluded parked, login-only, under-construction, PDF-only, app-only, non-service, duplicate, inaccessible and owner-excluded websites, alongside its listed restricted categories. Manual review could also remove or replace unsuitable, duplicated or unsupported records.
Because accessibility was part of selection, the high recorded access rate cannot establish that access problems are uncommon across service businesses. It describes the retained group.
What the protocol asked reviewers to inspect
| Page role | June protocol |
|---|---|
| Homepage | Required |
| Primary service page | Required |
| Contact or lead route | Required |
| Supporting information and About | Preferred |
| Location page | Optional |
The published row combines site and selected-page fields. It is not an exhaustive inventory of every page or piece of evidence a business owns.
What the collection recorded
The fields cover HTTP responses, robots evaluations, sitemap discovery, canonical and index/snippet directives, headings, visible text, schema, links, tables, lists and download signals. They also record possible WAF, CAPTCHA and JavaScript challenges.
The documented sitemap discovery was bounded: at most eight candidates, 20,000 parsed sitemap URLs and an eight-second timeout per request. A field that did not confirm a URL in a sitemap is therefore not proof that the site had omitted it from every sitemap.
Automated flags are observations to examine. A CAPTCHA signal in page content does not by itself establish that a platform crawler was blocked.
A permission check is different from a visit
The study stored robots flags for ten user-agent tokens. It recorded 48 successful robots-file responses, one 406 response and one 404 response. Both non-200 records still have all ten allowed flags set to true; the public package does not document the collector’s response-handling rule. It also recorded the response obtained by its own collection client. Neither establishes that the named platform visited, indexed or cited the page.
| Role | Tokens in the June records |
|---|---|
| Search crawlers | Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot, PerplexityBot |
| Training crawlers | GPTBot, ClaudeBot |
| User-requested fetchers | ChatGPT-User, Claude-User, Perplexity-User |
Current platform policies differ. OpenAI says ChatGPT-User may not follow robots.txt; Perplexity says its user-requested fetcher generally ignores it. Anthropic describes its bots as honoring robots directives. Preserve the historical token-check values, but consult the current documentation before interpreting them as effective access controls. Blocking training is not automatically a search problem.
What manual review added
The documented review considered page role, audience, service clarity, proof, methods, attribution, limitations, next steps and obvious collection errors. The final file labels 22 rows semantic_review_approved and 28 semantic_review_approved_with_caveat.
The caveat label remains in the public export. Approval means a record was retained for this study; it is not an endorsement of the business or evidence that all recorded features were exhaustively verified.
How observations became scores
The historical process applied the DUCR rubric, incorporated semantic-review overrides and produced the final category scores. The retained rubric assigns 25 points to Discoverable, 25 to Understandable, 30 to Citable and 20 to Routable.
These weights express our framework’s priorities. They were not calibrated against observed citations, sales or lead quality in this study. Some criteria refer specifically to benchmark material and SEOI service routes, which limits their use as a general service-page standard. The score guide explains how to interpret a category before recommending work.
What you can reproduce from the public files
You can count the retained rows, inspect flags and review labels, add the four stored category scores, and recalculate the published averages, medians and distributions. Site IDs connect the CSV, JSON and score example.
The corrected public export removes direct identifying URLs, page text, freeform notes and linkable hashes. Stable site IDs can still be linked to earlier downloaded versions, so this is not a promise of irreversible anonymity. The original collection also recorded provenance fields, but the public package does not supply the underlying pages, collection program or complete per-criterion point ledger. It supports record-level analysis, not an independent replay of the full historical audit.
The dataset page documents the October 5 export correction. Collection dates, site IDs, typed observations and category scores are retained. No new sample or observation is implied.
Current guidance for interpreting the study
The following guidance was checked on October 5, 2026. Later platform changes are not represented as checks completed in June.
| Primary source | What it helps distinguish |
|---|---|
| Google AI optimization guide | Ordinary SEO and eligibility from selection; no special AI schema or prescribed format |
| Google robots introduction | Crawl instructions from indexing and actual retrieval |
| Google Search generative AI control | Current site inclusion settings, separate from the June public-page checks |
| OpenAI crawler documentation | Search, potential training and user-requested visits |
| Anthropic crawler documentation | Claude bot roles and controls |
| Perplexity crawler documentation | Search crawler and user-requested fetcher behavior |
| Google’s people-first content guidance | Relevant evidence and audience usefulness without a mandatory research template |
Official documentation informs the interpretation of particular fields. It does not validate our points or establish that a higher DUCR score causes a better outcome.
From study evidence to a recommendation
Use the decision guide to determine what more you need to know. For a specific business question, bring the page and the decision to a Clarity Call. Rough notes are enough; preparing a benchmark score is optional.
The Outside View
See where you stand with Google, AI and your buyers.
Send us your website. In two business days you have it in writing: how you look from the outside, who really competes with you, and the ten fixes that matter. Free.