To check whether AI bots reach your service pages, group requests by the job each page does, count distinct pages as well as requests, and inspect the responses. A busy bot chart can conceal a service page with no recorded request or repeated unsuccessful attempts.
Microsoft introduced Page Classification in Clarity Bot Activity on September 22, 2026. It combines content and URL rules, allows custom categories and gives custom classifications priority over automatic ones. For a B2B service business, this is a useful reason to review how pages are grouped.
The exercise below shows the decision that grouping can support. We use invented records and an explicit inventory, then run the grouping locally. It is not a Clarity account demonstration or a customer result.
Establish what your report covers
Clarity’s Bot Activity uses server-side logs from supported infrastructure integrations. Its setup starts in Settings > AI Visibility. Microsoft also documents automatic availability through the latest Clarity WordPress plugin. Having the ordinary tracking script on a page does not, by itself, establish that bot requests are being collected. Bot Activity setup and scope.
Before interpreting a blank or quiet report, record the connected source, exact hostname, first available data, last complete period and current filters. Confirm that requests for the relevant service URLs pass through that source. Keep www, the main hostname and subdomains separate until their coverage is established.
Use the timezone shown by the source. If you combine records, retain the original timestamps and document their conversion. Our exercise uses UTC deliberately; it is not a claim about every Clarity view’s timezone.
If collection is missing or incomplete, the next task is to establish coverage. Reporting zero at that stage would turn a measurement gap into a finding about the website.
Give each important page a clear role
Start with a small inventory that a service owner can review. Include pages a suitable buyer needs, including those with no recorded activity. Building the inventory only from a list of requested URLs would hide untouched pages from the denominator.
| Primary role | What the page helps a buyer do |
|---|---|
| Service | Understand the work, its scope and whether the firm fits. |
| Proof | Examine relevant work, evidence and its limits. |
| Guide | Understand a problem or make a related decision. |
| Contact | Reach the appropriate enquiry step. |
These are our editorial roles, not a claim that Clarity uses this taxonomy. A guide may support a service without becoming a second service page. Choose one primary role for this report and record any secondary purpose separately.
Review the actual pages behind a category. A /services/ prefix can contain an overview, an individual service and a nested buying guide. Names alone cannot settle their roles. Flat URLs need the same review.
Our workbook assigns one role to each exact hostname and path. /services/security/ and /services/security/assessment/ are separate records, so a child request is not also counted against its parent’s URL. Duplicate inventory keys stop the calculation. Unmapped pages remain visible for review.
This is the workbook’s rule. Microsoft’s announcement does not specify every overlap between custom classifications. Check your project’s results with a parent path, child path and exception page before adding category totals together.
Keep request volume, coverage and response status separate
Clarity supports domain and path filtering, request-status exploration and a content-type breakdown. That matters because a requested image or JSON endpoint is not automatically a requested service page. Microsoft’s Bot Activity metrics update.
For a comparable reading, keep the hostname, period, bot/operator selection, content scope and classification version together. Changing those between exports can change the result even if the website stays the same.
Use three different counts:
- Requests: how many matching requests were retained. Repeat requests still count.
- Distinct inventory pages requested: how many eligible page keys received at least one matching request.
- Distinct inventory pages with a recorded 2xx response: a narrower response measure, with missing statuses kept visible.
Divide a distinct-page count by the eligible inventory for the same covered scope. Do not divide requests by page count and call it coverage. Our inventory coverage is a defined worksheet measure, not a reproduction of Clarity’s pages-crawled calculation.
A 2xx response still needs a content check when access is in question. It may not contain the intended service information. Keep redirects, unsuccessful statuses and unknown statuses separate. A missing status is not a success.
Retain how the source identified the bot. A user-agent label alone is different evidence from a verified operator identity. The crawler access study explains why collector responses and permission flags should not be reported as genuine platform visits.
Worked exercise: six service requests reach three pages
Everything in this exercise is synthetic: domains, bot names, request times, statuses and coverage assumptions. No Clarity account or customer logs were used. The downloads retain the inputs, excluded records and a runnable grouping procedure.
The invented firm has nine pages in its inventory. Eight pages belong to advisory.example, whose sample request stream is assumed complete. One service page belongs to insights.advisory.example, whose collection is unknown.
The window starts at October 1, 2026, 00:00 UTC and ends immediately before October 8, 00:00 UTC. The file contains 17 individual request records. Two fall outside the window, one targets an asset, one has an unverified identity and one targets a page outside the inventory. They remain in the file with their exclusion reasons.
That leaves 12 eligible requests. In this teaching calculation, verified identity is an explicit synthetic assumption for the included records, not an authentication test we performed.
| Page role on the covered hostname | Requests | Distinct pages requested / eligible pages |
|---|---|---|
| Service | 6 | 3 / 4 |
| Proof | 1 | 1 / 1 |
| Guide | 3 | 2 / 2 |
| Contact | 2 | 1 / 1 |
There are seven distinct requested pages among eight eligible pages, or 87.5%. That headline needs the service detail:
| Service page | Requests | Recorded responses |
|---|---|---|
| IT support | 3 | Three 200 responses. |
| Security | 2 | One 403 and one 429. |
| Security assessment | 1 | One 200 response. |
| Cloud migration | 0 | No matching request in the assumed-complete sample. |
Service request coverage is 3 of 4 pages, or 75%. Only 2 of 4, or 50%, have a 2xx record. Neither percentage establishes citations or buyer interest.
Across all 12 eligible requests, the status breakdown is eight 2xx, one 3xx, two 4xx/5xx and one unknown. The unknown status belongs to a contact-page request and stays in the total. Counting only known statuses would quietly reduce the denominator to eleven.
The subdomain service page stays unknown. It does not become a fifth fully measured service page, and its missing requests do not become zero. Report the four covered service pages and the one uncovered page together so the limitation remains visible.
Turn the grouping into an assigned check
The fictional security page deserves a request-level investigation because its two records have unsuccessful statuses. Retain the exact request, identity evidence and responsible response or control before proposing a change. Our Cloudflare access guide helps when that infrastructure is involved.
The cloud-migration page needs a different check. Zero means no matching request in this defined synthetic stream. It does not prove a block, missing index entry or poor content. Review the observation period, intended crawler access and how the page is discoverable before choosing a repair.
Before reading a zero as a block, run the page through our free AI Crawler Checker. It shows what robots.txt allows each crawler, how the server answers requests made in their names and how much text arrives without JavaScript. A clear result moves the question to discoverability; a Fix label names what to change first.
For the subdomain, establish collection coverage first. For the unmapped page, review its purpose and decide whether the inventory should include it in the next version. Keep that version change in the record instead of rewriting the old denominator.
The workbook leaves these actions proposed. Completing the classification exercise does not complete the access investigation.
Read referrals beside requests
Microsoft’s August 13 Scrape-to-Referral update adds an operator comparison, mapped-domain safeguards and links to filtered recordings. Check the covered domains before using such a comparison. A request total and a referral total can describe different populations.
Our exercise has no referral records, so its referral measure is unavailable. There is no legitimate calculation that turns these twelve requests into visits or enquiries. Even compatible aggregate totals would not prove that a particular request produced a particular visitor.
Use the ChatGPT referral guide for recorded visits and actual enquiry receipt. Use the monthly citation log for retained answer evidence. Keep those observations separate from the request ledger.
Download the page-role record
Download the blank CSV, the synthetic worked CSV or the Markdown workbook and grouping instructions.
Bring the inventory, coverage limits and next investigation into your review. Our AI SEO work uses evidence about important pages to choose the work. Request a free Clarity Call to discuss that decision. Keep private request logs in your team’s controlled records.