We'd like to use analytics tools to see how this site is used and what visits from AI assistants lead to. Not for ads. Decline: only a cookie-free page count runs. What we store.

Arbling

AI crawler visits: how do you tell bots from buyers?

Published Nov 15, 2026Intermediate · 11 min read

What you’ll learn

  • Separate requests, shopper visits and linked orders.
  • Check which traffic your setup can observe.
  • Diagnose a sudden rise or fall without guessing.

What do AI crawler visits measure, and where does coverage stop?

AI crawler visits in 2026 are requests that automated systems make for your pages, and they sit on a different measurement layer from shopper sessions and orders. A request means a machine asked your server for a page. A shopper session means a browser loaded the page and someone, or someone's agent, moved through the store. An order means a checkout was completed and recorded. Treating the three as one number can make a traffic dashboard seem to contradict a sales report. When a crawler fetches a product page, nothing has been bought and nothing has been attributed. In the same way, an AI referral is a visit that arrived from an assistant, and it is not by itself an attributed order until a recorded link ties it to one. Keeping the three layers apart, each with its own name and its own count, is the starting point for every diagnosis below.

Coverage decides what each layer can even see. A page script, the small piece of code a store adds to its pages, observes browsers that actually execute it. A crawler that downloads the page and leaves never runs that code, so a script alone cannot report it. Requests that never execute the script appear only in server or edge records, meaning the logs or analytics kept by your host or by a network layer in front of your site. Cloudflare's AI Crawl Control documentation describes that kind of request-level view, with crawler request volume, requested paths, response status and operator breakdowns. Nobody should promise complete crawler visibility from a Shopify page script, and this article does not. Coverage has to be checked for your own setup: which layer records what, on which pages, and where a gap exists.

Platform reports can also shift the picture. Shopify's September 2026 measurement update filters identified bot sessions out of session-related reports by default, and supported reports let readers adjust the human-or-bot filter. Shopify says the update can change session and conversion-rate comparisons without changing total orders, sales or customer counts. That describes how reports are built, so it is a reason to check the filter before trusting a before-and-after comparison, not a diagnosis of any particular store. A September 2026 merchant discussion welcomes the filtering because bot-heavy sessions distorted the author's reading of conversion rate. It is a reported experience, not a benchmark.

Shopping agents, which act on a buyer's behalf and belong to what people call agentic commerce, add another kind of visitor to this picture. They fetch pages for a person who may or may not buy later, so an assistant fetch needs its own label instead of being folded into crawler requests or into human visits.

How do you classify each kind of visit without guessing?

Classifying AI crawler visits in 2026 means giving every recorded event one label, backed by the evidence that supports it, and keeping an unknown label when the evidence is thin. Two shortcuts mislead people. A claimed user agent is a text string that the sender chose to declare, so by itself it is not verified identity. A missing referrer does not prove a bot either, because privacy settings, apps and ordinary browsing paths remove referrers from real people. Better classification combines several signals: the recording method, the requested paths, the response status, whether a script ran, what happened after the first page and whether an order record exists. The table in this section lists six kinds of measurement, the decision each one supports and the limit it carries. Use it as a labeling guide, and treat the unknown row as a legitimate outcome, not a failure.

MeasurementWhat it recordsDecision it supportsLimits
Crawler requestA server or edge record of an automated system requesting a page, with path, response status and operator when knownWhich pages automated systems read, and whether your server answers them properlyNot a shopper session and not a sale. A claimed user agent alone is not verified identity
Assistant fetchA request made by an assistant or agent while answering or acting for a personWhich pages assistants read when they handle buyer questionsShows a page was read, not that the person saw it, visited or bought
Human AI referralA browser visit that arrived from an AI assistant, recorded with its landing pageWhich assistant answers send people to which pagesAn AI referral is not an attributed order. A missing referrer never proves a bot, and a present one never proves intent
Ordinary visitorA browser session with no AI signal, such as search, direct or emailA baseline to compare AI visits againstThe lack of an AI signal does not rule out earlier AI influence
Linked orderAn order tied to a recorded visit through a named linking methodWhich visits led to orders, under that methodOnly as strong as the method. An order with no recorded visit stays unlinked
Unknown classificationAn event whose evidence supports no other labelWhere to improve recording or classificationNever count it as a bot, as a human or as zero. Keep it visible

Only the linked-order row supports a claim about sales, and even that claim carries the name of the method that made the link. If you cannot name the method, you have a guess about timing, not a linked order. When two rows seem to fit one event, pick the weaker label and write down why.Repeat counting needs a plain rule, offered here as a measurement design recommendation. A genuine new request or page visit is a new event, even when the same visitor or crawler comes back to the same page, so repeat visits stay in the count. Sending the same event twice, for example because a delivery failed and was retried, is delivery duplication, and the retry should be recognized as the same event so it is not counted again. Unique people, visits and requests are three different metrics. Several requests from one crawler are several requests, one operator and, depending on the method, possibly one visitor. Whichever metric you report, name the unit beside the number, and check how your own tools count before comparing figures from two of them.

How do you diagnose more traffic with no extra sales, or a sudden drop?

A rise in AI crawler visits with no extra sales, or a sudden apparent drop, is diagnosed with the same six checks in a fixed order, and since September 2026 one of those checks is the human-or-bot filter on session reports. The checks are the recording method, the bot classification, the landing page, the follow-up behavior, the order evidence and any change in filters. Each check rules out one cause before you look for the next. Start by checking whether the recording method changed, a filter hid a category, or a crawler read many pages and no person followed. The word apparent matters when traffic seems to fall, because a measurement change can move a chart while orders stay level. Work through the checks below, and write down what each one shows, including the cases where the honest answer is that you cannot tell.

If traffic rose but sales did not:

  • Recording method. Did you add or change a script, an edge rule or a report? A new method can reveal activity that was always there.
  • Bot classification. Are the extra visits crawler requests or assistant fetches instead of browser sessions? If so, more requests is expected and says nothing about buyers.
  • Landing page. Do the added visits cluster on product pages, a feed or a page no shopper would open?
  • Follow-up behavior. After the first page, did anything happen, such as a second page, a cart or a checkout? Fetches that stop at one page look like reading, not shopping.
  • Order evidence. Is any order linked to a recorded AI visit under a named method? If none is, write "no linked order", not "no AI sales".
  • Filter changes. Did a report default or a filter change on the dates in question? Shopify's update is one candidate to rule in or out.

If traffic seemed to drop suddenly:

  • Filters first. A human-or-bot filter that now hides sessions can lower session counts without lowering orders. Compare orders across the same dates. Stable orders are a reason to investigate the report; they do not rule out a real traffic change.
  • Recording method next. Did a script fail on some pages, a tag change, or a logging rule stop? A silent gap looks exactly like a drop.
  • Classification after that. Did visits move into the unknown group, or into a bot group, because the labeling rules changed?
  • Landing page and follow-up behavior. Did one entry page lose its visits while others held steady?
  • Order evidence last. Stable orders do not prove a measurement problem. If orders fall too, also investigate demand, stock and checkout behavior.Three kinds of source answer different pains here. If your pain is session comparisons that shifted, Shopify's measurement update page explains the filtering, though it does not say which assistant fetched which page. If your pain is not knowing what is requesting your pages, Cloudflare's AI Crawl Control documentation describes request volume, paths, status and operators, which are request-level facts, so shopper sessions and orders still need their own records. If your pain is telling AI-referred shoppers, shopping agents and crawlers apart and tying visits to orders, Arbling's product page describes distinguishing those groups, with the pages and activity observed, and following recorded visits to carts, checkouts and orders. None of this argues for blocking AI crawlers by default. Classify first, then decide page by page what you want machines to read.

For the definitions behind these labels, the guide on AI traffic analytics versus AI attribution covers what counters cannot tell you, and the one on product visibility in AI follows the path from recommendation to sale. As additional reading, a separate case study on telling bot requests apart in e-commerce traffic looks at a different situation, so this article does not repeat it.

Where Arbling fits. Arbling's Beacon records the visit and order evidence in this workflow. Arbling's product page distinguishes AI-referred shoppers, shopping agents and crawlers, including the pages and activity observed, and follows recorded visits to carts, checkouts and orders. It names the linking method and marks missing evidence as missing, so coverage that is unknown stays visible instead of being counted as zero. It then joins that product visibility with catalog-fact fixes approved by the merchant and repeated buyer-question measurements. Product questions lead to approved fixes, the fixes are followed through visits and orders, candidate patterns can be tested on other suitable products, and the same questions are measured again. To see how that loop is described, open the Arbling product visibility page.

Sources

See what AI assistants say about your business. Free scan, about a minute.

See who AI recommends
Share
Was this guide helpful?

Get new guides in your inbox.

No spam — one email when we publish something worth reading.

Yevgeniy Makhmutov
Founder, Arbling

Founder of Arbling. Arbling makes product catalogs readable, trusted, and buyable by AI shopping agents across regulated verticals.

Related guides

Related concepts