Last verified: October 11, 2026. Every figure below was checked against its source. Where a source is a vendor study or a secondary write-up, the text says so.
Direct answer: The evidence supports adding an AI shopping assistant on top of crawlable category pages. It does not support replacing them. The large conversion numbers behind the “replace” argument come mostly from shoppers who chose to use the assistant, and the strongest controlled experiments tested an assistant added to an existing store, not a store with its category pages removed. I searched for a published test of the replacement and found none.
Retailer pitches for conversational commerce usually make one jump. They start with a number like “assistant users are 60% more likely to buy” and end with “so retire the category grid.” The first half is documented. The second half has no published test behind it.
This article separates the two halves, then covers what to build instead and how to measure whether it works.
A note on names: Amazon retired the Rufus name in the US on May 13, 2026, folding it into a combined assistant called Alexa for Shopping. The studies below were done under the Rufus name, so this article uses it.
What the headline numbers measure
Amazon’s assistant is the largest deployment, so most public numbers come from it. They measure different things, and they come from sources of different quality.
| Claim | Who reported it | What it actually measures | Confidence |
|---|---|---|---|
| Rufus users are 60% more likely to complete a purchase; 300M customers used it in 2025; nearly $12B in incremental annualized sales | Amazon, as reported by Modern Retail (April 2026) | Company-stated figures. Methodology not published. | Medium: first-party, unaudited |
| Rufus sessions convert at 3.5× the rate of non-Rufus sessions (100,000+ sessions, 2025 holiday season) | Sensor Tower, as reported by The Drum (January 2026) | Observed conversion on modeled panel data; any session with even a brief interaction counts as a Rufus session | Medium-low: correlational |
| Rufus users are 2.74× more likely to buy | Sensor Tower blog (April 2026) | Observed purchase likelihood, sessions with vs without Rufus | Medium-low: correlational |
| Purchase sessions that included Rufus rose 75% day over day on Black Friday vs 35% for those without (100% vs 20% against the trailing 30 days) | TechCrunch, citing Sensor Tower (December 2025) | Growth in the number of purchasing sessions. It is not a conversion lift. | Low as evidence of effect |
Usage keeps growing. On its Q2 2026 earnings call, Amazon said more than 350 million shoppers had used the assistant over the prior 12 months, with interactions up five times year over year. In the coverage I reviewed, there was no updated conversion or incrementality figure with a published method.
Three problems run through these numbers.
The metrics don’t match. Amazon’s “60% more likely to complete a purchase,” Sensor Tower’s “3.5× conversion rate” and its “2.74× more likely to buy” are different quantities, taken at different times with different definitions. Sensor Tower’s April write-up also describes roughly double the conversion rate in the same piece that reports 2.74×. The Black Friday figures are weaker still. A growth rate in purchasing sessions between two days says nothing about intent or causation, yet one secondary write-up (Novadata) presented the 100% vs 20% gap as proof that AI-assisted discovery improves purchase intent. Treat that as a correction to circulating coverage.
Selection is built into the data. The columnist who reported Sensor Tower’s holiday study for The Drum flagged the limits: heavy Amazon shoppers, who tend to convert anyway, may also be the likeliest to open the assistant; the study counts brief interactions as Rufus sessions; and the figures rest on modeled panel data that may be revised.
Sensor Tower’s own breakdown of shopper behavior shows how the pool leans toward people already close to buying. Shoppers it labels “Early Deciders” ask one quick question at the start of a session and convert at about 85%. “Cart Reconsiderers,” about 7% of sessions, already have items in the cart and convert at about 50%. The most common pattern, “Search Assistant” (about 28% of sessions, about 35% conversion), is a shopper who searched several times without finding the right product and turned to the assistant to get unstuck. Sensor Tower’s own conclusion is that the assistant surfaces existing demand more than it creates demand.
The Search Assistant pattern is the one to notice. My reading, which no source tests: when navigation fails, shoppers reach for the assistant. That is an argument for fixing navigation, not for deleting it.
My judgment: these figures prove that shoppers who engage an assistant are high-intent. They do not prove the assistant creates intent. Both can be true at once, but only a holdout test separates them, and Amazon has not published one.
The causal evidence, and what it left untested
The strongest evidence I found is a randomized field experiment: Fang, Yuan, Zhang, Donati and Sarvary, “Generative AI and Firm Productivity: Field Experiments in Online Retail” (arXiv:2510.12049, also CESifo Working Paper 12201; the latest revision, June 2026, is retitled “Generative AI and Sales Productivity”). Between September 2023 and June 2024, GenAI features were rolled out across seven consumer-facing workflows on a leading cross-border retail platform, with users or products randomly assigned.
Columbia Business School’s summary of the paper reports that the AI pre-sale chatbot produced the largest gain: sales up 16.3% and conversion up 21.7%. Gains were larger for inexperienced shoppers (conversion up 26%) than for veteran buyers (about 20%). Across all seven workflows, sales effects ranged from 0% to 16.3%, and one (AI-generated Google ad titles) came out negative, though not significantly. Across the workflows, conversion rose with no significant change in average cart value, and product returns and customer ratings did not get worse.
A second randomized experiment points the same way. Wang et al. (Information Systems Research, 2025) tested an AI assistant on a livestream selling platform and found sales up 3.00% and product returns down 12.55%.
These are causal results, and they are the best evidence for the “add” side. They still stop short of the “replace” claim, for five reasons.
- Both tested an addition. In the Fang chatbot experiment, the control group received no pre-sale customer support, and the store’s browsing experience stayed intact. Nobody removed the category pages.
- The Fang chatbot answered service questions. It was a pre-sale service chatbot, not a guided-selling system that elicits attributes and narrows a catalog.
- It was one platform. The setting is a cross-border marketplace selling mostly from small businesses, with support in about 20 languages. Generalizing to a mid-market fashion or electronics store is an inference. The paper also discloses that one author is an employee of the partner platform and two others served as its consultants.
- The models are from 2023-2024. Assistant quality has moved since, in both directions: better models, but also more ambitious agentic features.
- The livestream trial is a different format. It tested an assistant inside live video selling, which is further still from a category-page replacement.
The practical reading: an assistant added to a store raised conversion in controlled tests. Whether it can carry the load of a category page is an open question.
Category pages are often bad, and that is the argument for fixing them first
The best case for assistants is that category pages and filters are frequently terrible. Baymard’s research backs that up. In its product list usability study, sites with mediocre product-list tooling saw 67-90% abandonment on product-finding tasks, against 17-33% for sites with just slightly better tooling, with participants hunting for the same kinds of products. Baymard’s current benchmark of 170+ manually scored sites rates 58% of desktop sites and 78% of mobile sites “mediocre” or worse on product-list UX. Baymard also finds that 38% of sites don’t offer filters for attributes they already display in listings.
Read those numbers for what they are. They come from lab tasks and a particular benchmarked set of sites, so they describe the typical failure and not a universal rate. I could not find a test date for the 67-90% result on Baymard’s page, and Baymard has since re-tested product finding (a 2024 update covering 12 sites). The failure they describe, though, is specific: a missing attribute taxonomy. Shoppers can’t narrow by something the site clearly knows.
This is the part most coverage misses. A guided-selling assistant is an attribute-elicitation system. When it asks “how much space do you have, and do you cook for more than two?”, it is collecting values for attributes the catalog must already hold in structured form. If your category pages are weak because attributes are missing or inconsistent, the assistant inherits the same defect, and fluent language hides it. The assistant doesn’t replace the data work. It makes the data layer matter more.
Why removing category pages carries a search cost
Category pages are also how search engines understand a catalog. Google’s faceted navigation documentation says filter combinations that generate unique URLs can cause overcrawling, and that you can either block them or, if you want them indexed, follow specific URL practices. It also states that filters implemented with URL fragments have no effect on crawling, positive or negative.
Google’s link guidance adds the second half. Google can generally crawl a link only when it is an <a> element with an href attribute, and it can’t reliably extract URLs from elements that act like links only through script events. A chat panel that exists only as client-side state produces no URLs at all.
The consequence is simple. If a shopper’s journey (“quiet dishwasher for a small apartment”) ends in a recommendation but no indexable page, the URL that would capture that demand never exists for search.
The design rule below keeps both halves working:
| Layer | What lives there | Rule |
|---|---|---|
| Structured attribute catalog | Specs, availability, compatibility, regulated-product fields | One source of truth that feeds every layer above it |
| Crawlable category pages | Real URLs, static <a href> links | Filters on every attribute you display |
| Guided-selling assistant | Asks questions, fills attributes | Resolves to a real, linkable category URL, never a dead-end chat transcript |
| Shopper (and crawler) | Uses either path | Both paths end on the same indexable pages |
Where assistants fail: product truth
Assistants fail in a way category pages don’t: they fail fluently. A filter that returns zero results looks broken. A confident, wrong answer looks fine.
The independent evidence here is thin and mostly vendor-run, so weigh it accordingly. Smarter Sorting, a product-data and regulatory-classification company, published a January 2026 test of five general AI shopping systems (ChatGPT, Gemini, Perplexity, Claude and Copilot) across 100 products, 10 categories and 2,500 interaction steps. It found availability lookup failing in over half of cases, along with gaps in completeness for regulated products and in checkout execution.
A September 2026 study from Product.ai, a startup that checks product claims against evidence, asked 220 shopping questions five times each across free and paid tiers of ChatGPT, Gemini, Claude and Perplexity. Of the 217 questions with complete answer sets, 86% produced at least one confirmed factual conflict. Head-to-head comparison questions did worst, with conflicts on 97% of them, against 75% for straightforward factual questions. The headline “wrong 86% of the time” in some coverage overstates this: the 86% is the share of questions that produced a conflict, not the share of answers that were wrong. The tests also ran through provider APIs rather than consumer apps, and Google said its consumer app draws on live product listings that the API does not.
Both studies test general-purpose chatbots, not retailer-built assistants grounded in a catalog, so they say little about how often a well-built assistant errs. What they share, and what is plausible on mechanics, is where errors cluster: availability, compatibility, price and regulated-product details. Nearly all of those trace to data, not language ability.
Rank this evidence below the Fang experiment and above the Rufus panel figures on the narrow question “does it make mistakes.” It is the weakest of the three on “how often.”
What to build, by category type
This part is my analysis. No source I found tests it.
- Considered purchases with interacting constraints (appliances, mattresses, cameras, anything with compatibility or room-size logic): put the assistant first. These are the cases where a grid forces the shopper to translate a need into specs. Keep the category URL as the landing page the assistant resolves to.
- Commodity or repeat items (refills, batteries, basics): keep the grid and filters first. A conversation adds steps to a task that needs none.
- Deal-driven browsing: leave the grid alone. Sensor Tower reported that even heavy Rufus users temporarily dropped the assistant during Amazon’s Big Spring Sale, which suggests assistants serve decisions more than bargain-hunting.
- Regulated or safety-relevant products: the assistant may narrow, but availability, dosage, fitment and compliance fields must come from the catalog, never from model generation.
Three engineering rules follow from the evidence above:
- Add a filter for every attribute you display in listings. It is the cheapest fix with the best evidence behind it, and the assistant will need those fields.
- Make every assistant outcome a shareable, crawlable category or product-set URL.
- Log the assistant’s attribute choices per session. That log shows which attributes shoppers actually think in, information a click-stream on category pages can’t give you.
How to find out whether it works
You can’t answer “does it cause lift?” with engagement data. Run a randomized holdout: show the assistant to a random share of eligible visitors and not to the rest, then compare conversion and revenue per visitor over the same period.
A sizing illustration (my assumptions, not benchmarks): suppose baseline conversion is 2.0% and you want to detect a 10% relative lift (to 2.2%) at 80% power and 5% significance. A standard two-proportion calculation gives about 80,700 visitors per arm. A store with 10,000 eligible visitors a week, split evenly, needs about 16 weeks. A 20% relative lift (to 2.4%) needs about 21,100 per arm, or roughly 4 weeks on the same traffic. That is the real reason most published numbers are observational: clean tests of small effects are expensive for small stores. If you can’t reach the sample, test on your highest-traffic considered-purchase category only, and say clearly that the result applies to that category.
Watch average order value alongside conversion. In the Fang experiments, average cart value didn’t move while conversion did, so revenue per visitor tracks conversion. If your AOV changes, check whether the mix of buyers changed before crediting the assistant.
The question this article raised, answered
Can category pages be replaced? Not on current evidence, and probably not at all. The experiment that would settle it: take two comparable categories, serve one with an assistant plus a crawlable grid and the other with an assistant only (grid noindexed or removed), and compare organic landings, conversion and returns over at least a quarter. As of October 2026, I found no published version of that test. Until one appears, “replace” is a forecast, not a finding.
Frequently asked questions
Do AI shopping assistants increase conversion?
In the two randomized experiments I found, yes, when added to an existing store: sales rose 16.3% (pre-sale service chatbot) and 3.00% (livestream assistant). The much larger figures from Amazon and Sensor Tower are observational and mix the assistant’s effect with the fact that high-intent shoppers choose to use it.
Can an AI assistant replace category pages?
No published test supports it. The controlled experiments tested an assistant added to an existing store, and removing crawlable category pages also removes the URLs that search engines index.
Do AI shopping assistants hurt SEO?
Not by themselves. The risk comes from replacing crawlable pages with a chat panel that creates no URLs. Google generally crawls only links that are <a> elements with an href, so assistant outcomes should resolve to real category or product-set URLs.
How do I test whether an assistant works on my store?
Run a randomized holdout and compare conversion and revenue per visitor. At 2.0% baseline conversion, detecting a 10% relative lift takes roughly 80,700 visitors per arm.
Sources
- Modern Retail, “Amazon says its AI shopping assistant is gaining traction, with Rufus users up 115%” (April 30, 2026): https://www.modernretail.co/technology/amazon-says-its-ai-shopping-assistant-is-gaining-traction-with-rufus-users-up-115/
- Retail Dive, Amazon Q2 2026 earnings and Alexa for Shopping adoption: https://www.retaildive.com/news/amazon-alexa-for-shopping-agentic-ai-adoption/826806/
- GeekWire, “Amazon unifies Alexa+ and Rufus” (May 2026): https://www.geekwire.com/2026/amazon-unifies-alexa-and-rufus-as-ai-rivals-move-into-online-shopping/
- Sensor Tower, “From Scroll to Sold: What Amazon’s Rufus Really Tells Us About Shopper Intent” (April 2026): https://sensortower.com/blog/scroll-to-sold-what-amazon-rufus-tells-us-about-shopper-intent
- The Drum, “While everyone debates agentic shopping, Amazon’s Rufus is racking up sales” (January 29, 2026): https://www.thedrum.com/opinion/while-everyone-debates-agentic-shopping-amazon-s-rufus-is-racking-up-sales
- TechCrunch, “Amazon’s AI chatbot Rufus drove sales on Black Friday” (December 1, 2025): https://techcrunch.com/2025/12/01/amazons-ai-chatbot-rufus-drove-sales-on-black-friday
- Novadata, “Rufus AI Drove 100% Purchase Increase on Black Friday 2025”: https://novadata.io/resources/news/rufus-ai-black-friday-100-percent-purchase-lift
- Fang, Yuan, Zhang, Donati and Sarvary, “Generative AI and Firm Productivity: Field Experiments in Online Retail,” arXiv:2510.12049: https://arxiv.org/abs/2510.12049
- Columbia Business School, “Generative AI Is Quietly Supercharging Online Retail” (November 18, 2025): https://business.columbia.edu/insights/digital-future/ai-supercharging-online-retail
- Wang et al., “Artificial Intelligence (AI) Assistant in Online Shopping: A Randomized Field Experiment on a Livestream Selling Platform,” Information Systems Research (2025): https://pubsonline.informs.org/doi/10.1287/isre.2023.0103
- Baymard Institute, product list and filtering research: https://baymard.com/research/ecommerce-product-lists
- Baymard Institute, “The Current State of E-Commerce Product List UX”: https://baymard.com/research-articles/current-state-product-list-and-filtering
- Baymard Institute, “Filter List Design: Have Filters for All Displayed List Item Info”: https://baymard.com/research-articles/have-filters-for-list-item-info
- Google Search Central, “Managing crawling of faceted navigation URLs”: https://developers.google.com/crawling/docs/faceted-navigation
- Google Search Central, “Link best practices for Google”: https://developers.google.com/search/docs/crawling-indexing/links-crawlable
- Smarter Sorting, “How We Tested 5 AI Shopping Assistants Across 2,500 Interactions” (January 28, 2026): https://www.smartersorting.com/post/how-we-tested-5-ai-shopping-assistants-across-2-500-interactions
- TechRepublic, coverage of the Product.ai study (September 2026): https://www.techrepublic.com/article/news-chatgpt-gemini-product-recommendations/
- Business Insider Japan, coverage of the Product.ai study including Google’s response: https://www.businessinsider.jp/article/2609-study-shows-ai-errors-shopping-tools-struggle-accuracy/
- DNYUZ (Business Insider repost), “AI can’t take over your holiday shopping quite yet”: https://dnyuz.com/2026/09/25/ai-cant-take-over-your-holiday-shopping-quite-yet-a-new-study-suggests/
