AEOsome’s AI shopping study: what the corrections mean

Before using an AI shopping league table to choose where to spend your marketing budget, check which models were tested, when answers were collected and what the metric counts. AEOsome’s corrected research offers a useful example of why those details matter.

What changed in the study

In a corrections chapter dated 11 October 2026, AEOsome describes eight changes to earlier reporting. It says its aggregate ChatGPT product-card rate combined different served models, whose results varied substantially. Some intent tables also included duplicate retries despite being labelled deduplicated.

The vendor further says an earlier association between urgency language and product cards disappeared when it used the correctly tagged, deduplicated data. These are corrections from reanalysing existing observations, not a fresh collection of shopping answers. We have reviewed the published explanation, not independently reproduced its dataset.

Understand the observation window

The study’s current research hub describes its second dataset as 22,335 answers after removing duplicate retries, collected from 5–14 May 2026 across four engines and seven product categories. Collection used logged-out, US-English consumer web interfaces.

A product card means a structured result with product information and a merchant link; a brand mentioned in ordinary answer text is a different outcome. The study did not collect merchant-feed data, so its observations cannot establish a store’s feed eligibility. Its separate studies also changed categories, dates and models, limiting comparisons over time.

Make your own checks comparable

Our recommendation is to start with questions real customers ask about your products. Keep separate records for brand mentions, product cards, visits and orders. A card appearing in an answer is not a sale.

Save the exact question, date, market and interface, plus the model when it is visible. Repeat comparable checks before treating one answer as a pattern. When your testing setup changes, mark the break in the record rather than presenting the results as a continuous trend.

Use third-party studies to design questions worth investigating. Avoid rewriting product copy or shifting a channel budget solely because an aggregate platform percentage looks impressive. For the operational work, our Shopify agentic commerce guide covers product-data and channel checks.