What Shoptank’s 60 search catalogue test actually shows

Shoptank has published a 60 search experiment showing which stores appeared most prominently in its vitamin D searches of Shopify’s global product catalogue. It offers a useful example of measuring visibility, but does not establish why those products ranked or what an AI assistant ultimately recommended.

The AI visibility vendor’s 16 October results and methodology describe 15 queries repeated for the United States, Canada, the UK and Australia. Queries covered generic terms, formats and product attributes.

What the score measures

For each query, Shoptank scored a store’s highest placed relevant product: ten points for first, nine for second, declining with position, with at least one point for appearing. It then added scores across the 60 searches. This is Shoptank’s scoring system, not a Shopify quality rating.

Black Girl Vitamins led its combined table with 223 points, followed by Nutrition Geeks with 174 and Healf with 145. Shoptank reported seeing 259 products. These are vendor reported results from one category on one day, not an independently replicated benchmark or an assessment of the supplements.

What it cannot establish

Shoptank highlights clear names, product variety and strong reviews among successful listings. Those observations can suggest things to investigate. They do not isolate which attribute caused a ranking: the published method does not describe changing one factor while holding the others constant.

Its methodology names Shopify’s catalogue as the search surface. It does not report a controlled comparison of complete answers from different AI assistants. A catalogue result position should therefore be recorded separately from whether an assistant recommends the product to a shopper.

A useful audit starts with separate measurements

For your own category, choose a fixed set of customer questions and record the exact query, market, date and surface tested. Keep catalogue results in one record and complete assistant answers in another. Retain the actual results, including absences, rather than only a combined score.

Use the findings to investigate inaccurate or incomplete product information first. If you change a title or attribute, record what changed and repeat the same checks over time. A subsequent improvement is a signal to investigate, not proof that the edit caused it. Do not expand inventory or promise a ranking lift on this single experiment.

Shoptank sells visibility tracking, so its commercial interest belongs alongside the findings. The value here is a transparent starting question and a published scoring method, rather than a proven formula for winning AI recommendations.

Shoptank’s original results announcement, 16 October 2026

The vendor’s original announcement is embedded above. Read it on X.

19 October gift experiment: keep the same limits

Shoptank’s second experiment ran 60 gift searches across the same four countries, using local price caps. Its published page says six of the top ten products named the recipient in their titles. This is an observation about the results, not a controlled test proving that adding a recipient changes rankings.

The vendor reported a self-care or pamper box leading each country’s table. That does not mean the same product won everywhere: its overall winner, Cozy Gift Box for Women, appeared only in the US searches. Keep market-level results separate when evaluating an international campaign.

For giftable products, check that titles accurately identify the product and intended recipient where relevant. Record any edits and repeat the same queries; avoid creating near-duplicate pages or adding unsupported claims just to match a vendor’s winning examples. Read Shoptank’s original 19 October announcement.