Why Shopify Sidekick uses specialist AI models
Shopify’s GraphQL specialist delivered lower latency and serving costs in its own test. Here is what that does—and does not—mean for merchants.
Shopify has explained how Sidekick combines a general-purpose frontier model with specialist open models for focused tasks. Its engineering report, published on 21 October, describes a GraphQL specialist built from NVIDIA Nemotron 3.5 Lightning.
In Shopify’s traffic-replay test, median full-response latency was 0.87 seconds against 6.50 seconds for its frontier-model baseline, with roughly 90% lower serving costs. These are results for a specific backend workload—not a promise that every Sidekick conversation becomes seven times faster, or that merchants’ bills fall.
Shopify reports that post-training raised the specialist’s score on its GraphQL judge from 37.7 to 74.5, compared with 73 for the frontier baseline. These are company-reported evaluations, not an independent benchmark or proof of error-free answers.
What merchants should take from this
Our practical takeaway: assess Sidekick on the work you need completed. A faster response is useful only when it retrieves the right records and preserves the meaning of your request.
For a repeatable check, choose a store question whose answer you already know. Specify the reporting period, currency and whether you mean orders or individual items. Compare the answer with the underlying records, then try a less tidy example involving a refund or a changed order. Keep a note of mismatches rather than treating one successful demonstration as a reliability guarantee.
For app teams, the interesting design choice is the division of labour: broad conversational reasoning can sit alongside narrowly evaluated tools. The lesson is to define a measurable task and a meaningful failure case before deciding which model should perform it.
The original announcement
Shopify Engineering — 23 October 2026
More from AI
See all →
