Visa benchmark finds 22 AI models failed 42.8% of shopping test runs
Visa says 54% of failed runs still ended in a purchase, and all three attempts passed in only 34.2% of scenario runs.
What happened
Visa built a test that asks AI shopping assistants to buy things like a TV or laptop. The AI has to chat with a shopper, find out what they really need, then buy the right item or decide not to buy. Across the tests, the models failed 42.8% of runs, Visa says.
Why it matters to you
If you let an AI shop for you, it may buy the wrong thing. Visa says 54% of failed runs still ended in a purchase.
The takeaway
AI shopping agents can work, but they are not yet reliable enough to trust with your money without checking.
What it means for you
Consistency is the weak point: models that passed a scenario often failed when repeated, so test agents across multiple runs.
Merchants and payment firms can use the public benchmark to see where AI shopping works today and where it fails.
Visa, which reports the results, ties the benchmark to its Visa Intelligent Commerce effort, and says the two are not directly linked.