AIAIAI.FYI
LIVE

Eedi and Google DeepMind trial of 1,179 students finds AI tutor matched human tutors in-session, not on test

After 10 weeks, neither AI nor human tutoring moved scores on the Star Maths standardized test, and tutored boys tended to do worse than boys given static practice.

What happened

A London ed-tech startup, Eedi, and Google DeepMind tested an AI math tutor on 1,179 students in 10 U.K. public schools. When students got a question wrong, the AI helped them fix the mistake about as well as an expert human tutor. But a standard math test taken after 10 weeks showed no gain from either kind of tutoring.

Why it matters to you

Schools often cannot afford human tutors, so an AI that works as well in the moment is attractive. This study shows that doing well in the moment did not yet translate into higher scores on a broad test.

The takeaway

AI tutoring helped students fix mistakes in the moment, but this trial found no lift on a broad math test.

What it means for you

If you build software

The AI asked follow-up mastery questions more than 90% of the time versus about 60% for human tutors, and 11 drafts contained false math statements, so over-questioning and accuracy checks need attention.

If you run a business

Eedi's chief impact officer says human tutors are expensive and keeping humans approving each AI message is not scalable, which is why the next trial reduces human oversight.

Get stories like this every morning