When AI tutors help: the studies with good news

The evidence in one table

The Harvard trial

Kestin and colleagues tested a custom AI tutor in an introductory physics course. Each student had one lesson at home with the AI tutor and one lesson in an active class, in a random order. So every student was compared with themselves.

• Test scores. The median score after the lesson was 4.5 with the AI tutor and 3.5 after the class.
• Learning gain. The median gain was more than double with the tutor. The effect size was 0.63.
• Time. A median of 49 minutes with the tutor, against 60 minutes in class.

The design is the point. The tutor was instructed to keep students active and to walk them through each part step by step. Solutions written by experts were placed in its instructions so it would be accurate.

The hint-only tutor in schools

In the Bastani field trial, one group of students practised maths with a tutor version of GPT-4 that gave hints, not answers. Their practice scores were 127% higher than for students with no AI. On the exam with no AI, the 17% drop seen in the plain chatbot group was largely avoided.

Read that with care. The good news is that the harm mostly went away. The study does not report that the hint tutor raised exam scores above the group with no AI.

What the review of 52 studies says

In November 2026 the Stanford SCALE Initiative reviewed 52 studies that test cause, most of them in colleges and universities and many of them preprints. Its conclusion: effects depend on design and use. AI used to explain or to guide practice looks more promising than AI that produces the answer. But even that does not reliably improve learning, and students often like AI tools without any measurable gain.

The review lists results in both directions. A trial in Sierra Leone with 1,423 students found a maths gain of 0.258 standard deviations. A trial with 6,997 US students in grades 6 to 8 found no overall gain in later learning. And one study of 199 US college students found that immediate answers with explanations beat layered hints.

The result that went the other way

In October 2026 a team at the University of Maryland posted a randomised trial with 2,379 undergraduates and 30 instructors. Within the same course, access to an AI tutor lowered final grades by 0.37 standard deviations. Participation in the course's existing learning system fell by 0.90 standard deviations. The losses were larger for students who were first in their family to attend university.

The authors suggest the tutor may have pushed out other learning activities. We have read the summary only, and it does not describe how the tutor was designed. It is a working paper, not peer reviewed as of 13 November 2026.

What these studies did not show

• Not that any AI tutor works. The Harvard tutor was built with care for one course. A product with "tutor" in its name is not the same thing.
• Not long-term learning. The Harvard trial tested students right after each lesson, on two topics, at one university.
• Not a replacement for teaching. The Harvard authors say the tutor should add to teaching, not replace it.

What to do with this

The tutors that helped had one thing in common: the student did each step. You can ask any chatbot to work that way. Tell it your attempt first, and ask it to guide you with questions. Here is the wording.

Questions