Better essays, no more learning: what a study of 117 students found
Students who revised an essay with ChatGPT improved their essay scores the most. They did not learn more: knowledge gain and the ability to use that knowledge elsewhere were no different from the other groups. The study had 117 university students and is peer reviewed. Two more peer-reviewed studies, comparing AI with web search, found the same split between an easier task and shallower understanding.
The three studies in one table
| Study | People | Compared | Status |
|---|---|---|---|
| Fan and others, 2025 | 117 university students | ChatGPT, a human expert, a checklist tool, or no support | Peer reviewed. British Journal of Educational Technology |
| Stadler, Bannert and Sailer, 2024 | 91 university students | ChatGPT or Google search | Peer reviewed. Computers in Human Behavior |
| Melumad and Yun, 2025 | 10,462 across 7 experiments | AI summaries or web links | Peer reviewed. PNAS Nexus |
The essay study
Fan and colleagues put 117 students into four groups at random. Everyone read some material, wrote an essay and then revised it. The groups differed in the help they had for revising: ChatGPT, a human expert over chat, a writing checklist tool, or nothing.
- Essay scores. The ChatGPT group improved the most.
- Knowledge gain. No significant difference between groups.
- Knowledge transfer. No significant difference either. Transfer means using what you learned on a new problem.
- Motivation. No difference in how much students wanted to do the task for its own sake.
The groups also went about the work differently. The authors warn of dependence on the tool and of what they call "metacognitive laziness": letting the AI do the planning and checking that learners normally do for themselves.
Why a better essay is not the same as learning
An essay score measures the page. It does not measure the person. If the AI improved the page, the score goes up whether or not the student now understands more. That is the same gap seen in a field trial with about 1,000 students, where practice scores rose with a plain chatbot and exam scores fell.
ChatGPT against web search
The German experiment. Stadler and colleagues gave 91 students a science question about sunscreen and asked them to research it and justify a recommendation. Each student was assigned at random to ChatGPT or to Google. The ChatGPT group reported less mental effort on all three measures. Their written reasons scored lower: 1.20 against 1.87 on a scale where the best score reached was 4. The spread of recommendations was the same in both groups.
The seven experiments. Melumad and Yun ran seven experiments with 10,462 people in total. People learned about an everyday topic either from an AI summary or from web links, then wrote advice for someone else. Those who learned from the AI summary felt less invested in their advice. Their advice was sparser and less original, and readers were less likely to take it. This held even when the facts shown were identical, and even when the AI summary came with web links.
What these studies did not show
- Not that ChatGPT made students learn less. In the essay study, learning was the same across groups, not lower. The finding is that a better essay came with no extra learning.
- Not a lasting effect. The essay study was a single lab session with one task. The German task was short. None of the three followed people over time.
- Not every subject. The search experiments used everyday topics such as gardening, health and scams. The German study used one science topic and an older AI model.
- Not that search is always better. These studies measured depth of understanding after a short task. They did not measure speed or convenience, where AI summaries may win.
A fair one-line summary. "In a peer-reviewed experiment with 117 students, ChatGPT gave the biggest improvement in essay scores but no more knowledge gain or transfer."
What to do with this
Decide what you want before you start. If you only need a finished page, AI help is efficient. If you need to know the subject afterwards, for an exam or a job, do the first draft and the first round of checking yourself. Then ask the AI what you missed.
Questions
Does ChatGPT improve essay grades?
In this experiment, the ChatGPT group had the largest improvement in essay scores. Their knowledge of the subject did not improve more than anyone else's.
What is metacognitive laziness?
It is the authors' phrase for leaving the planning, monitoring and checking of your own work to the AI. The study raises it as a risk. It did not measure it as a lasting habit.
Is Google better than ChatGPT for learning?
In two sets of experiments, people who searched and read for themselves ended up with deeper understanding than people who read an AI answer. The tasks were short and the topics narrow.
Why would the same facts lead to shallower knowledge?
The seven experiments found the effect even with identical facts. A summary hands you the result of the sorting and linking. Searching makes you do some of it yourself.