AI tutors and learning outcomes
AI tutoring systems in schools mostly helped, but less so against simpler non-AI software
Researchers pulled together school studies, published after 2009, that tested intelligent tutoring systems: software that adapts practice and feedback to each student. Most compared the system with ordinary teaching and found it helped. But when the comparison was a simpler, non-adaptive tutoring program, the advantage mostly disappeared, and many studies were very short.
| Paper | A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education |
| Authors | Angélique Létourneau, Marion Deslandes Martineau, Patrick Charland, John Alexander Karran, Jared Boasen, Pierre-Majorique Léger |
| Published | 14 May 2025 · npj Science of Learning 10:29 |
| Study type | Systematic review (PRISMA), no meta-analysis · Peer-reviewed |
| What we read | Full text read. We read the open-access full text. Figures are from the published article; the abstract indexed in some databases gives older totals (20 studies, 2,853 students). |
| Summarized |
What they did
A systematic review following PRISMA guidelines. The team searched ERIC and Scopus for experiments in formal school settings that measured an intelligent tutoring system's effect on learning or performance, screened them with two reviewers, and grouped the results by the kind of comparison group. They chose not to pool effect sizes because the designs varied too much.
Who took part
28 studies in 26 articles, with 4,597 primary and secondary students in total, mostly in middle and high school STEM classes, and mostly in the United States and Asia.
What they found
- The studies were mainly quasi-experimental: an experimental group used the system and a control group learned the same material another way, with tests before and after.
- Against teacher-led instruction, seven of eight articles reported a positive effect in favour of the tutoring system, of medium to large size.
- Against non-intelligent tutoring software, one of four studies found a learning advantage for the AI system; the other three found no significant difference.
- Many interventions were very short, which the authors warn may inflate effects through novelty.
- In one long study the benefit only appeared in the second year, at about +0.20 standard deviations in high school, with a non-significant trend in middle school.
- None of the studies addressed ethical issues.
Effect size: No pooled effect size; individual studies are described in the text (for example +0.20 SD in one two-year algebra study).
Limitations
- These are mostly pre-ChatGPT adaptive tutoring systems, not today's chat-based AI tutors.
- Mostly quasi-experiments rather than randomized trials, many of them short and small.
- Comparisons with "traditional teaching" are hard to combine because that control looked different in each study.
- No meta-analysis, so there is no single overall effect size.
What this means for learners and parents
- Adaptive practice and immediate feedback seem to help compared with ordinary lessons, but much of that may come from structured practice rather than "AI" as such.
- Short trials can flatter new tools. Look for evidence over months, not one or two sessions.
- The authors note these systems are reported to work best alongside teacher guidance, not as a replacement for it.
Put it into practice (free tools)
Try it: Ask Poppy is voice-first homework help for kids, built first for ages 7 to 10 (teens and adults use it too), on iPhone, iPad and Mac. It asks guiding questions instead of handing over the answer, so children learn how to find it. Free with 30 messages a day, then $3.99/week or $9.99/month. (We make it.)
Citation
Angélique Létourneau, Marion Deslandes Martineau, Patrick Charland, John Alexander Karran, Jared Boasen, Pierre-Majorique Léger (2025). A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education. npj Science of Learning 10:29. https://doi.org/10.1038/s41539-025-00320-7
Related summaries
- Unrestricted ChatGPT help raised practice scores but lowered exam scores · Hamsa Bastani et al., 2025 · Preregistered randomized controlled trial
- A carefully designed AI tutor beat an active-learning physics class on a short-term test · Greg Kestin et al., 2025 · Randomized controlled trial with a crossover design
- Supervised AI tutoring matched human tutors in UK maths classrooms · LearnLM Team (Google) et al., 2025 · Exploratory randomized controlled trial
- An AI coach for human tutors raised topic mastery, most for weaker tutors · Rose Wang et al., 2024 · Preregistered randomized controlled trial