AI tutors and learning outcomes Socratic and dialogic tutoring
An AI coach for human tutors raised topic mastery, most for weaker tutors
In short: Giving human tutors an AI tool that suggests expert-style next moves made their students 4 percentage points more likely to master a topic, and 9 points more likely for students of lower-rated tutors. Tutors with the tool asked more guiding questions and gave away answers less often. End-of-year test scores did not improve significantly.
| Paper | Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise |
| Authors | Rose Wang, Ana T. Ribeiro, Carly D. Robinson, Susanna Loeb, Dorottya Demszky |
| Published | October 2024 · arXiv preprint 2410.03017 (Stanford) |
| Study type | Preregistered randomized controlled trial · Working paper / preprint |
| What we read | Full text read. We read the full preprint. |
| Summarized |
What they did
Tutors in a chat-based tutoring programme were randomly assigned to get Tutor CoPilot or not. When a student made a mistake, the tool suggested responses based on a model of how expert teachers think (for example, ask a guiding question, prompt the student to explain). Researchers analysed hundreds of thousands of chat messages with classifiers to see which strategies tutors used.
Who took part
900 tutors and students in grades 3–6 from a large school district in the southern United States, most of them performing below grade level; 79% identified as Hispanic and 67% were classed as economically disadvantaged. The main analysis covered 1,013 students who attended at least one session.
What they found
- Students of tutors with Tutor CoPilot were 4 percentage points more likely to pass the end-of-lesson mastery check.
- For students of lower-rated tutors the gain was 9 percentage points.
- Tutors with the tool used more high-quality strategies, such as guiding questions, and were less likely to give away the answer.
- No statistically significant improvement in end-of-year maths test scores.
- The tool cost about $20 per tutor per year.
Effect size: +4 percentage points in topic mastery (p<0.01); +9 points for lower-rated tutors.
Limitations
- Outcomes were short-term lesson checks; no significant effect on end-of-year tests.
- One tutoring provider and one district.
- Some suggestions were not grade-level appropriate, according to tutors.
- A preprint at the time of summary.
What this means for learners and parents
- Holding back the answer and asking a good question is a learnable skill, for tutors and for parents helping with homework.
- AI may be most useful as a coach for the adult, not only as a tutor for the child.
- Short-term mastery gains do not automatically show up in end-of-year exams.
Put it into practice (free tools)
Citation
Rose Wang, Ana T. Ribeiro, Carly D. Robinson, Susanna Loeb, Dorottya Demszky (2024). Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise. arXiv preprint 2410.03017 (Stanford). https://doi.org/10.48550/arXiv.2410.03017
Related summaries
- Unrestricted ChatGPT help raised practice scores but lowered exam scores · Hamsa Bastani et al., 2025 · Preregistered randomized controlled trial
- A carefully designed AI tutor beat an active-learning physics class on a short-term test · Greg Kestin et al., 2025 · Randomized controlled trial with a crossover design
- Supervised AI tutoring matched human tutors in UK maths classrooms · LearnLM Team (Google) et al., 2025 · Exploratory randomized controlled trial
- Six weeks of teacher-guided AI tutoring improved English results in Nigeria · Martin De Simone et al., 2025 · Randomized controlled trial