--- title: "AI Tutor vs Active Learning Class: Harvard RCT (2025)" description: "Summary of a Harvard physics RCT (N=194): a custom, scaffolded AI tutor produced larger short-term learning gains than an in-class active-learning lesson." canonical: https://research.askpoppy.ai/papers/kestin-2025-ai-tutoring-outperforms-active-learning/ last_updated: 2026-10-21 --- [AI tutors and learning outcomes](/topics/ai-tutors/) [Motivation](/topics/motivation/) # A carefully designed AI tutor beat an active-learning physics class on a short-term test **In short:** In a Harvard introductory physics course, students learned more on a short post-test when a lesson was delivered by a custom-built AI tutor at home than when the same material was taught as an in-class active-learning lesson. The tutor was not a plain chatbot: instructors wrote question-specific prompts and solutions and structured each step. | | | | --- | --- | | Paper | [AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting](https://doi.org/10.1038/s41598-025-97652-6) | | Authors | Greg Kestin, Kelly Miller, Anna Klales, Timothy Milbourne, Gregorio Ponti | | Published | 3 June 2025 · Scientific Reports 15:17458 | | Study type | Randomized controlled trial with a crossover design · Peer-reviewed | | What we read | Full text read. We read the open-access full text. | | Summarized | 21 October 2026 | ## What they did Each student had one lesson (surface tension or fluid flow) with the AI tutor at home and the other as an in-class active-learning lesson, with the order swapped between two groups. Both versions followed the same pedagogy. The AI tutor used GPT-4 with instructor-written, question-specific prompts and provided solutions, so it did not rely on the model to work out answers itself. Students took a pre-test and a post-test for each lesson and answered survey questions on engagement and motivation. ## Who took part 194 undergraduates in Physical Sciences 2, an introductory physics course for life-science students at Harvard University, autumn 2023. ## What they found - Median learning gains in the AI-tutor condition were more than double those in the active-learning class. - Mean post-test score: 4.5 with the AI tutor versus 3.5 in class. - Students in the AI condition spent a median of 49 minutes on the lesson, against a 60-minute class. - Students reported feeling more engaged and more motivated with the AI tutor. **Effect size:** Effect size about 0.63 standard deviations in the main regression; 0.73 to 1.3 SD in analyses designed to account for the ceiling on post-test scores. ## Limitations - One course at one highly selective university, two lessons. - The post-test came straight after the lesson, so long-term retention was not measured. - The AI tutor was carefully engineered by experienced instructors; results may not carry over to general-purpose chatbots. - Engagement and motivation were self-reported. ## What this means for learners and parents - A well-structured AI tutor can teach a first pass of new material efficiently, at least for motivated university students. - The design did a lot of the work: step-by-step scaffolding and prepared solutions, not open-ended chat. - For lasting learning, it still makes sense to come back to the material later and test yourself without help. ## Put it into practice (free tools) - [Compare AI tutors: answers or hints, voice, prices](https://learn.askpoppy.ai/compare/) - [Come back on day two: the 7-day challenge](https://learn.askpoppy.ai/learn-out-loud/) **Try it:** [Ask Poppy](https://apps.apple.com/app/apple-store/id6757080904?ct=research-kestin-2025-ai-tutoring-outperf&mt=8) is an AI tutor for iPhone, iPad and Mac that asks questions and gives one hint at a time instead of handing over the answer. Free with 30 messages a day, then $3.99/week or $9.99/month. (We make it.) ## Citation Greg Kestin, Kelly Miller, Anna Klales, Timothy Milbourne, Gregorio Ponti (2025). AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting. *Scientific Reports 15:17458*. ## Related summaries - [Unrestricted ChatGPT help raised practice scores but lowered exam scores](/papers/bastani-2025-generative-ai-without-guardrails-harms-learning/) · Hamsa Bastani et al., 2025 · Preregistered randomized controlled trial - [Supervised AI tutoring matched human tutors in UK maths classrooms](/papers/learnlm-eedi-2025-ai-tutoring-uk-classrooms-rct/) · LearnLM Team (Google) et al., 2025 · Exploratory randomized controlled trial - [An AI coach for human tutors raised topic mastery, most for weaker tutors](/papers/wang-2024-tutor-copilot-human-ai-tutoring/) · Rose Wang et al., 2024 · Preregistered randomized controlled trial - [Six weeks of teacher-guided AI tutoring improved English results in Nigeria](/papers/de-simone-2025-chalkboards-to-chatbots-nigeria/) · Martin De Simone et al., 2025 · Randomized controlled trial Summary by the Ask Poppy team, AI-assisted and checked against the paper. Summarized 21 October 2026. Spotted an error? [simon@askpoppy.ai](mailto:simon@askpoppy.ai) · [Editorial policy](/about/) ```json {"@context": "https://schema.org", "@type": "Article", "headline": "AI Tutor vs Active Learning Class: Harvard RCT (2025)", "alternativeHeadline": "A carefully designed AI tutor beat an active-learning physics class on a short-term test", "description": "Summary of a Harvard physics RCT (N=194): a custom, scaffolded AI tutor produced larger short-term learning gains than an in-class active-learning lesson.", "url": "https://research.askpoppy.ai/papers/kestin-2025-ai-tutoring-outperforms-active-learning/", "mainEntityOfPage": "https://research.askpoppy.ai/papers/kestin-2025-ai-tutoring-outperforms-active-learning/", "author": {"@type": "Organization", "name": "Ask Poppy team", "url": "https://research.askpoppy.ai/about/", "parentOrganization": {"@id": "https://askpoppy.ai/#org"}}, "publisher": {"@id": "https://askpoppy.ai/#org"}, "datePublished": "2026-10-21", "dateModified": "2026-10-21", "inLanguage": "en", "about": [{"@id": "https://doi.org/10.1038/s41598-025-97652-6"}, {"@type": "Thing", "name": "AI tutors and learning outcomes"}, {"@type": "Thing", "name": "Motivation"}], "citation": {"@id": "https://doi.org/10.1038/s41598-025-97652-6"}, "isBasedOn": "https://doi.org/10.1038/s41598-025-97652-6", "image": "https://research.askpoppy.ai/og/kestin-2025-ai-tutoring-outperforms-active-learning.png", "isAccessibleForFree": true} {"@context": "https://schema.org", "@type": "ScholarlyArticle", "@id": "https://doi.org/10.1038/s41598-025-97652-6", "headline": "AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authe", "name": "AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting", "author": [{"@type": "Person", "name": "Greg Kestin"}, {"@type": "Person", "name": "Kelly Miller"}, {"@type": "Person", "name": "Anna Klales"}, {"@type": "Person", "name": "Timothy Milbourne"}, {"@type": "Person", "name": "Gregorio Ponti"}], "datePublished": "2025-06-03", "isPartOf": {"@type": "Periodical", "name": "Scientific Reports"}, "identifier": {"@type": "PropertyValue", "propertyID": "DOI", "value": "10.1038/s41598-025-97652-6"}, "sameAs": "https://doi.org/10.1038/s41598-025-97652-6", "url": "https://doi.org/10.1038/s41598-025-97652-6"} {"@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [{"@type": "ListItem", "position": 1, "name": "Home", "item": "https://research.askpoppy.ai/"}, {"@type": "ListItem", "position": 2, "name": "Summaries", "item": "https://research.askpoppy.ai/papers/"}, {"@type": "ListItem", "position": 3, "name": "AI Tutor vs Active Learning Class", "item": "https://research.askpoppy.ai/papers/kestin-2025-ai-tutoring-outperforms-active-learning/"}]} {"@context": "https://schema.org", "@type": "Organization", "@id": "https://askpoppy.ai/#org", "name": "Ask Poppy", "legalName": "Poppy Labs AS", "url": "https://askpoppy.ai/", "logo": "https://askpoppy.ai/poppy-mark.png", "email": "hello@askpoppy.ai", "foundingLocation": "Oslo, Norway", "address": {"@type": "PostalAddress", "addressLocality": "Oslo", "addressCountry": "NO"}, "founder": {"@type": "Person", "name": "Simon Dreyer"}} ```