The trick: Lab Not Field
Khan Academy sells Khanmigo as an always-available tutor.
A two-year trial found students rarely asked it anything.
Khan Academy markets Khanmigo as an always-available tutor that never gives you the answer.
Before you read on. Your call?
TRUE, BUT
17%
A two-year randomized trial in 18 Tennessee middle schools, by Oreopoulos and Low, found assignment raised maths by 1.3 national percentile ranks per term, about 0.06 to 0.08 standard deviations a year.
The twist
The authors say these gains resemble Khan Academy practice without AI. 96 percent tried Khanmigo, but the median student messaged it in only 17 percent of sessions with a mistake.
There’s more to this story.
Membership opens the full investigation, the strongest counterargument and what to do with what you’ve learned.
Get tomorrow’s check free by email.
30 days free. Payment card required. One introductory trial per customer.
Your trial ends on 30 days after you start. Unless you cancel before then in Account → Manage membership, we charge A$89 for the first year. It renews automatically at A$89/yr until cancelled.
You can ask for a full refund within 14 days after any annual payment, renewals included, with no reason needed, by emailing [email protected]. This voluntary refund does not limit your rights under the Australian Consumer Law.
By starting your trial, you agree to the Terms.
BS Killer is published by Inferno Tech Pty Ltd, ABN 27 647 413 474.
Couldn't check your access. That's on us.
The trick has a name
We call it Lab Not Field: it works in the test conditions, not the deployed ones. You'll see it again. Learn to spot it →
Receipts
- Context edworkingpapers.com:
Assignment raises math achievement by 1.3 national percentile ranks per term, or about 0.06 to 0.08 standard deviations over a school year
- Refutes edworkingpapers.com:
These gains resemble those from Khan Academy practice without AI assistance.
- Context edworkingpapers.com:
96 percent of students tried Khanmigo at least once
- Refutes edworkingpapers.com:
and in only 17 percent of the exercise sessions in which they made a mistake.
- Refutes edworkingpapers.com:
Messages that students did send were mostly bare answers or clicks on suggested prompts.
- Context edworkingpapers.com:
The binding constraint appears to be engagement
- Context edworkingpapers.com:
a two-year cluster randomized trial in 18 Tennessee middle schools
- Supports edworkingpapers.com:
the implied effect of a full year of active participation reaches 0.14 standard deviations.
- Context edworkingpapers.com:
Generative AI has been promoted as the technology that could transform education by providing every student a personal tutor.
- Supports khanmigo.ai:
Khanmigo is your always-available tutor on just about every topic.
- Supports khanmigo.ai:
Unlike ChatGPT, Khanmigo never gives you the answer.
- Context news.ycombinator.com:
AI tutoring with Khanmigo in a two-year school experiment
Open the Receipts Pack → What each source proves, every figure traced, and what would change our verdict.
