This summer, three elementary teachers in a rural school district in the eastern United States ran a three-week camp for 78 second and third graders. The curriculum paired English Language Arts lessons with ToyTalk, a chatbot platform the research team built so children could design their own talking AI toy, including its name, its personality, its tone, and the rules it follows. The teachers filled out a reflection after every camp day, met for group reflections, and sat for interviews when the camp ended. Fasika Melese and six colleagues posted the study to arXiv last month.
The paper is about the work the teachers did to make the tool usable for their students, and most of that work wasn’t written into the curriculum. The researchers sorted it into four kinds: repair, differentiation, translation, and balancing.
Four kinds of extra work
Repair was the most visible. The microphones didn’t consistently pick up students’ voices, and the problem was worst for children with speech impediments or different accents. Teachers troubleshot equipment, switched to backups, and helped students through whatever broke that day. The researchers describe this as the teacher supplying the reliability the system lacks. The fix the teachers preferred was a manual start and stop button, because the children already knew that control from the speech-to-text tools they used.
Differentiation took the most time. The lessons leaned on worksheets and independent work, which was hard on students who were still learning to read and write. Teachers split lessons across several days when they ran too long for second graders, transcribed for children who couldn’t yet write what they meant, set up peer tutoring, and read over students’ shoulders to fix their prompts. One teacher said sentence starters would help some students, but worried about making the work “too easy where they can’t learn to say or type anything.”
Translation meant turning abstract AI ideas into something a second grader could hold onto. Teachers explained hallucination through The Boy Who Cried Wolf, with the difference that the AI isn’t lying the way the boy was. When speech-to-text misheard a word, a teacher used the slip to teach homophones and to show why you check what the AI gives back. Another teacher read a student’s question about how long a caterpillar stays in its cocoon and saw that the toy had answered with general facts instead of anything from the story the class was reading, so the two of them talked about why the answer missed.
Balancing was the quietest of the four. The camp wanted children to be creative, and it also had reading goals to meet. Students first designed their toy’s purpose themselves and were later redirected to make it tell The Very Hungry Caterpillar. The teachers spent the three weeks working out how much freedom to give and how much structure the children needed to finish anything.
None of these four jobs fell to the tool, and each one depends on knowing the children. You have to know which child has a speech impediment to tell whether the microphone failed or the child stopped trying, and you have to know who can’t write yet before you decide who gets a transcriber and who gets a sentence starter. Noticing that an answer ignored the story the class was reading takes knowing which story that is.
The same gap in college computer science
A second study, posted to arXiv this month by Xi Zhao, Xinyue Jiao, and Zhen Xu, looks at a setting about as far from a summer camp as you can get, and the tools there leave the same work to the teacher. The authors used a dataset of 3,041 student answers to 50 open-ended computer science questions, each scored by human instructors and by three commercial language models. The models produced rubric scores, written comments, and suggestions for improvement, which looks thorough on the page. Their written feedback detected student misconceptions 5 to 7 percent of the time, against 15 to 31 percent for the teachers. The models’ tone also stayed positive no matter how good or weak the answer was, while the human feedback got firmer when an answer needed it.
The authors say the AI feedback works more like a coverage checklist than a diagnosis. A checklist can tell a student which parts of an answer are present, but someone has to be paying attention to that student’s thinking to say where it went wrong.
What the book says about entry points
Chapter Fourteen of The AI Doesn’t Know Your Students covers differentiation and EAL support, and it starts from an idea that has held up for me over years of teaching students who write in their second or third language: most apparent low effort is a missing entry point. A blank page, a language gap, or a skill nobody taught can look like a student who isn’t trying. AI is good at producing what fixes it, whether that’s a sentence stem, a leveled reading, or a graphic organizer.
The camp study fills in the other half of that idea. The ToyTalk curriculum came with worksheets and a plan for every day, and the teachers still spent each lesson deciding which child needed which support. A tool can generate a sentence starter in a second. It can’t tell you which child understands the story and can’t yet spell the words to say so, or which one has stopped talking because the microphone hasn’t heard them all morning.
The chapter also argues that every scaffold depends on something the scaffold can’t create, which is a student who trusts the teacher enough to try. That trust comes from a relationship that has to exist before the scaffold does. A child whose voice the toy keeps missing needs an adult who notices and steps in before the frustration turns into giving up.
What the study can't show
Three teachers, one summer camp, and a platform the researchers built themselves make a small study. It gives us a description, in the teachers’ own words, of the work that usually goes unrecorded when a school pilots a new tool, though it can’t say how often that work happens across schools.
The authors’ recommendations point at planning and time. They want teachers given ready analogies, worked examples, and prompts before they start, and they want daily reflection built into how a tool is rolled out. When a school adopts an AI product, the plan usually covers licenses, accounts, and a training session. The repair, differentiation, translation, and balancing in this paper happen during class, after the training is over, and they rarely appear in the plan.
Takeaway for Teachers
Before you run an AI tool with a class this week, spend five minutes with the four categories from this study. Write down what is most likely to break (repair), which students will need a different pace or a different way in (differentiation), which idea in the lesson will need an analogy your students already know (translation), and where the tool’s open-endedness might pull against what you need students to learn (balancing). After the lesson, add two lines about what you had to fix. The ToyTalk teachers wrote a reflection after every camp day, and the research team used that feedback to revise the platform. If you’re also using AI to draft feedback on student work, read each comment for one thing before it goes back: does it name what this student misunderstood? If it doesn’t, add that sentence yourself.
If you’re thinking through questions like this one, about what an AI tool leaves for the teacher to do, my book goes deeper. The AI Doesn’t Know Your Students is available on Amazon and at shouldiuse.ai/book.
David Jacobson is a high school history teacher. He writes about AI, education, and the messy intersection of the two at shouldiuse.ai.
