Two studies posted this year, one in July and one in September, come at the same question from opposite ends: when should an AI tutor help a student, and how much should it say? The first found that AI tutors step in too early and give away too much. The second found that when a tutor is built to hold back, students rate it lower, and some of them open a different chatbot that will hand over more.
Read together, they describe a decision teachers make by feel many times a class period, which is when to leave a student alone with a hard problem and when to step in. Neither kind of tutor makes that decision well, and I think the better place to make it is the assignment.
The tutor that steps in early
Verona Teo and three colleagues built a test they call Int-Bench, published under the title “AI Assistants Overassist.” A language model plays the student and works through a math problem, a piece of broken code, or a brain teaser, and its reasoning appears on screen a little at a time. A second model plays the teacher, watching the work come in and deciding whether to step in, when, and what to say.
Three of the models tested (GPT-5.2, GPT-OSS-120B, and DeepSeek-V3.2) stepped in on more than 90% of problems, and two of them were stepping in before the student had produced 100 characters of reasoning, about one sentence. They also stepped in when the student was on track. GPT-5.2 intervened on 98% of the questions the student got right when left alone, and GPT-OSS-120B intervened on all of them. The researchers also tried rewording the teacher’s instructions, including telling the model to step in only when necessary, and the results came out about the same.
The study also had 50 adults, recruited through an online research platform, play the teacher on the same brain teasers while watching the same simulated student. The people stepped in less often and later, and when they did, they tended to nudge. The models gave away the full solution in 14.2% of their interventions, against 5.4% for the people, and offered a near-complete walk-through about 48% of the time, against about 22%. Teaching experience made no measurable difference: participants who had taught waited about as long as those who hadn’t.
The help also didn’t carry over. When the simulated students moved on to new versions of the problems, having been helped on the originals made no consistent difference to their accuracy. The authors conclude that current assistants “often optimize for short-term success” at the cost of the reasoning that builds learning over time. The student in this study is a model, not a teenager, so I wouldn’t lean too hard on the result about new problems. The timing result doesn’t depend on a model standing in for the student, since it measures what the tutor does.
The tutor students route around
The obvious fix is a tutor that asks questions instead of handing over answers. Madeleine Eastwood and five colleagues tested that idea in “Guardrails or Roadblocks?”, a randomized trial with 132 students in an introductory Python course at a large public university in the United States during the fall 2025 semester.
Each student was randomly given one of four AI teaching assistants for a code-writing task and two debugging tasks. Two of the assistants used a Socratic style and two gave direct instruction. Within each pair, one could see the problem and the student’s code, and the other could see only the chat.
Students rated the Socratic assistant that could see their code lowest of the four for helping them finish the task: an average of 3.53 out of 5, against 4.12 to 4.27 for the other three, a gap that held up statistically. The same group turned to an outside chatbot most often (7 of 30 students, against 9% to 15% in the other groups) and had the lowest share of students who could fully explain their solution afterward. Those last two gaps were small enough to be chance, and the authors say so. The Socratic assistant that couldn’t see the code was rated about as well as the direct ones, so asking questions wasn’t the whole problem.
When students explained why they had gone elsewhere, the most common reason was that the assistant didn’t give them enough to work with. One student called the Socratic version “super rigid.” Another described “circle reasoning,” where the assistant kept answering questions with more questions. These were university students in one programming course working through a single assignment, so it’s a narrow window, and I’d want to see the same test run with high school students.
Attempt first
Chapter Twelve of my book, The AI Doesn’t Know Your Students, makes the case for a sequence. The student attempts the task first, without AI, and the attempt does its work by exposing what the student doesn’t understand yet. Then AI comes in as part of the instruction, explaining what went wrong or modeling a stronger version, and the student revises. The idea draws on Manu Kapur’s research on productive failure, which has found that students who struggle with a problem before instruction tend to understand it better afterward than students who get the instruction first.
In a history class, the attempt might be a student’s first pass at sourcing a document: who wrote it, for whom, and why at that moment. A rough answer to those three questions, even a wrong one, gives the student something specific to ask about. A student who opens the chatbot first will likely get a finished sourcing paragraph with no gap to ask into.
Set beside that chapter, the two studies show what happens when a tool tries to run the sequence on its own. The first tutor steps in during the attempt, before the attempt has exposed anything. The second protects the attempt, and the students who used it rated it lowest and said it didn’t give them enough to work with. Each tutor follows one rule from the first minute of the task to the last.
The Int-Bench study also tested a condition where the model teacher read the student’s entire reasoning before deciding whether to step in. With the whole picture, the models intervened on 54% of problems on average instead of 90%, stepped in later, and on the brain teasers stopped intervening on the problems the student was going to solve anyway. Given a full view of how the student was thinking, the tutor behaved like a teacher who has already seen the student’s first attempt.
So I’d put the timing decision in the assignment. A tutor that can’t see whether the student has tried yet will jump in or hold back by a fixed rule, so the assignment should require the attempt and put it in front of the teacher before the tool. That also changes what I’d ask at a vendor demo. The first thing I’d want to know is whether I can decide when students open the tool, and whether it can see their first attempt when they do.
Takeaway for Teachers
Pick one assignment this week where students would normally use an AI tool, and split it into two stages. In the first, students work for ten to fifteen minutes with no tool, on paper if you can manage it, and finish by writing one sentence about where they got stuck. Collect those sentences, or walk the room and read them before moving on. In the second stage, the tool opens, and students start the chat by pasting in their stuck-point sentence instead of the assignment prompt. You get a quick read on where the class is struggling, and the AI enters the work at the point where it can help.
If you’re thinking through questions like this one, about when a tool should help and when a student needs to try the problem alone first, my book goes deeper. The AI Doesn’t Know Your Students is available on Amazon and at shouldiuse.ai/book.
David Jacobson is a high school history teacher. He writes about AI, education, and the messy intersection of the two at shouldiuse.ai.
