In a randomized trial of about 1,000 high school students, those who practiced math with unrestricted ChatGPT scored 48 percent higher during practice than classmates without AI, then 17 percent lower on an exam taken without it. When AI does the beginner work, you can lose the learning the hard part produces, and you will not feel it happening.
The payoff from difficulty shows up later
Robert Bjork named the category in 1994: desirable difficulties, meaning conditions that make learning slower and harder while it is happening and better afterward. The list includes spacing practice out, mixing problem types rather than blocking them, reducing how often you get feedback, and testing yourself rather than rereading.
The cleanest demonstration is Roediger and Karpicke, 2006. Students read prose passages, then either restudied them repeatedly or tested themselves, and were examined later.
Five minutes after studying, repeated restudy won: 83 percent recall against 71 percent for the tested group. One week later the ranking reversed completely. The tested group recalled 61 percent. The restudy group recalled 40 percent. Over that week the restudy group forgot 52 percent of what they had known. The tested group forgot 14 percent.
The restudy group had also read the passage 14.2 times during acquisition. The tested group read it 3.4 times. They did roughly a quarter of the reading and retained half again as much a week later.
That reversal is the entire mechanism in one experiment. The easier method produces better performance in the moment and worse retention later. Measure at the wrong time and you will reach the wrong conclusion.
The pattern holds at scale. A meta-analysis of 159 comparisons found that testing yourself beats restudying by half a standard deviation on average, and by more when the final test comes at least a day later (0.69 against 0.41). Generating an answer yourself instead of reading it shows the same shape across 445 effect sizes and 17,711 subjects: 0.64 at delays beyond a day, 0.41 immediately.
Both effects grow when you wait to measure.
The same pattern, with AI as the difficulty remover
The trial in the opening tested this directly. It ran in about 50 classrooms and was published in PNAS in 2025. Students practiced math in one of three conditions: unrestricted GPT access, a guardrailed AI tutor that gave hints but withheld answers, or neither.
During practice, the unrestricted GPT group outperformed the control group by 48 percent. Then they sat an exam with no AI available, and scored 17 percent worse than students who had never used it.
Practice performance up. Retained capability down. The same reversal as the restudy experiment, produced by a different difficulty remover.
The guardrailed tutor produced a 127 percent improvement during practice and no significant difference from control on the unassisted exam. The harm disappeared. The tool was not the variable. Whether the tool preserved the effortful step was the variable.
It shows up in people who already have the skill
In a 2025 study in The Lancet Gastroenterology & Hepatology, researchers examined endoscopists at four centers who had each performed more than 2,000 procedures, before and after AI assisted colonoscopy became routine. On colonoscopies performed without AI, the same doctors’ adenoma detection rate fell from 28.4 percent to 22.4 percent after the AI period. Their unassisted performance got worse. The study is observational rather than randomized, so it establishes a pattern, not a cause.
In a 2014 simulator study of 16 airline pilots, 7 missed an assigned heading or approach point on a manual navigation task, and only one completed the sequence without error. Their hand-eye instrument scanning was largely intact. What had degraded was the cognitive work: tracking position, anticipating the next step, holding the plan.
The manual skill survived. The judgment did not. That is the specific thing at risk.
What happens in the brain when you offload
Psychologists call handing mental work to a tool cognitive offloading. The trade shows up consistently in controlled experiments: offloading raises performance on the task in front of you and lowers your memory of what you offloaded afterward. Telling people a memory test was coming offset part of that loss.
Brain imaging shows the regions that go quiet. In a 2017 fMRI study at University College London, 24 people navigated a simulation of London’s Soho. When they chose their own route, the hippocampus, which builds mental maps, and the prefrontal cortex, which plans ahead, spiked each time they entered a new street, tracking how many options that street opened up. When they followed turn-by-turn directions, that extra activity did not appear.
Habitual use appears to compound. Among 50 drivers, more lifetime GPS use went with worse spatial memory when navigating unaided. Of the 13 retested three years later, those who had leaned on GPS more showed a steeper decline in hippocampus-dependent spatial memory. The follow-up sample is small, so treat it as suggestive.
Retrieval runs the other way. fMRI studies of the testing effect find that pulling an answer out of your own memory strengthens hippocampal activity at a delayed test and sharpens how the medial prefrontal cortex represents what you learned. Generating the answer yourself is one of the steps that builds memory, and it is the step an answer-giving AI removes.
Where this argument breaks
Struggle does not help everyone. For novices, studying worked solutions beats solving unaided (Sweller and Cooper, 1985: 43.6 seconds against 78.1 per problem, 0.18 errors against 1.64), and that advantage fades and then reverses as expertise grows. A 2025 review in the Quarterly Journal of Experimental Psychology finds difficulty helps on simpler material and overwhelms learners on complex material, and difficulty unrelated to the skill, such as unscrambling anagrams, buys nothing. The widely shared MIT Media Lab “brain rot” EEG study is a preprint with 18 participants in its key session, its lead author rejects that framing, and it is not used here.
What the evidence actually supports
The strongest evidence for using AI well is a Harvard physics trial. Across two weeks, 194 students alternated between in-class active learning taught by experienced instructors and a purpose-built AI tutor that guided them step by step without handing over answers. Median post-test scores were 4.5 after the tutor against 3.5 after class, more than double the learning gains.
Read alongside the PNAS trial, the finding is consistent. Unrestricted AI during practice damaged unassisted performance. Structured AI that withheld the answer did not, and in the Harvard case beat the human-taught alternative.
So the operative question is whether, on the specific thing you are trying to get good at, the tool is removing the step where the learning happens.
Three distinctions the research supports. Difficulty related to the skill helps, and unrelated difficulty does not. The benefit shows up on delayed measurement, so judging by how practice feels can mislead you. And the right amount of support falls as competence rises, so the setting that helped you last year may be the wrong one now.
The uncomfortable implication is that you cannot tell from the inside. In the PNAS trial, students using unrestricted AI were performing dramatically better during practice, right up until the exam. In a separate randomized trial of experienced software developers, participants who had been measurably slowed down by AI still believed they had been sped up.
Whatever you are currently outsourcing, the feeling of fluency is not evidence. That belongs with the other questions AI cannot answer for you, and it sits underneath the argument about who AI actually displaces.
So test it. Pick the skill you most want to own, do it once a week without AI, and compare the result to the week before. If your unassisted work is slipping, switch the tool from giving answers to giving hints for that skill.
Sources: Bjork, “Memory and metamemory considerations in the training of human beings,” 1994. Roediger & Karpicke, Psychological Science 17(3), 2006. Rowland, Psychological Bulletin 140(6), 2014. Bertsch, Pesta, Wiscott & McDaniel, Memory & Cognition 35(2), 2007. Bastani et al., PNAS 122(26), 2025. Budzyn et al., The Lancet Gastroenterology & Hepatology, 2025. Casner, Geven, Recker & Schooler, Human Factors 56(8), 2014. Risko & Gilbert, “Cognitive Offloading,” Trends in Cognitive Sciences, 2016. Grinschgl, Papenmeier & Meyerhoff, “Consequences of cognitive offloading: Boosting performance but diminishing memory,” Quarterly Journal of Experimental Psychology, 2021. Javadi et al., “Hippocampal and prefrontal processing of network topology to simulate the future,” Nature Communications, 2017. Dahmani & Bohbot, “Habitual use of GPS negatively impacts spatial memory during self-guided navigation,” Scientific Reports, 2020. Wiklund-Hörnqvist et al., “Retrieval practice facilitates learning by strengthening processing in both the anterior and posterior hippocampus,” Brain and Behavior, 2021. Ye et al., “Retrieval practice facilitates memory updating by enhancing and differentiating medial prefrontal cortex representations,” eLife, 2020. Sweller & Cooper, Cognition and Instruction 2(1), 1985. Kalyuga, Ayres, Chandler & Sweller, Educational Psychologist 38(1), 2003. Pyke, Lunau & Javadi, Quarterly Journal of Experimental Psychology 78(10), 2025. Kestin et al., “AI tutoring outperforms in-class active learning,” Scientific Reports 15, 2025. Kosmyna et al., arXiv preprint 2506.08872. Becker, Rush, Barnes & Rein, METR, 2025.
