AI in education

AI in Education: Documented Failures, Real Costs, and the Rules

Published: Updated: by

Research current to 12 October 2026. Each claim links to the source it came from. Where I could only reach a secondary source, the text says so.

The short version

  • The best-documented failures are not “the chatbot got a fact wrong.” They fall into three patterns: a procurement bet on a vendor that collapsed (Los Angeles), AI-writing detectors treated as proof of cheating (Australian Catholic University, Adelphi University), and general-purpose chatbots that lifted practice scores while lowering exam scores (a peer-reviewed field experiment in PNAS).
  • The costs are real but badly counted. No one has published an aggregate bill for AI failures in education. What exists is case-level: about $3 million paid to a vendor that went under, a six-figure legal bill for one family, and months-long misconduct holds on student transcripts.
  • The same research also contains strong positive results, but only when the tool is built to make students think. The OECD’s 2026 synthesis puts it plainly: AI can raise task performance without raising learning.
  • The rules are thin and still moving. The EU classifies several education uses as high-risk, but those obligations now start on 2 December 2027. The US has no federal mandate, only grant incentives plus guidance from 37 states that is mostly non-binding. In a spring 2026 Gallup survey, 18% of teachers said they had received formal guidance.

How to read the evidence

I sort sources into three tiers. Tier A: court opinions, official gazettes, peer-reviewed papers and intergovernmental reports. Tier B: major news outlets and law-firm analyses. Tier C: trackers, aggregators and blogs, used only for counts and flagged when used.

Backdrop: how widely AI is already used

  • Gallup and the Walton Family Foundation report that about six in ten US teachers used AI for work in the 2024-25 school year, three in ten weekly (Gallup, 2026, Tier A).
  • The OECD’s 2024 TALIS survey found 37% of lower-secondary teachers across participating systems used AI in their job, while 72% agreed AI can let students pass off work as their own (OECD Digital Education Outlook 2026, Tier A, via the Australian Policy Online mirror).
  • The guidance gap is the headline: in the 2026 Gallup survey of 2,069 teachers (February-March), only 18% reported formal guidance, 48% only informal guidance, and 34% none (Education Week, Tier B).

Case file

Case 1: Los Angeles Unified and the “Ed” chatbot (procurement failure)

What happened. LAUSD signed a five-year, $6 million contract with AllHere to build “Ed,” a chatbot meant to guide families through their child’s schooling. About three months after launch, and after roughly $3 million had been paid, the district dropped the project as AllHere collapsed (WLRN, Tier B). AllHere’s founder, Joanna Smith-Griffin, was arrested in November 2024 and charged with securities fraud, wire fraud and identity theft. Those are allegations, not findings.

The 2026 sequel. Superintendent Alberto Carvalho resigned on 21 June 2026 amid an FBI investigation that, according to the Los Angeles Times‘ law-enforcement sources, partly concerns his dealings with AllHere. As of the reporting I found, neither he nor the former AllHere sales representative Debra Kerr has been charged (ABC affiliate report, Tier B).

Root cause. The failure was due diligence on a startup’s finances and promises, not model quality. The AI was never given a fair test.

What it does not prove. It does not show that district-level chatbots cannot work. It shows that a flagship launch can rest on a single fragile vendor.

Case 2: Australian Catholic University and the AI detector (detection treated as proof)

What happened. Internal documents obtained by the ABC showed nearly 6,000 alleged misconduct cases in 2024, about 90% involving AI, many resting on Turnitin’s AI indicator. Turnitin itself warns that the tool can be wrong and should not be the sole basis for adverse action. The deputy vice-chancellor said the figures were “substantially overstated” and that around a quarter of referrals were dismissed after investigation; the university stopped using the indicator from 17 March 2025 (summary of the ABC investigation, Tier B; EducationHQ, Tier B).

Human cost. One nursing student reported six months with her transcript marked “results withheld” and said it contributed to missing a graduate job offer (Futurism’s account, Tier B; this is her account, not an adjudicated finding).

Scale arithmetic. Academic-integrity researcher Phillip Dawson estimated in 2023 that even a 0.7% false-positive rate would wrongly flag more than 100,000 assessments a year across Australian higher education (EducationHQ). Small error rates become large numbers at institutional volume.

The bias evidence. A Stanford team tested seven detectors on 91 TOEFL essays by non-native English writers. The average false-positive rate was 61.3%, and at least one detector flagged 97.8% of the essays as AI-written (Liang et al., Patterns, 2023; figures via Tech & Learning, Tier A/B). Caveat: the study is from 2023 and tested the detectors of that time. Vendors say their current models perform better; I found no independent replication at comparable scale.

Case 3: Newby v. Adelphi University (the courtroom test)

What happened. A professor ran a freshman’s history essay through Turnitin, which returned a 100% AI score. The student, who had worked with tutors in a university support program, produced results from two other detectors rating the essay as human-written. A Nassau County, New York court ruled for the student on 28 January 2026 (court opinion, 2026 NY Slip Op 26021, Tier A). According to the Chronicle of Higher Education, the court annulled the violation and ordered the record expunged. It also noted that the administrator who made the finding heard the appeal, and the family spent six figures to clear the record (Chronicle, May 2026, Tier B).

What it does and does not prove. The ruling turned on the university’s process, including evidence it failed to consider. It is not a ruling that detectors are unreliable in general. Outcomes elsewhere are mixed: a University of Minnesota doctoral student’s challenge to an expulsion has not produced a win so far, according to law-firm and press coverage (Steptoe client alert, Tier B).

Case 4: Guardrail-free chatbots and the learning penalty (design failure)

What happened. In a field experiment with nearly a thousand high-school math students, those given a ChatGPT-style GPT-4 interface scored 48% higher during practice, then 17% lower than a control group on an exam taken without AI. A version engineered to give hints instead of answers scored 127% higher in practice and about the same as control on the exam (Bastani et al., PNAS, 2025, Tier A; Wharton summary).

Root cause. Students used the tool as a crutch. The authors’ reading is that practice felt productive while the thinking was outsourced.

What it does not prove. It is one study, on one model generation, in one subject. The tutor with safeguards neutralized the harm but did not add exam gains in this experiment.

Case 5: England’s 2020 grading algorithm (the precedent regulators remember)

What happened. With exams cancelled, Ofqual moderated teacher predictions with a statistical model. About 39.1% of teacher-estimated A-level grades in England were lowered (ITV News, citing Ofqual data, Tier B). Pupils at schools with weaker past results were hit harder; the government reversed course to teacher-assessed grades within days, and Ofqual’s chief resigned on 25 August (BERA, Tier B).

Important label. This was a statistical standardization model, not generative AI or machine learning. It belongs here because it is the high-profile case behind the idea that automated systems deciding students’ futures need oversight, which is the logic of the EU’s high-risk category.

Collateral damage: Chegg

Chegg did not deploy AI badly; AI undercut its business. In 2025, it cut 22% of staff (248 people) and reported first-quarter revenue down 30% year on year, with management blaming Google’s AI Overviews and free student access to chatbots (TechRadar, Tier B). Those are the company’s own explanations.

The ledger: what did it cost?

CaseDocumented costHuman costConfidence
LAUSD / AllHereAbout $3M paid of a $6M, five-year contract; project ended within monthsFounder charged; superintendent resigned amid a federal inquiry (no charges against him reported)High on the money, moderate on later events
Australian Catholic UniversityNo published dollar figureHundreds to thousands of students investigated; months-long holds in reported cases; about a quarter of referrals dismissedModerate: figures disputed by the university itself
AdelphiFamily legal costs in six figuresDisciplinary record, then expungementModerate: one press account
Guardrail-free chatbot studyNone (research setting)17% lower exam scores for the unguarded groupHigh (peer-reviewed)
Ofqual 2020No consolidated figure foundAbout 39% of grade estimates lowered; admissions disruptedHigh on the percentage

I could not find any credible aggregate estimate of total losses, and I would distrust any article that offers one.

The counter-evidence: when AI helped

  • Harvard physics RCT. In a randomized crossover trial with 194 students, a purpose-built AI tutor produced significantly more learning in less time than an in-class active-learning lesson, and students reported higher engagement (Kestin et al., Scientific Reports, 2025, Tier A). Limits: one elite course, short lessons, a custom tool built on the same pedagogy as the comparison class.
  • OECD synthesis (January 2026). AI used with clear pedagogical intent can improve learning and skills such as critical thinking; used as a shortcut it raises performance without learning (OECD Digital Education Outlook 2026, Tier A).
  • Teacher time. The Brookings report cites a US study finding teachers who use AI save nearly six hours a week (NPR on Brookings, January 2026, Tier B).
  • Brookings’ verdict. After consulting 500+ people in 50 countries and reviewing 400+ studies, its Center for Universal Education concluded that for children, the risks currently outweigh the benefits. It calls this a “premortem” and describes the damage as fixable (Brookings, Tier A). It is a synthesis of existing evidence, not a causal experiment.

What the cases have in common

This section is my interpretation, not a finding from any single source.

  1. Nobody owned verification. In LAUSD, vendor claims; at ACU and Adelphi, a probability score treated as a verdict.
  2. The unit of failure is the workflow, not the model. The same detector score is harmless as a prompt for a conversation and harmful as sole evidence.
  3. Student-facing tools succeed or fail on design. The PNAS and Harvard results differ mainly in whether the tool withholds answers and structures thinking.

The rules in October 2026

European Union

  • High-risk status. Annex III of the AI Act lists education uses: determining admission or access, evaluating learning outcomes, assessing the appropriate level of education, and monitoring students for prohibited behavior during tests (Regulation (EU) 2024/1689, Tier A; I did not re-open the Annex text during this research, so verify the wording there).
  • Timing changed. The Digital Omnibus on AI, Regulation (EU) 2026/1744, was published on 24 July 2026 and entered into force on 27 July 2026. It moves stand-alone Annex III obligations from 2 August 2026 to 2 December 2027, and product-embedded systems to 2 August 2028 (National Law Review, Tier B; Orrick, Tier B; DLA Piper, Tier B).
  • What did not move. The delay concerns application dates, not classification, and the substantive requirements (risk management, data governance, documentation, human oversight) are unchanged (usd AG analysis, Tier B). The Omnibus also adds two new prohibitions (nudification and child sexual abuse material) applying from 2 December 2026 (National Law Review).
  • Open question. Whether a take-home-essay AI detector counts as “monitoring prohibited behavior during tests” is a legal interpretation I could not confirm from a primary source. Institutions should ask counsel rather than assume.

United States, federal

  • No federal statute governs classroom AI. Policy runs through the 23 April 2025 executive order on AI education for American youth (White House), a 22 July 2025 Dear Colleague letter saying AI uses are fundable under existing programs (ED press release), and a final grant priority published 13 April 2026 at 91 FR 18774, effective 13 May 2026 (Federal Register notice, Tier A).
  • A supplemental priority steers discretionary grant competitions. It is an incentive to adopt AI, not a safety rule for schools.

United States, states and districts

  • As of June 2026, 37 states plus Puerto Rico had published K-12 AI guidance, according to the AI for Education tracker as relayed by Hechinger (La Nación republication, Tier B/C). Counts differ by tracker and date; another document put it at 31 in January 2026.
  • Most guidance is not binding, and districts must convert it into local policy. A New Mexico legislative staff memo citing NCSL and ECS counts 134 AI-in-education bills introduced across 31 states in 2026 (memo, Tier B/C).

Universities and courts

  • Institutions are retreating from detector-as-evidence: ACU stopped in March 2025, and Curtin University announced it would disable Turnitin’s AI-writing detection from 1 January 2026 (CASRAI summary, Tier C).
  • The legal lesson from Newby is procedural: weigh counter-evidence, document the basis for findings, and use an independent appeal.
  • Remote-proctoring surveillance has its own boundary: in 2022 a federal court held that Cleveland State’s mandatory room scan violated the Fourth Amendment (NPR via KUOW, Tier B).

What a school or university can do now

  1. Never let a detector score stand alone. Use it, at most, to open a conversation, and keep process records.
  2. Contract for failure. Test vendors’ financial health, require data-return and exit terms, and pilot before district-wide launch.
  3. Prefer tutor-style designs for student-facing tools: hints, questions and explanations before answers, and an AI-free check of what students can do alone.
  4. Write down what is allowed, per course. Gallup’s 18% figure suggests most teachers are improvising.
  5. Plan to the EU date if you operate there. December 2027 is a planning horizon, not an exemption.

What I could not verify, and the limits of this study

  • No credible aggregate cost figure exists; the ledger above is case-level only.
  • The detector-bias numbers come from 2023 tests. Current detector accuracy is contested, and I did not find independent large-scale replication.
  • LAUSD’s later events (the resignation and FBI inquiry) rest on press reports citing law-enforcement sources; no charges against district officials were reported in the sources I read.
  • The 37-state count is a tracker figure and varies by source and date.
  • I relied on law-firm and news summaries for the Omnibus’s dates; confirm them against the Official Journal text before citing in legal or compliance work.
  • Brookings is a synthesis and a forward-looking “premortem”; Kestin and Bastani are single studies. Treat them as strong signals, not settled law.

Sources

Primary or near-primary (Tier A)

Reputable reporting and legal analysis (Tier B)

Trackers and secondary summaries (Tier C, used for counts only)