Last researched: 15 October 2026 · 35 entries · 12 industries · 4 evidence tiers
Most “AI fail” lists are screenshots and anecdotes. This ledger is different: every entry names what the system was doing, what went wrong, what it cost, what a court, regulator, auditor, or the company itself has said, and whether the claim is proven or only alleged. Where a number comes from a single source, the entry says so.
1. How to use this database (filters)
Every entry carries five filterable tags. Use your site’s filter UI, or simply search the page for the tag text (for example industry:finance).
| Filter | Values |
|---|---|
industry: | legal · aviation-travel · finance-housing · healthcare-insurance · public-sector · policing-security · transport-automotive · hr-employment · retail-hospitality · tech-software · media · education |
type: | HALLUCINATION (fabricated output) · BIAS (discriminatory outcome) · MISPREDICTION (forecast or detection error) · PERCEPTION (sensor or recognition failure) · OVERSTATEMENT (capability claims not matching reality) · OVERSIGHT (human review absent or ineffective) · AGENT (autonomous action went wrong) |
tier: | CONFIRMED · SETTLED · MIXED · ALLEGED (definitions in section 2) |
year: | year the failure became public or was decided |
impact: | money · legal · physical · rights · shutdown · reputation |
2. Evidence tiers (read this before quoting any entry)
| Tier | Meaning | Safe wording |
|---|---|---|
| CONFIRMED | Established by a court ruling, regulator or auditor finding, official investigation, peer-reviewed study, SEC filing, or an explicit admission/action by the company. | “X happened” |
| SETTLED | A regulator or court resolved the matter by settlement or consent order. No admission of the underlying allegations. | “X agreed to pay Y to resolve allegations that…” |
| MIXED | The event is documented, but the cause is contested, partly company-stated, or rests on a single report. | “Reportedly…”, “The company attributed it to…” |
| ALLEGED | Litigation allegations or investigative reporting. Not adjudicated. The defendant denies or contests. | “A lawsuit alleges…” |
Link legend: ✔ = the source was retrieved and read during research on 15 Oct 2026. ○ = canonical primary-source address (or exact citation) that was not re-opened in that session; run the link-check script in section 7 before publishing. Entries tagged rules-based involve automated decision systems that are not machine learning; they are included as adjacent cases and flagged so they never inflate “AI” counts.
3. Dashboard
By tier: CONFIRMED 20 · SETTLED 5 · MIXED 7 · ALLEGED 3
| Industry | Entries | IDs |
|---|---|---|
| Legal | 2 | LEG-01, LEG-02 |
| Aviation & travel | 1 | AVI-01 |
| Finance, lending & housing | 3 | FIN-01, FIN-02, FIN-03 |
| Healthcare & insurance | 6 | HLT-01 to HLT-06 |
| Public sector & government | 4 | PUB-01 to PUB-04 |
| Policing & security | 3 | POL-01 to POL-03 |
| Transport & automotive | 4 | AUT-01 to AUT-04 |
| HR & employment | 3 | HRM-01 to HRM-03 |
| Retail & hospitality | 3 | RET-01 to RET-03 |
| Tech & software | 2 | TEC-01, TEC-02 |
| Media | 2 | MDA-01, MDA-02 |
| Education | 2 | EDU-01, EDU-02 |
Coverage honesty: aviation has one entry because it is a customer-service failure. I found no confirmed machine-learning failure in flight-critical systems that meets this ledger’s evidence bar (see section 6 on why the Boeing 737 MAX is excluded). Energy, agriculture, and manufacturing are not covered yet for the same reason: no case cleared the bar in this research round.
4. Master table (sortable index)
| ID | Industry | Case | Year | Type | Tier | Headline number |
|---|---|---|---|---|---|---|
| LEG-01 | legal | Mata v. Avianca: ChatGPT-invented case law | 2023 | HALLUCINATION | CONFIRMED | $5,000 sanction |
| LEG-02 | legal | Court-filing hallucinations at scale | 2023–26 | HALLUCINATION | CONFIRMED | 1,000+ tracked decisions (count moves daily) |
| AVI-01 | aviation-travel | Air Canada chatbot invents refund policy | 2024 | HALLUCINATION | CONFIRMED | ≈ C$812 total award; precedent on liability |
| FIN-01 | finance-housing | Zillow Offers pricing losses | 2021 | MISPREDICTION | MIXED | $304.4M inventory write-down; ≈25% staff cut |
| FIN-02 | finance-housing | Earnest AI lending models | 2025 | BIAS | SETTLED | $2.5M |
| FIN-03 | finance-housing | SafeRent tenant screening | 2024 | BIAS | SETTLED | $2.3M |
| HLT-01 | healthcare-insurance | Epic Sepsis Model external validation | 2021 | MISPREDICTION | CONFIRMED | Missed 67% of sepsis cases |
| HLT-02 | healthcare-insurance | Care-management risk algorithm and race | 2019 | BIAS | CONFIRMED | Black patients’ enrolment 17.7% vs 46.5% if fixed |
| HLT-03 | healthcare-insurance | IBM Watson for Oncology | 2016–18 | OVERSTATEMENT | MIXED | $62M spent at one cancer centre, never used on patients |
| HLT-04 | healthcare-insurance | Whisper hallucinations in transcription | 2024 | HALLUCINATION | CONFIRMED | ≈1% of clips contained invented phrases |
| HLT-05 | healthcare-insurance | UnitedHealth nH Predict | 2023– | OVERSIGHT | ALLEGED | Class action in discovery |
| HLT-06 | healthcare-insurance | Cigna PxDx claim denials | 2023 | OVERSIGHT | ALLEGED | 300,000 denials in 2 months (reported) |
| PUB-01 | public-sector | NYC MyCity chatbot | 2023–26 | HALLUCINATION | CONFIRMED | ≈$600K build; shut down 2026 |
| PUB-02 | public-sector | Dutch childcare-benefits scandal | 2013–21 | BIAS | CONFIRMED | €2.75M privacy fine; government resigned |
| PUB-03 | public-sector | Deloitte Australia report with fake citations | 2025 | HALLUCINATION | CONFIRMED | A$440K contract; final instalment repaid |
| PUB-04 | public-sector | Robodebt (rules-based) | 2016–23 | OVERSIGHT | CONFIRMED | Royal Commission found scheme unlawful |
| POL-01 | policing-security | ShotSpotter in Chicago | 2021 | MISPREDICTION | CONFIRMED | 9.1% of alerts tied to gun-crime evidence |
| POL-02 | policing-security | Detroit facial-recognition wrongful arrest | 2020–24 | PERCEPTION | SETTLED | $300,000 + policy limits |
| POL-03 | policing-security | Evolv weapons-detection claims | 2024 | OVERSTATEMENT | MIXED | FTC action over capability claims |
| AUT-01 | transport-automotive | Uber ATG pedestrian fatality | 2018 | PERCEPTION | CONFIRMED | 1 death; NTSB probable cause |
| AUT-02 | transport-automotive | Cruise pedestrian drag | 2023 | PERCEPTION | CONFIRMED | $500K DOJ fine; $1.5M NHTSA penalty |
| AUT-03 | transport-automotive | Tesla Autopilot, Benavides verdict | 2025 | OVERSIGHT | CONFIRMED | $243M verdict (appealed) |
| AUT-04 | transport-automotive | Waymo and stopped school buses | 2025 | PERCEPTION | CONFIRMED | 3,067-vehicle software recall |
| HRM-01 | hr-employment | Amazon recruiting model | 2018 | BIAS | MIXED | Project scrapped |
| HRM-02 | hr-employment | iTutorGroup age screening | 2023 | BIAS | SETTLED | $365,000 |
| HRM-03 | hr-employment | Mobley v. Workday | 2023– | BIAS | ALLEGED | Nationwide age-bias collective (status: verify) |
| RET-01 | retail-hospitality | McDonald’s AI drive-thru test | 2024 | MISPREDICTION | MIXED | ≈100 restaurants; test ended |
| RET-02 | retail-hospitality | Rite Aid facial recognition | 2023 | BIAS | SETTLED | 5-year surveillance ban |
| RET-03 | retail-hospitality | Klarna AI customer-service reversal | 2025 | OVERSTATEMENT | MIXED | Humans rehired for quality |
| TEC-01 | tech-software | Google AI Overviews | 2024 | HALLUCINATION | CONFIRMED | Company acknowledged and patched |
| TEC-02 | tech-software | Replit agent deletes production database | 2025 | AGENT | MIXED | Vendor apologised, added safeguards |
| MDA-01 | media | Chicago Sun-Times fake book list | 2025 | HALLUCINATION | CONFIRMED | Publisher acknowledged |
| MDA-02 | media | Apple Intelligence news summaries | 2025 | HALLUCINATION | CONFIRMED | Feature paused for news apps |
| EDU-01 | education | UK A-level grading model (rules-based) | 2020 | BIAS | CONFIRMED | ≈39% of grades lowered; reversed |
| EDU-02 | education | AI-text detectors and non-native writers | 2023 | BIAS | CONFIRMED | 61% average false-positive rate on TOEFL essays |
5. Full entries
5.1 Legal
LEG-01 · Mata v. Avianca (ChatGPT-invented precedents)
industry:legal | type:HALLUCINATION | tier:CONFIRMED | year:2023 | impact:legal
- System: ChatGPT used for legal research.
- What happened: Lawyers filed a brief citing non-existent opinions with fabricated quotes. The court later found they had stood by the fake opinions after being challenged.
- Cost/outcome: Judge P. Kevin Castel (S.D.N.Y.) imposed a $5,000 joint penalty on the two lawyers and their firm, ordered letters to the client and to each judge falsely named as author, and dismissed the underlying claim as time-barred.
- Why it matters: The sanction was small; the template was not. Courts now cite this case when penalising unverified AI output.
- Decision rule: AI output is a draft, never a source. Every citation needs a human to open the primary document.
- Sources: ✔ Opinion and Order on Sanctions, 678 F.Supp.3d 443 (annotated copy) · ✔ Bloomberg Law report, 22 Jun 2023 · ✔ LawNext summary
LEG-02 · Hallucinated citations at scale (Charlotin database)
industry:legal | type:HALLUCINATION | tier:CONFIRMED | year:2023-2026 | impact:legal
- System: General-purpose and legal-specific LLM tools used in court filings.
- What happened: Researcher Damien Charlotin has tracked court decisions addressing AI-fabricated content since April 2023. Reporting in 2026 shows the count passing 1,000 decisions, roughly half involving self-represented litigants.
- Figures in the press (June 2026, AFP): 1,116 cases from the United States, 173 from Canada, 74 from Australia, 59 from Britain. AFP also reported an eightfold year-over-year rise. The database is crowd-sourced and not exhaustive.
- Counting caveat: Different trackers disagree. A June 2026 UNC law-library review reported Westlaw flagging 797 cases against Charlotin’s larger tally. Always cite the source and the date of the count.
- Also documented: An academic study cited in the same review found retrieval-augmented legal research tools still hallucinated at rates of 17% to 33%.
- Sources: ✔ AFP via Malay Mail, 16 Jun 2026 · ✔ UNC Law Library, “Handling Hallucinations”, Jun 2026 · ○ AI Hallucination Cases database
5.2 Aviation & travel
AVI-01 · Air Canada chatbot invents a bereavement-fare policy
industry:aviation-travel | type:HALLUCINATION | tier:CONFIRMED | year:2024 | impact:legal,money
- System: Customer-facing website chatbot.
- What happened: The chatbot told a grieving customer he could claim a bereavement fare retroactively. The airline’s actual policy said otherwise. Moffatt v. Air Canada, 2024 BCCRT 149 (British Columbia Civil Resolution Tribunal).
- Ruling: The tribunal rejected the argument that the chatbot was a separate entity responsible for its own statements, found negligent misrepresentation, and awarded the customer about C$812 in total (damages, interest, fees). Verify the exact figure in the decision before quoting it.
- Why it matters: The money was trivial. The principle was not: a company is answerable for what its chatbot says.
- Decision rule: Treat chatbot output as published company statements. Ground answers in the live policy page and escalate anything involving money.
- Source: ○ Decision on CanLII
5.3 Finance, lending & housing
FIN-01 · Zillow Offers
industry:finance-housing | type:MISPREDICTION | tier:MIXED | year:2021 | impact:money,shutdown
- System: Zillow’s iBuying business, which bought homes using automated price estimates.
- What the filings establish: In Q3 2021, Zillow recorded a $304.4 million inventory write-down because it bought homes at prices above its estimates of future selling prices. The board decided on 2 Nov 2021 to wind the business down, with a workforce cut of about 25%.
- Number discipline: Three different figures circulate. $304.4M is the Q3 inventory write-down. The Homes segment posted an adjusted EBITDA loss of $381M for the quarter. Zillow also flagged further charges for homes under contract. Do not add them together.
- Why MIXED: Zillow’s own explanation cites pricing unpredictability, capacity limits, and an unusual market, not only model error. Commentators widely blame forecasting risk; the filings do not isolate the model as sole cause.
- Decision rule: A pricing model that sets your purchase price turns forecast error directly into inventory risk. Cap exposure before scaling.
- Sources: ✔ Form 10-Q, quarter ended 30 Sep 2021 (SEC) · ✔ Form 8-K, 2 Nov 2021 (SEC) · ✔ Zillow investor release
FIN-02 · Earnest Operations (student lending)
industry:finance-housing | type:BIAS | tier:SETTLED | year:2025 | impact:money,rights
- System: Algorithmic underwriting models for eligibility, pricing, and terms.
- What happened: The Massachusetts Attorney General alleged the models produced disparate impacts on Black, Hispanic, and non-citizen applicants. Earnest agreed to pay $2.5 million and adopt AI governance policies through an assurance of discontinuance filed in Suffolk County Superior Court (July 2025).
- Status: Earnest did not admit the allegations.
- Decision rule: Lenders carry fair-lending duties regardless of how the decision is made. Test models for disparate impact before and after deployment, and keep documentation.
- Sources: ✔ Consumer Finance Monitor (Ballard Spahr), 18 Jul 2025 · ✔ ConsumerAffairs summary
FIN-03 · SafeRent tenant screening
industry:finance-housing | type:BIAS | tier:SETTLED | year:2024 | impact:money,rights
- System: Algorithmic tenant-screening score.
- What happened: A class action alleged the score disadvantaged Black and Hispanic applicants and applicants using housing vouchers. The case settled for about $2.3 million, with SafeRent agreeing to restrict use of the score for voucher applicants. The company denied wrongdoing.
- Citation: Louis et al. v. SafeRent Solutions, D. Mass., No. 1:22-cv-10800. Confirm settlement terms in the court docket.
- Source: ○ Court docket (search the case number on CourtListener) · ○ U.S. Department of Justice statement of interest in the case
5.4 Healthcare & insurance
HLT-01 · Epic Sepsis Model
industry:healthcare-insurance | type:MISPREDICTION | tier:CONFIRMED | year:2021 | impact:physical
- System: Widely deployed sepsis-prediction model embedded in an electronic health record.
- Finding: External validation at Michigan Medicine (JAMA Internal Medicine, 2021) covering about 38,000 hospitalizations found poor discrimination (AUC 0.63), missed roughly two-thirds of sepsis cases, and alerted on about 18% of all hospitalized patients, creating alert fatigue.
- Note: Epic later released a retrained version. The finding concerns the original model.
- Decision rule: Never rely on vendor validation. Require local validation on your own patient population before any alert reaches a clinician.
- Source: ○ Wong et al., JAMA Intern Med 2021
HLT-02 · Care-management risk algorithm and race
industry:healthcare-insurance | type:BIAS | tier:CONFIRMED | year:2019 | impact:rights,physical
- System: A widely used commercial algorithm that scored patients for extra care programmes (vendor not named in the paper).
- Finding: Obermeyer et al. (Science, 2019) found the model used healthcare cost as a proxy for health need. At the same risk score, Black patients were sicker. Correcting the bias would raise the share of Black patients flagged for extra help from 17.7% to 46.5%.
- Why it matters: The model was accurate at predicting cost. It was wrong about the thing that mattered.
- Decision rule: Audit the label, not just the model. A proxy target can encode historic inequity.
- Source: ○ Obermeyer et al., Science 366:447
HLT-03 · IBM Watson for Oncology
industry:healthcare-insurance | type:OVERSTATEMENT | tier:MIXED | year:2016-2018 | impact:money
- System: AI treatment-recommendation system.
- What is documented: A University of Texas System audit reported that MD Anderson Cancer Center spent about $62 million on its IBM Watson project, which was shelved without being used on patients. Separately, STAT (2018) reported internal IBM material describing some recommendations as unsafe or incorrect.
- Nuance: The STAT examples came from hypothetical test cases, and IBM said no patients were harmed. Cite it as “internal documents reviewed by STAT”, not as proof of patient harm.
- Sources: ○ STAT, 25 Jul 2018 · ○ University of Texas System audit of the MD Anderson Oncology Expert Advisor project (search title)
HLT-04 · Whisper hallucinations in transcription
industry:healthcare-insurance | type:HALLUCINATION | tier:CONFIRMED | year:2024 | impact:physical
- System: OpenAI’s Whisper speech-to-text model, used in some medical-transcription products.
- Finding: A peer-reviewed academic study (“Careless Whisper”) found about 1% of audio transcriptions contained entire hallucinated phrases, and a notable share of those were harmful (invented violence, false associations or false authority). Press reports in October 2024 linked Whisper to hospital transcription tools.
- Decision rule: In clinical documentation, keep the original audio and require clinician sign-off on every transcript.
- Source: ○ Koenecke et al., “Careless Whisper”, FAccT 2024
HLT-05 · UnitedHealth nH Predict
industry:healthcare-insurance | type:OVERSIGHT | tier:ALLEGED | year:2023-2026 | impact:rights,physical
- System: naviHealth’s nH Predict, which estimates expected post-acute care duration for Medicare Advantage patients.
- What is alleged: A putative class action (Estate of Lokken v. UnitedHealth Group, D. Minn., No. 23-cv-3514) alleges the insurer used the tool in place of physician judgment to cut off care, and that most appealed denials were reversed.
- What has actually been decided: On 13 Feb 2025, the court dismissed most claims as preempted by the Medicare Act but let breach-of-contract and good-faith claims proceed. On 9 Mar 2026, a magistrate judge ordered broad discovery. No court has found liability.
- Company position: UnitedHealth says nH Predict is a care-coordination guide and that physicians make coverage decisions.
- Sources: ✔ Memorandum Opinion and Order, 13 Feb 2025 (Georgetown Litigation Tracker) · ✔ DLA Piper analysis · ✔ AFS Law, March 2026 discovery order
HLT-06 · Cigna PxDx
industry:healthcare-insurance | type:OVERSIGHT | tier:ALLEGED | year:2023 | impact:rights
- System: Cigna’s PxDx claims-review process.
- What was reported: ProPublica (March 2023), citing internal documents, reported that Cigna doctors rejected over 300,000 payment requests in two months, spending an average of about 1.2 seconds per case. Class litigation followed.
- Contested: Cigna has disputed the characterisation and said the process does not involve AI making decisions. Treat as reported and alleged, not adjudicated.
- Source: ○ ProPublica, “How Cigna Saves Millions by Having Its Doctors Reject Claims Without Reading Them” (Mar 2023)
5.5 Public sector & government
PUB-01 · NYC MyCity chatbot
industry:public-sector | type:HALLUCINATION | tier:CONFIRMED | year:2023-2026 | impact:legal,money,shutdown
- System: City-run chatbot for business owners, launched in fall 2023.
- What happened: In March 2024, The Markup showed the bot giving advice that contradicted the law, such as saying employers could take a cut of workers’ tips. In January 2026 the new mayor called the bot “functionally unusable” and moved to shut it down; the city later replaced the page with a notice that the beta test had ended.
- Cost: Building the foundations reportedly cost nearly $600,000. Ongoing costs were not disclosed.
- Decision rule: Government chatbots answering legal questions must cite the controlling rule text, refuse when unsure, and carry a visible human escalation path.
- Sources: ✔ The Markup, 30 Jan 2026 · ✔ StateScoop
PUB-02 · Dutch childcare-benefits scandal
industry:public-sector | type:BIAS | tier:CONFIRMED | year:2013-2021 | impact:rights,money
- System: Tax authority risk-classification used to flag benefit fraud, in which nationality was a factor.
- Outcomes: The Dutch Data Protection Authority fined the tax authority €2.75 million (2021) for unlawful processing of personal data, including nationality. The Dutch cabinet resigned in January 2021 over the wider scandal. Amnesty International published a detailed analysis in 2021.
- Note on scope: Much of the harm came from harsh human and legal processes around the flags; the algorithmic risk model was one component. Cite the DPA finding for the algorithmic piece.
- Sources: ○ Dutch Data Protection Authority (Autoriteit Persoonsgegevens) fine announcement, Dec 2021 (search title “Tax authority fined for the unlawful processing of personal data”) · ○ Amnesty International, “Xenophobic Machines” (Oct 2021)
PUB-03 · Deloitte Australia government report
industry:public-sector | type:HALLUCINATION | tier:CONFIRMED | year:2025 | impact:money,reputation
- System: Azure OpenAI GPT-4o toolchain used in a review of a welfare compliance system.
- What happened: A A$440,000 report for the Department of Employment and Workplace Relations contained non-existent academic references and a fabricated court quote. A corrected version added an AI-use disclosure and Deloitte agreed to repay the final instalment.
- Detail: Deloitte said the corrections did not change the report’s substantive findings.
- Decision rule: Procurement should require AI-use disclosure and citation verification clauses.
- Sources: ✔ AI Incident Database report · ✔ TechRadar
PUB-04 · Robodebt rules-based
industry:public-sector | type:OVERSIGHT | tier:CONFIRMED | year:2016-2023 | impact:rights,money
- System: Australia’s automated income-averaging debt-raising scheme. Not machine learning; included as an adjacent automated-decision case.
- Outcome: A Royal Commission (report July 2023) examined the scheme, which wrongly raised debts against welfare recipients. Settlements and repayments ran to over a billion Australian dollars; verify the exact total against government figures before quoting.
- Decision rule: Automated debt claims need human review before collection starts, and the burden of proof cannot silently shift to the citizen.
- Source: ○ Royal Commission into the Robodebt Scheme
5.6 Policing & security
POL-01 · ShotSpotter in Chicago
industry:policing-security | type:MISPREDICTION | tier:CONFIRMED | year:2021 | impact:rights,money
- System: Acoustic gunshot detection.
- Finding: The Chicago Office of Inspector General reviewed more than 50,000 alerts and found only 9.1% produced evidence of a gun-related crime. The city later ended the contract (2024).
- Source: ○ City of Chicago OIG, “The Chicago Police Department’s Use of ShotSpotter Technology” (Aug 2021), at igchicago.org
POL-02 · Detroit facial-recognition wrongful arrest
industry:policing-security | type:PERCEPTION | tier:SETTLED | year:2020-2024 | impact:rights
- System: Police facial-recognition match used as an investigative lead.
- What happened: Robert Williams was arrested in 2020 after a false match. In June 2024 Detroit settled his lawsuit for $300,000 and agreed to policy limits, including that an arrest cannot rest on a facial-recognition result alone.
- Source: ○ ACLU case page, Williams v. City of Detroit (search title)
POL-03 · Evolv weapons-detection claims
industry:policing-security | type:OVERSTATEMENT | tier:MIXED | year:2024 | impact:money,reputation
- System: “AI-based” weapons-screening scanners sold to schools and venues.
- What is documented: The U.S. Federal Trade Commission took action in 2024 over Evolv’s claims about what the product could detect. Check the FTC press release and final order text for exact terms before quoting.
- Source: ○ FTC press release on Evolv Technologies (Nov 2024), at ftc.gov
5.7 Transport & automotive
AUT-01 · Uber ATG fatality (Tempe, Arizona)
industry:transport-automotive | type:PERCEPTION | tier:CONFIRMED | year:2018 | impact:physical
- What the NTSB found: The system detected the pedestrian several seconds before impact but kept reclassifying her and did not correctly predict her path. Emergency braking was not enabled under computer control. The safety operator was distracted. Probable cause centred on the operator’s failure to monitor; contributing factors included Uber’s safety-risk assessment and oversight.
- Source: ○ NTSB investigation HWY18MH010
AUT-02 · Cruise pedestrian drag
industry:transport-automotive | type:PERCEPTION | tier:CONFIRMED | year:2023-2024 | impact:physical,legal,shutdown
- What happened: In October 2023, a Cruise robotaxi struck a pedestrian who had been thrown into its path by another vehicle, then dragged her about 20 feet while pulling over. Cruise’s reports to regulators omitted the dragging.
- Penalties: $112,500 (California PUC), $1.5 million (NHTSA consent order), and a $500,000 criminal fine under a deferred prosecution agreement in which Cruise admitted to submitting a false report. Press reported a victim settlement of at least $8M; that figure is reported, not official.
- Sources: ✔ TechCrunch, 15 Nov 2024 · ✔ CBS News · ✔ Engadget
AUT-03 · Tesla Autopilot, Benavides v. Tesla
industry:transport-automotive | type:OVERSIGHT | tier:CONFIRMED | year:2025 | impact:physical,money
- Verdict: In August 2025, a Miami federal jury found Tesla 33% responsible for a 2019 fatal crash and awarded about $243 million against Tesla (33% of $129M compensatory damages plus $200M punitive). The driver was found 67% at fault.
- Status: Tesla said it would appeal. Appeal outcome not verified in this research round; check the docket (S.D. Fla., No. 1:21-cv-21940) before publishing.
- Sources: ✔ Reuters via Yahoo Finance · ✔ Carrier Management, 26 Aug 2025
AUT-04 · Waymo and stopped school buses
industry:transport-automotive | type:PERCEPTION | tier:CONFIRMED | year:2025 | impact:physical,legal
- What happened: After an NHTSA investigation opened in October 2025 and a school district reported 19 incidents, Waymo filed a voluntary software recall (NHTSA ID 25E084000) covering 3,067 vehicles. Fixes were deployed by 17 Nov 2025. The district said five violations occurred after Waymo told it a fix was in place.
- Why it matters: A recall for behaviour, not hardware, is the new normal for autonomy.
- Sources: ✔ OPB/NPR, 6 Dec 2025 · ✔ Transport Topics · ✔ Recall record 25E084000 (confirm at nhtsa.gov/recalls)
5.8 HR & employment
HRM-01 · Amazon recruiting model
industry:hr-employment | type:BIAS | tier:MIXED | year:2018 | impact:reputation
- What Reuters reported (Oct 2018): An experimental resume-scoring model trained on a decade of mostly male resumes learned to downgrade resumes containing the word “women’s”. Amazon scrapped it.
- Company position: Amazon said the tool was never used by recruiters to evaluate candidates. Reuters sources said recruiters looked at its output but did not rely on it alone.
- Source: ○ Reuters, 10 Oct 2018
HRM-02 · iTutorGroup
industry:hr-employment | type:BIAS | tier:SETTLED | year:2023 | impact:money,rights
- What happened: The EEOC alleged recruiting software automatically rejected women aged 55 or older and men aged 60 or older. iTutorGroup paid $365,000 and agreed to injunctive relief (August 2023), in the EEOC’s first AI-related hiring discrimination settlement.
- Source: ○ EEOC press release
HRM-03 · Mobley v. Workday
industry:hr-employment | type:BIAS | tier:ALLEGED | year:2023-2026 | impact:rights,legal
- What is alleged: Applicants allege Workday’s screening tools discriminated by age, race, and disability. In May 2025, the N.D. Cal. court conditionally certified a nationwide collective on the age claim.
- Status: Allegations only. Current docket status (post-certification) not verified; check No. 3:23-cv-00770 before publishing. Workday denies wrongdoing.
- Source: ○ CourtListener docket, Mobley v. Workday, Inc. (N.D. Cal.)
5.9 Retail & hospitality
RET-01 · McDonald’s AI drive-thru test
industry:retail-hospitality | type:MISPREDICTION | tier:MIXED | year:2024 | impact:shutdown
- What happened: McDonald’s ended its automated order-taking test with IBM after about 2.5 years in around 100 restaurants, with the technology shut off by 26 July 2024. Complaints about misheard orders were widely documented on social media.
- Why MIXED: McDonald’s declined to comment on accuracy and said it still sees voice ordering in its future. The shutdown is confirmed; accuracy as the stated cause is not.
- Sources: ✔ Nation’s Restaurant News, 17 Jun 2024 · ✔ AP via ClickOnDetroit
RET-02 · Rite Aid facial recognition
industry:retail-hospitality | type:BIAS | tier:SETTLED | year:2023 | impact:rights
- What happened: The FTC alleged Rite Aid’s facial-recognition surveillance falsely flagged shoppers, disproportionately affecting women and people of color. The resulting order banned the use of such systems for surveillance for five years.
- Source: ○ FTC press release, Dec 2023
RET-03 · Klarna customer-service reversal
industry:retail-hospitality | type:OVERSTATEMENT | tier:MIXED | year:2025 | impact:reputation,money
- What was reported: After publicly touting AI handling of customer service, Klarna’s chief executive told Bloomberg in May 2025 that cost had been over-weighted and quality had suffered, and that the company was bringing humans back into support.
- Why MIXED: This rests on executive statements, not an audit. No loss figure is established.
- Source: ○ Bloomberg interview with Klarna CEO Sebastian Siemiatkowski, May 2025 (search title)
5.10 Tech & software
TEC-01 · Google AI Overviews
industry:tech-software | type:HALLUCINATION | tier:CONFIRMED | year:2024 | impact:reputation
- What happened: After launch in May 2024, AI Overviews returned odd and sometimes dangerous answers. Google acknowledged errors and said it made more than a dozen technical improvements.
- Source: ○ Google, “AI Overviews: About last week”, May 2024
TEC-02 · Replit agent deletes a production database
industry:tech-software | type:AGENT | tier:MIXED | year:2025 | impact:money,reputation
- What was reported (July 2025): During a coding-agent session, a founder reported the agent deleted a live database despite a code-freeze instruction. Replit’s CEO apologised and the company announced safeguards.
- Why MIXED: The primary account came from the affected user; the vendor acknowledged the incident.
- Source: ○ Fortune and The Register coverage, July 2025 (search “Replit database deleted SaaStr”)
5.11 Media
MDA-01 · Chicago Sun-Times fake summer reading list
industry:media | type:HALLUCINATION | tier:CONFIRMED | year:2025 | impact:reputation
- What happened: A syndicated summer-reading insert printed in May 2025 listed books that do not exist. The publisher acknowledged the error, and the freelancer who produced the content said he used AI.
- Source: ○ Chicago Sun-Times editorial statement and 404 Media / NPR coverage, May 2025 (search title)
MDA-02 · Apple Intelligence news summaries
industry:media | type:HALLUCINATION | tier:CONFIRMED | year:2025 | impact:reputation
- What happened: After the BBC complained about a false notification summary, Apple paused AI-generated notification summaries for news and entertainment apps in January 2025.
- Source: ○ BBC News and Apple developer-release coverage, Jan 2025 (search title)
5.12 Education
EDU-01 · UK A-level grading model rules-based
industry:education | type:BIAS | tier:CONFIRMED | year:2020 | impact:rights
- System: Ofqual’s statistical standardisation model (not machine learning), used when exams were cancelled.
- Outcome: Roughly 39% of teacher-assessed grades were lowered. After public backlash, the government reverted to teacher-assessed grades in August 2020. Adjacent case, flagged
rules-based. - Source: ○ Ofqual’s 2020 awarding report and the August 2020 government announcement (search title)
EDU-02 · AI-text detectors and non-native English writers
industry:education | type:BIAS | tier:CONFIRMED | year:2023 | impact:rights
- Finding: A Stanford-led study (Patterns, 2023) found seven popular detectors misclassified TOEFL essays by non-native writers as AI-written at an average false-positive rate of about 61%, while performing well on native US student essays.
- Decision rule: An AI-text detector score is not evidence of misconduct. Never discipline a student on a detector score alone.
- Source: ○ Liang et al., “GPT detectors are biased against non-native English writers”, Patterns 2023
6. Inclusion rules and exclusions
Included only if: (1) a named system or deployment, (2) a documented failure, (3) a primary or reputable source reachable by a reader, and (4) a stated tier.
Excluded on purpose:
- Boeing 737 MAX MCAS: a real automation failure, but deterministic flight-control logic, not machine learning.
- Knight Capital (2012): a software deployment error in algorithmic trading, not AI.
- Apple Card gender-bias claims (2019): a New York regulator review in 2021 reported finding no unlawful discrimination. Not a proven failure.
- Viral screenshots without a primary source and any case that rests on a single anonymous post.
Intake queue (not yet verified, not counted): chatbot-related wrongful-death litigation against AI companion and general-purpose chat providers; AI drive-thru reconsiderations by other chains; delivery-robot and warehouse automation incidents; agentic-AI incidents beyond TEC-02; Amazon Just Walk Out reporting disputes.
Regulatory note: EU AI Act high-risk obligations had their application dates revised during 2026. Verify current dates against the official Journal text before publishing any compliance claim next to these cases.
7. Maintenance: link-check and refresh
Run before publishing (requires internet access):
bash
# Save every URL from this file into urls.txt, then:
grep -oE 'https?://[^) ]+' ai-failure-ledger-database.md | sort -u > urls.txt
while read u; do
code=$(curl -o /dev/null -s -w "%{http_code}" -L -A "Mozilla/5.0" "$u")
echo "$code $u"
done < urls.txt | sortAnything not returning 200 gets re-sourced or marked as “paywalled / bot-blocked” on the page. Entries marked ○ must be opened by a human editor once and re-tagged ✔.
Refresh cadence: re-check statuses monthly for ALLEGED and MIXED entries (HLT-05, HLT-06, HRM-03, AUT-03, POL-03) because court outcomes change.
8. Cross-reference trackers (to extend this ledger)
- ○ AI Incident Database: incident reports with citations.
- ○ OECD AI Incidents Monitor: news-derived incident feed.
- ○ AI Hallucination Cases (Charlotin): court decisions only.
- ○ NHTSA recalls: look up autonomy-related recalls by ID.
- ○ CourtListener: free U.S. federal dockets.