By Tom Morgan · Last verified: October 21, 2026
Direct answer: The best-documented AI and automation failures in finance were rarely about the algorithm alone. The expensive part was missing controls, untested assumptions, overstated claims, or a human process that trusted a screen.
- The cost. Penalties in the two AI-specific enforcement matters below (Delphia and Global Predictions, Earnest) total about $2.9 million. Zillow’s home-buying arm took a $304.4 million inventory write-down in a single quarter amid market and operational strain, roughly 100 times larger.
- The rules. SR 26-2 replaced SR 11-7 on April 17, 2026, and leaves generative and agentic AI outside its scope. The CFPB removed disparate-impact liability from Regulation B effective July 21, 2026. The EU moved its high-risk deadline, which includes credit assessment, to December 2, 2027. The FSB issued 12 non-binding sound practices on October 15, 2026.
- The bottom line. With no regulator handing you a template for generative or agentic AI, you can defend your program only if you can show what you inventoried, tested, and logged.
About this rebuild. An earlier version of this article focused on adoption statistics and governance. This version is a case study. Every case below is tied to a regulator filing, court record, or named-outlet report, and each one says plainly what is proven, what is alleged, and what I could not verify. Adoption and investment figures from the old version were dropped because they were not re-checked in this pass.
Contents
- How these cases were chosen
- The scorecard
- Case 1: Zillow Offers, a model-dependent business meets a shifting market (2021)
- Case 2: Knight Capital, the non-AI precedent every AI team should study (2012)
- Case 3: Arup, a deepfake video call and $25.6 million (2024)
- Case 4: Delphia and Global Predictions, claiming AI you did not use (2024)
- Case 5: Nate, when the AI was allegedly people (2025)
- Case 6: Earnest, an underwriting model nobody tested for disparate impact (2025)
- Case 7: The K1 “AI hedge fund” dispute (2017 to 2019)
- Case 8: Apple Card, the failure that did not happen
- What the cases have in common
- What the rules look like now
- A defensible program, built from the failures
- FAQ
- Sources and verification notes
How these cases were chosen
I used four filters. The event had to be documented by a regulator, a court filing, a company filing, or a major named outlet. The dollar figure had to come from one of those sources, not from a vendor blog. The case had to involve either an AI or machine-learning system, an automated system with direct lessons for AI, or AI used against a financial process. And I had to be able to say what was proven versus alleged.
Two honesty notes. First, Zillow and Arup are not banks. Zillow ran a home-buying business with a mortgage arm, and Arup is an engineering firm whose finance team was defrauded. They are included because the failure mechanisms transfer directly to financial institutions. Second, Knight Capital is not an AI case. It is included as the cleanest example of what happens when an automated system runs without effective controls, which is exactly the question regulators now ask about AI.
How to read the figures. Each number comes from one of three tiers: a filing or regulator statement, a named press report, or an allegation. Only the first tier is treated as established. Nate and the K1 dispute are the two cases still at the allegation stage. They illustrate failure patterns, and this article does not conclude who was right.
The scorecard
| Case | Year | Documented dollar figure | Failure type | Status |
|---|---|---|---|---|
| Zillow Offers | 2021 | $304.4M Q3 inventory write-down; more than $540M total write-down announced | Model-dependent business under market and operational strain | Business wound down; ~25% workforce cut |
| Knight Capital (non-AI) | 2012 | $460M+ trading loss; $12M SEC penalty | Uncontrolled automated system | SEC settlement |
| Arup deepfake fraud | 2024 | HK$200M (about $25.6M) across 15 transfers | AI used as a weapon against a human process | Police investigation; company confirmed |
| Delphia / Global Predictions | 2024 | $400K combined penalties | Misrepresenting AI use | SEC settled orders |
| Nate Inc. | 2025 | Over $42M raised from investors (alleged) | Alleged false AI claims | ALLEGED: criminal and SEC charges, no verified outcome |
| Earnest Operations | 2025 | $2.5M settlement | Untested underwriting models | State AG settlement, no admission |
| K1 / Tyndaris dispute | 2018 to 2019 | $20.5M single-day loss; $23M counterclaim | Alleged overstated AI capability | DISPUTED: litigated, outcome not verified |
| Apple Card / Goldman | 2019 to 2021 | No violation found | Suspected algorithmic bias | State regulator cleared it |
Case 1: Zillow Offers, a model-dependent business meets a shifting market (2021)
What happened. Zillow’s iBuying arm used the company’s valuation technology to make cash offers on homes, then resold them. On November 2, 2021, the board decided to wind the business down. In its SEC filing, Zillow disclosed a $304.4 million inventory write-down because it had bought homes in the third quarter at higher prices than its estimates of future selling prices, and it expected another $240 million to $265 million of losses on homes under contract. CNN’s report put the total announced write-down above $540 million and the layoffs near 2,000 jobs, about a quarter of the workforce. Zillow bought 9,680 homes in that quarter. The Wall Street Journal later reported a full-year Zillow Offers loss of $881 million, as relayed by Input; treat that figure as press-reported rather than taken from a primary filing I could open.
What failed. Zillow’s own 10-Q says the decision reflected home-price unpredictability, capacity constraints, and other operational problems made worse by an unprecedented market, a pandemic, and labor and supply-chain strain. So this was not a pure model bug, and anyone who tells you it was is simplifying. It was a model-dependent business with thin tolerance for error. For scale, Zillow’s FY2022 10-K reports a Zestimate median error of 2.7% on listed homes and 7.6% on off-market homes. That is excellent for a consumer estimate tool. My analysis: it is a hard standard for a business that buys at scale and resells on slim margins.
The lesson for finance. Valuation and credit models are trained on a regime. When the regime shifts, accuracy statistics from the calm period stop describing your risk. Position limits, stress scenarios, and a human right to say stop matter more than the headline accuracy number.
Case 2: Knight Capital, the non-AI precedent every AI team should study (2012)
What happened. On August 1, 2012, a defect in Knight Capital’s automated equity router sent more than 4 million orders into the market in about 45 minutes while trying to fill just 212 customer orders. Knight traded more than 397 million shares and ended up with a loss above $460 million. Per Reuters, the SEC found that the firm’s own systems had generated 97 automated emails flagging the problem before the open, and nobody acted. Knight settled for a $12 million penalty and agreed to an independent consultant review. It was the SEC’s first enforcement action under the Market Access Rule (Rule 15c3-5), and WilmerHale’s summary lays out the control failures in detail.
Why it belongs here. This was ordinary software, not machine learning. But the regulatory theory, that firms must have controls reasonably designed to limit the risk of automated systems, is exactly the lens examiners now bring to AI agents. FINRA’s 2026 report flags agents acting beyond intended scope and hard-to-audit behavior. Knight is what that looks like when it goes wrong.
The lesson. Hard limits outside the system, working alarms, and a tested kill switch are not optional. The warning existed. The process to act on it did not.
Case 3: Arup, a deepfake video call and $25.6 million (2024)
What happened. A finance employee in Arup’s Hong Kong office received a message that seemed to come from the UK-based CFO about a confidential transaction. The employee suspected phishing, then joined a video call in which the CFO and other colleagues appeared and sounded real. According to Hong Kong police and CNN’s reporting, all of them were deepfake recreations. After the call the employee made 15 transfers totaling HK$200 million, about $25.6 million. Arup confirmed fake voices and images were used and said its financial stability and internal systems were not affected. Its global CIO said such attacks were rising sharply.
What failed. The model did not fail. The verification process did. A video call was treated as proof of identity, and payment release did not require an independent, out-of-band confirmation.
The lesson. For any payment above a set threshold, require a second channel the attacker does not control. Seeing and hearing someone is no longer authentication.
Case 4: Delphia and Global Predictions, claiming AI you did not use (2024)
What happened. On March 18, 2024, the SEC announced settled charges against two investment advisers for false and misleading statements about their use of AI, the first enforcement actions of this kind. Delphia, based in Toronto, had claimed from 2019 to 2023 that it used AI and machine learning with client data in its investment process. Global Predictions, in San Francisco, made misleading claims in 2023 on its website and social media, including calling itself the first regulated AI financial advisor. Penalties were $225,000 and $175,000, $400,000 in total, without admitting or denying the findings. O’Melveny’s analysis notes Global Predictions also had contract-language and conflict-disclosure problems.
The lesson. “AI-powered” is now a regulated claim. If marketing says the product uses AI, compliance needs evidence of what the system actually does, and the claim must match it.
Case 5: Nate, when the AI was allegedly people (2025)
What happened. On April 9, 2025, the SEC and the Southern District of New York filed parallel civil and criminal actions against Nate’s founder, Albert Saniger. According to DLA Piper’s summary, prosecutors allege he raised about $42 million by claiming the shopping app used advanced AI to complete purchases autonomously, when contract workers in the Philippines and elsewhere were allegedly doing the work by hand. Debevoise reports claimed automation rates of 93% to 97%.
Read this carefully. These are charges and allegations. I did not find a verified final outcome, so I do not state any of it as established fact. Nate is e-commerce, not a bank. It matters to finance because the same pattern, automation rates that cannot be audited, shows up in diligence on fintech vendors.
The lesson. When you buy or invest in an AI product, ask how the automation rate is measured and request an independent test. Treat “fully autonomous” as a claim to verify.
Case 6: Earnest, an underwriting model nobody tested for disparate impact (2025)
What happened. On July 10, 2025, the Massachusetts Attorney General announced a $2.5 million settlement with student lender Earnest Operations. The AG alleged the company failed to test its models for disparate impact and trained them on arbitrary, discretionary human decisions. Kelley Drye’s write-up adds that the allegations covered a Cohort Default Rate input, a knockout rule denying applicants without a green card, and inaccurate adverse action notices. Earnest did not admit the allegations. It must build AI governance, write policies, and report to the AG. DLA Piper describes it as one of the first state enforcement actions on algorithmic bias.
What it shows. The cost was small, $2.5 million. The obligations were not. A multi-year governance build is the real price, and the settlement reads like a checklist: test for disparate impact, document human overrides, give specific adverse-action reasons.
An important twist for 2026. The federal legal theory behind this kind of claim has since narrowed. See the CFPB section below. The state route Massachusetts used did not go away.
Case 7: The K1 “AI hedge fund” dispute (2017 to 2019)
What happened. A Hong Kong investor, Samathur Li Kin-kan, put money behind an AI-driven trading platform called K1, marketed by Tyndaris Investments. According to Bloomberg, Tyndaris sued for about $3 million in unpaid fees and Li counterclaimed for $23 million, alleging the system’s abilities were overstated. Financial Advisor magazine’s retelling of Bloomberg’s reporting says a stop-loss triggered by surprise inflation data left the fund $20.5 million poorer in a day in February 2018, and the parties disputed who set that stop-loss and what had been promised. The same piece cites Eurekahedge data showing AI-focused hedge funds earned less than half the S&P 500’s return in the three years to 2018.
Limits. This was widely described as the first known lawsuit over losses from an autonomous trading system. All of the above is an allegation and dispute from 2018 to 2019 filings. I could not locate a published judgment, so I make no claim about who was right.
The lesson. In any AI-managed mandate, write down who sets risk limits, whether a human or the model, and what was backtested. Ambiguity there is where the litigation starts.
Case 8: Apple Card, the failure that did not happen
A good case study includes one where the scary story did not hold up. In November 2019, a prominent developer said the Apple Card’s algorithm gave him a credit limit 20 times his wife’s. New York’s Department of Financial Services investigated, reviewing thousands of pages of records and data on about 400,000 New York applicants. In March 2021, it reported no evidence of unlawful discrimination, and that men and women with similar credit characteristics generally had similar outcomes.
The regulator still criticized deficiencies in customer service and a perceived lack of transparency that undermined trust. The takeaway is uncomfortable and useful: an unexplainable decision can cost you an investigation and your reputation even when the decision itself was lawful. Being right is not enough if you cannot show why.
What the cases have in common
A few patterns hold up across very different events.
- The model was rarely the only problem. Knight had an unheeded alarm. Arup had a weak verification step. Earnest had no disparate-impact testing. Zillow had a business that could not absorb error in a shifting market.
- Claims got ahead of reality. Delphia, Global Predictions, and (allegedly) Nate and Tyndaris all involved describing AI capability that regulators or counterparties say was not there.
- Fines understate the bill. The AI-specific penalties here ($400,000 and $2.5 million) are tiny next to operating losses. The real costs were write-downs, lost business lines, multi-year consent obligations, and legal fees.
- Explainability is a defense. The Apple Card outcome and the Earnest allegations both turned on whether the lender could show how decisions were made.
A note on projections. Forward-looking fraud figures, such as Deloitte’s scenario of US fraud losses rising from $12.3 billion in 2023 to $40 billion by 2027 (a 32% compound annual growth rate, about 3.3x), are planning scenarios built on assumed adoption rates. They are not measured losses, and none of the case figures above depend on them.
What the rules look like now
This is the position as of October 21, 2026. Most of it is guidance, not statute, and that is the point.
| Area | Current position | What it means in practice |
|---|---|---|
| US bank model risk | SR 26-2, OCC Bulletin 2026-13, FDIC FIL-15-2026, issued April 17, 2026, rescinding the 2011 guidance. Generative and agentic AI are expressly outside its scope. | Traditional and non-generative ML models stay in scope. For generative and agentic tools, the agencies say to apply existing risk-management and governance practices. |
| US federal AI request for information | The agencies said in April they plan to issue an RFI “in the near future.” | I could not find a published RFI as of today. Check the OCC and Fed sites before relying on this. |
| CFPB, fair lending | Final Regulation B rule published April 22, 2026, effective July 21, 2026, states that ECOA does not authorize disparate-impact liability. | Disparate treatment and intentional use of proxies remain prohibited. State fair-lending laws and the Fair Housing Act may still impose disparate-impact liability, and private plaintiffs may test the rule. |
| SEC | AI-washing enforcement: Delphia and Global Predictions (2024), Nate (2025). | Marketing and filing claims about AI are treated as statements that must be accurate. |
| FINRA | 2026 Annual Regulatory Oversight Report, released December 9, 2025, added a generative AI section and, for the first time, AI agents. | No new rules, but supervision, recordkeeping, and communications rules apply. Agent-specific supervision and logging are suggested. |
| EU AI Act | Regulation (EU) 2026/1744 entered into force July 27, 2026. Annex III high-risk obligations, which include credit assessment, now apply from December 2, 2027. Annex I from August 2, 2028. | The delay covers the high-risk tier only. Other obligations continue on their own schedules. |
| FSB (international) | 12 sound practices finalized October 15, 2026. | Non-binding, board-level, lifecycle-focused. The closest thing to a global reference for AI governance in finance. |
| Colorado | SB 26-189 signed May 14, 2026, repealing and replacing the 2024 AI Act. Effective January 1, 2027, with a narrower notice-and-explanation approach. | Enforcement is by the state AG with no private right of action. Litigation could still affect timing. |
The key sources behind that table
- SR 26-2. The OCC bulletin and news release state that generative and agentic AI models are novel and rapidly evolving and not covered. Sullivan & Cromwell summarizes the rescissions and scope.
- Regulation B. Venable, Greenberg Traurig and Cooley all describe the rule, its effective date, and what remains prohibited.
- EU. White & Case and K&L Gates confirm publication on July 24 and entry into force on July 27, 2026.
- FSB. The FSB’s own release and the ABA Banking Journal.
- Colorado. Goodwin and Nixon Peabody.
- FINRA. The report itself and Debevoise’s summary.
What the rules do not say
Two gaps matter more than anything in the table.
There is no federal template for generative or agentic AI model risk. Out of scope is not out of supervision, because consumer protection, third-party risk, cybersecurity, and recordkeeping obligations still apply. The practical effect is that examiners can ask why you built what you built, and the answer has to be on paper.
The federal fair-lending safety net for algorithmic bias got smaller, but not gone. An Earnest-style case would not rest on the federal disparate-impact theory today. Massachusetts used its own consumer-protection law, and other states have issued AI guidance. If you operate across states, the strictest relevant state is your real standard. This is my reading of the sources, and it is a question for counsel, not a conclusion.
A defensible program, built from the failures
Each control below answers a specific case, not a generic best practice.
- Keep hard limits outside the model (Knight, Zillow). Position, order-rate, and spend limits that the system cannot override, plus a kill switch you have actually tested.
- Act on the alarm (Knight). Every automated warning needs an owner and a response time.
- Verify out of band (Arup). Payments above a threshold need confirmation through a channel the requester does not control.
- Test for disparate impact and document overrides (Earnest). Even where federal law narrowed, state exposure and reputational exposure remain.
- Match every AI claim to evidence (Delphia, Global Predictions, Nate). Marketing, filings, and investor decks go through compliance, with a record of what the system actually does and how automation rates are measured.
- Define who sets risk limits in any AI-managed mandate (K1). In writing, before launch.
- Be able to explain decisions (Apple Card). Specific adverse-action reasons and a plain-language explanation path.
- Write down decisions made without guidance, with dates (SR 26-2 gap). For generative and agentic AI, your documented interim standard is the defense. The FSB’s 12 practices are a sensible template to map against.
FAQ
What is the most expensive documented AI failure in finance? By the filings I could verify, Zillow Offers: a $304.4 million write-down in one quarter and more than $540 million in announced write-downs overall. Zillow is a real-estate business and its filing cites several causes, not only the model. If you count non-AI automation, Knight Capital’s loss exceeded $460 million.
Did regulators fine anyone heavily for AI? Not in the cases here. AI-specific penalties were $400,000 (Delphia and Global Predictions) and $2.5 million (Earnest). The larger costs were operating losses and compliance obligations.
Is generative AI covered by US bank model risk guidance? No. SR 26-2 places generative and agentic AI outside its scope. Banks are still expected to manage those tools under their broader risk and governance practices.
Did the EU delay its AI rules for credit scoring? Yes. Regulation (EU) 2026/1744 moved the Annex III high-risk date, which includes credit assessment, to December 2, 2027. It is law, not a proposal.
Is algorithmic discrimination in lending still illegal? Intentional discrimination and use of proxies are still prohibited under ECOA and Regulation B. The CFPB’s 2026 rule removed disparate-impact liability from Regulation B, but state laws and the Fair Housing Act may still apply. Talk to counsel about your footprint.
Was the Apple Card biased? New York’s regulator found no evidence of unlawful discrimination after reviewing data on about 400,000 applicants, though it criticized transparency and customer service.
Are the Nate and K1 cases proven? No. Both are described here as allegations or disputes, and I could not verify a final outcome for either.
Sources and verification notes
Primary and regulator sources
- Zillow Group, Form 8-K (Nov. 2, 2021), 10-Q (Q3 2021), 10-K FY2021, 10-K FY2022, on SEC EDGAR
- SEC press release 2024-36, Delphia and Global Predictions
- Massachusetts Attorney General, Earnest settlement announcement
- New York DFS, Apple Card findings
- OCC, Bulletin 2026-13 and News Release 2026-29
- FSB, final sound practices release
- FINRA, 2026 Annual Regulatory Oversight Report
- Deloitte Center for Financial Services, Deepfake banking fraud risk on the rise
Named secondary sources
- Law-firm analyses: Sullivan & Cromwell, Venable, Greenberg Traurig, Cooley, White & Case, K&L Gates, Goodwin, Nixon Peabody, DLA Piper, Debevoise, Kelley Drye, O’Melveny, WilmerHale (links above)
- News reporting: CNN (Zillow, Arup), Reuters (Knight), Bloomberg and Financial Advisor magazine (K1), Courthouse News (Knight), ABA Banking Journal (FSB), Input (relaying the Wall Street Journal on Zillow’s $881 million)
What I could not verify, so I did not assert
- A published US interagency AI request for information as of October 21, 2026
- An outcome in the Nate prosecution or the K1 litigation
- The Wall Street Journal’s $881 million Zillow figure from a primary document
- The NAIC insurer AI examination pilot and the KPMG and CCAF figures from the earlier version of this article, which were not re-checked in this pass
Limits. Penalty and settlement figures are what parties agreed to, often without admitting findings. Loss figures are as reported by the firms or regulators and are not audited cost-of-failure studies. Cross-case comparisons are directional. This article is general information, not legal advice, and regulatory status should be re-confirmed against primary sources before any compliance filing.