Self-Driving Car Ethics: Why the Trolley Problem Misleads

Confidence key: Established = confirmed by primary sources · Probable = strong evidence with some interpretation · Speculative = my interpretation, open to debate

TL;DR: Self-driving systems don’t decide who lives or dies. They aim to minimize expected harm based on one clear principle: never prioritize one person over another.

  • Self-driving systems don’t decide who lives or dies. They aim to minimize expected harm based on one clear principle: never prioritize one person over another.
  • The 2018 Uber death that started this debate wasn’t a moral dilemma — it was a sensor that misclassified a pedestrian three times while emergency braking sat disabled.
  • Germany’s 2017 Ethics Commission and the EU AI Act both regulate the design table, not the crash instant — and there’s a real, unresolved seam in that logic, worth naming honestly.
  • New 2026 crash data shows real safety gains. It doesn’t prove the trolley scenario itself is solved — that’s a separate, mostly unanswered question.

The death that got misread

In March 2018, a self-driving Uber test vehicle killed pedestrian Elaine Herzberg in Tempe, Arizona — the first fatality involving a fully autonomous car. The public argument that followed was about what the car should have chosen. (Confidence: Established)

The NTSB’s investigation found something far less dramatic. Sensors detected Herzberg roughly six seconds before impact, then misclassified her three times — unknown object, then vehicle, then bicycle. Emergency braking had been disabled because computer-controlled braking made test rides uncomfortably jerky.

There was no dilemma. There was a stack of ordinary, fixable engineering failures. That distinction is the whole article — and it needs no confidence label, because you can check it against the report yourself.

Why the trolley problem doesn’t transfer

Philippa Foot’s 1967 trolley problem runs on certainty: you know exactly who dies under each option, right now, and you alone decide. None of that holds for a car.

A car’s crash behavior is designed months in advance by teams, not chosen mid-swerve by one person. It faces open-ended situations, not two clean options. And critically, it runs on probabilities, not known outcomes — a perception output might read “62% pedestrian,” never “one person, certainly.”

Researchers formalized this directly in a 2021 Philosophy & Technology paper: risk is collision probability times expected harm, and once quantified, it becomes an ordinary cost to minimize — not a dilemma to adjudicate. Run the classic trolley scenario through their framework and the Bayesian term lands on the utilitarian answer — fewer expected deaths wins — while the equality and maximin terms can’t decide at all. (Confidence: Established for the paper’s math; Probable for my framing of its implications)

The moral weight didn’t vanish. It moved from the instant of impact to the constraints built around the whole system, long before any car is on the road.

What 40 million clicks actually proved

MIT’s Moral Machine experiment, published in Nature in 2018, logged roughly 40 million hypothetical crash decisions from 233 countries. People leaned toward sparing more lives and younger people, with enough cultural variance that researchers grouped countries into three distinct clusters. (Confidence: Established — the data; the leap to “use this as a spec” is where it breaks)

It’s not a requirements document, and the authors never claimed it was. Three reasons: a browser click under fake certainty doesn’t tell you how to score real probability estimates; most preferences weren’t cross-cultural majorities, meaning encoding them would mean different moral settings by country — exactly what strong regulation forbids; and nobody who clicked “spare the young” personally paid the cost of demoting everyone older. The study’s real contribution is sociological — moral intuitions differ by culture and are measurable — not prescriptive.

Germany’s fix, and its one honest weak spot

Germany’s Ethics Commission on Automated and Connected Driving, chaired by former constitutional court judge Udo Di Fabio, published 20 guidelines in June 2017. Two matter most: protecting individuals comes before utilitarian trade-offs, and a system may never rank people by age, fitness, or status — full stop.

LayerOwned byMoral functionExample
Constraint (no ranking)Manufacturer, ethics reviewDeontological floorGerman Commission rules
Risk minimizationEngineeringReduce expected harmTrajectory cost functions
Safety caseManufacturer, auditedEvidence of acceptable riskISO 26262 / SOTIF testing
Market authorizationRegulatorGatekeeping before deploymentEU AI Act conformity checks

Here’s the part I won’t smooth over: the Commission’s own report treats an identified victim differently from a statistical one — sacrificing a specific person violates dignity, but designing in advance to minimize total victims, before anyone’s identity resolves, is fine. Whether someone counts as “identified” depends on when the sensors happen to resolve them. Sensor timing isn’t a moral distinction, and ethicists flagged the tension contemporaneously — Christoph Lütge’s 2017 analysis of the report notes exactly this asymmetry. (Confidence: Established that the tension exists in the report and the literature; Speculative that it’s the report’s weakest joint) The honest reading: it’s a stipulated rule, defensible because it’s simple enough to audit, not a theorem derived from first principles.

What the 2026 crash data actually shows

One scope note before the numbers, because it belongs next to the numbers: only Waymo publishes mileage data at this scale, so everything in this section is Waymo-specific. That’s a real limitation, not a footnote — other operators may look better or worse, and we simply can’t say. (Confidence: Established — this is a fact about data availability)

In July 2026, IIHS published Rise of the Machines, comparing ~50 million driverless Waymo miles against ~222 billion human-driver miles in the same four cities. Waymo had 68% fewer police-reportable crashes, 85% fewer single-vehicle crashes, 81% fewer injury crashes. Results varied by city — Phoenix and Los Angeles saw the biggest gains, Austin actually ran slightly higher: about 3% per the peer-reviewed paper, 4% per IIHS’s news release. Researchers attribute that to a small sample. That could be noise. It hasn’t been shown to be noise. (Confidence: Established for the topline numbers; Probable for the Austin explanation)

IIHS’s own caveat matters as much as the headline: only about 22% of reported automation crashes qualified as police-reportable in a human-comparable sense, and the study flags federal crash-reporting infrastructure as inadequate for monitoring a larger rollout. Take the safety finding and the warning together, or take neither. Note too what this data doesn’t show: it tracks overall crash rates, not how often true forced-choice dilemmas occur. Those appear rare enough in the record that they barely register as their own category.

What’s still genuinely unresolved

Three problems don’t dissolve into risk math, and pretending otherwise would be dishonest:

  • Risk moves, it doesn’t vanish. A car that brakes hard and accepts a rear-end hit to protect a pedestrian shifts risk onto the driver behind — who never agreed to that trade.
  • Remote oversight is thinner than the name suggests. When Cruise was still operating, data reported by the New York Times showed a remote intervention every 2.5–5 miles and 1.5 support staff per vehicle at its peak; Cruise said the true ratio was one remote-assist worker per 15–20 cars, with 98% of calls connected within three seconds and 80% resolved without the human acting. Waymo’s own 2026 description is blunter: its remote assistance “does not continuously monitor” and offers “advice, not control” — and the car may ignore the advice. No regulation anywhere specifies safe supervision ratios. What counts as adequate human oversight is currently whatever the company says it is.
  • The public still believes the wrong problem is the live one. Every “robot chooses who dies” headline builds demand for the exact life-ranking module every serious framework agrees should never exist.

None of these have a clean fix. They’re questions about power, consent, and communication — not inputs to a cost function.

The lever nobody has to build

The trolley problem is solved in the only sense that matters: nobody has to build the lever. What’s left isn’t in the car. It’s who absorbs the risk it redistributes, who watches the watchers, and whether a decade of mythology gets corrected before it pressures someone into building the module everyone agreed should never exist.

FAQ

Does a self-driving car ever choose who to hit?
No regulatory framework or major manufacturer’s published approach builds that module. Systems minimize expected harm under constraints that forbid ranking people by personal traits.

What actually caused the 2018 Uber fatality?
Sensor misclassification and disabled emergency braking, per the NTSB — not a forced ethical choice.

Does the EU AI Act require an ethics decision module in self-driving cars?
No. It requires risk-management documentation, logging, and human-oversight design — governance evidence, not a moral algorithm.

Are self-driving cars safer than human drivers right now?
IIHS’s 2026 Waymo data shows large reductions in police-reportable crashes in the cities studied, with Austin as an unresolved outlier. And it’s Waymo-specific data — the honest answer for self-driving cars generally is “we know about one fleet.”

Who is legally liable when a driverless car injures someone?
Genuinely unsettled, and this piece doesn’t pretend otherwise. Liability regimes vary by jurisdiction and are still being litigated into shape; the trend points toward manufacturer responsibility, but no coherent framework exists yet.

Does the safety data prove the cars are safer for pedestrians and cyclists?
Not directly. IIHS measured crash and injury rates overall, not disaggregated by who was hurt. Given how much of AV ethics concerns protecting vulnerable road users, that disaggregation is the study someone should run next.


About the author: Tom Morgan covers AI policy and product ethics for FutureNow. This piece draws on primary regulatory and crash-safety sources.