AI plagiarism detectors misfire on genuine human writing because their core signal — text perplexity — penalizes simple, predictable sentence structure, which is common in non-native English and plain prose. Peer-reviewed testing has found false-positive rates as high as 61% in some writer populations, even as vendors advertise under 1%.
TL;DR
- Detectors don’t read for meaning. They score how statistically predictable your word choices are, and predictable writing looks “AI-like” regardless of who wrote it.
- The most-cited independent study (Liang et al., Stanford, 2023) found a 61.3% average false-positive rate across seven detectors on non-native-English essays, against a near-zero rate for native-speaker control essays.
- Turnitin’s own published figure — under 1% false positives on qualifying documents — describes a specific, narrow test condition, not a universal accuracy rate.
- More than 50 universities had disabled or restricted AI-writing detection by mid-2026, and a January 2026 New York court case turned a false-positive dispute into the first detailed legal precedent in the space.
- The fix isn’t a better score. It’s treating a detector output as one input into an investigation, never as the investigation’s conclusion.
A zero-grade, an expulsion threat, and a paper about Christianity and Islam
In the fall of 2024, Orion Newby, a first-year student at Adelphi University, submitted a paper on Christianity and Islam for a World Civilizations course. He wrote it with support from Adelphi’s Bridges program, which assists students with documented learning and neurological differences. His professor ran the paper through Turnitin. The tool returned an AI-generated score of 100%.
Newby denied using generative AI and said he’d used Grammarly for grammar help. He received a zero and an academic integrity violation. A second flagged assignment, he was told, could mean expulsion. His family ran the same paper through two other detectors — Grammarly’s own checker and ZeroGPT — and both classified it as human-written. Adelphi upheld the finding anyway. The same administrator who made the original determination also decided the appeal.
The family hired attorney Mark Lesko and sued. On January 28, 2026, Justice Randy Sue Marber of the New York State Supreme Court, Nassau County, granted the family’s Article 78 petition, annulled the finding, and ordered Adelphi to expunge Newby’s record and restore his standing. The family reportedly spent more than $100,000 in legal fees to get there.
Matter of Newby v. Adelphi University is the case that made AI-detection false positives a legal-liability question rather than a classroom dispute. It’s worth understanding exactly what the court did and didn’t decide — because most coverage of it gets that part wrong, and the distinction matters for anyone relying on a detector score today. We’ll come back to it. First, the more useful question: why did a paper Newby says he wrote himself score 100% AI-generated in the first place?
What a detector is actually measuring
No AI detector reads for authorship, intent, or meaning. Every mainstream text classifier — Turnitin’s, GPTZero’s, Originality.ai’s, and the academic tools tested in peer-reviewed studies — scores some version of two statistical properties of the text itself: perplexity and burstiness.
- Perplexity measures how “surprised” a language model is by your next word choice. Predictable, high-probability word sequences score low perplexity. Unusual, idiosyncratic phrasing scores high perplexity.
- Burstiness measures variation in sentence length and structure across a document. Human writing tends to alternate between short and long, simple and complex sentences. Machine-generated text (and any writing that’s deliberately plain) tends to be more uniform.
Large language models, trained to produce the statistically most likely continuation of a prompt, generate text that is naturally low-perplexity and low-burstiness. That’s the signal detectors were built to catch. The problem is that plenty of human writing shares those same statistical properties — and not randomly. It clusters in specific populations.
That overlap — between “written by a language model” and “written simply, by a human, in a non-native language or a plain register” — is the entire false-positive problem in one sentence. Liang et al.’s original 2023 paper found that TOEFL essays unanimously flagged by all seven detectors tested had measurably lower perplexity than essays that weren’t flagged — evidence pointing directly at the mechanism, not just the outcome.
Who actually gets flagged
The populations affected by this aren’t random. They cluster around anyone whose writing is naturally lower in perplexity and burstiness for reasons that have nothing to do with AI use:
- Non-native English writers, per Liang et al. — simpler syntax and more repeated phrasing are common features of second-language academic writing, and they overlap heavily with the statistical signature detectors are built to catch.
- Writers with documented learning or neurological differences, as in the Newby case — tutoring-supported writing can shift toward more regularized sentence structure without any AI involvement.
- Formulaic academic genres. A forensic-linguistics review of detection failures found that literature-review sections in particular are prone to false positives, because their formal, convention-bound style resembles LLM output more than other academic prose does; the same review cites medical-informatics research finding false-positive rates exceeding 8% in some journal types on fully human-written manuscripts.
- Anyone using grammar-correction tools. Originality.ai’s own documentation acknowledges that AI-assisted editing tools — Grammarly’s rewrite features among them — can trigger false positives, which is directly relevant to the Newby case, since Grammarly was the one tool he says he actually used.
None of this means detectors are useless. It means their error rate is not evenly distributed, and a score that looks precise (a single percentage) is masking a much less precise reality underneath.
The other half of the problem: real AI text can walk through undetected
A false-positive rate this high would be easier to accept as a necessary trade-off if the tools were at least reliably catching actual AI-generated text. They aren’t, consistently. Liang et al.’s own experiment found that applying a simple second-round instruction to ChatGPT — asking it to “elevate” the text with more literary language — dropped detection rates from 100% to 13% across the same seven detectors, by artificially raising the perplexity of machine-generated text back into the human range. Vendors have since built countermeasures aimed at these “bypasser” techniques — Turnitin updated its model in August 2025 specifically to catch paraphrased and AI-polished submissions — but the arms-race dynamic means any fixed detection threshold is a moving target, not a settled fact. A 2025 evaluation of AI-polished text found detection rates for minimally edited machine text ranging from 10% to 75% depending on which of twelve detectors was used, with detectors also struggling to distinguish light polishing from heavy rewriting.
The practical implication is uncomfortable but important: a low or “human” score is weak evidence of human authorship for the same statistical reasons a high score is weak evidence of AI authorship. Both errors run through the same perplexity-based mechanism, in opposite directions.
The evidence, ranked by what it actually shows
Accuracy claims for AI detectors vary by an order of magnitude depending on who ran the test, on what text, and under what conditions. Ranking the evidence by type — rather than repeating whichever number is loudest — is the only honest way through it.
| Source | Evidence type | What it found |
|---|---|---|
| Liang, Yuksekgonul, Mao, Wu & Zou, Patterns (2023), Stanford | Peer-reviewed, independent | 61.3% mean false-positive rate across 7 detectors on 91 non-native-authored TOEFL essays; near-perfect accuracy on native-speaker control essays; all 7 detectors unanimously misflagged 19.8% of the TOEFL set |
| Weber-Wulff et al. (2023) | Peer-reviewed, independent, 14 tools tested | None of the 14 tools exceeded 80% overall accuracy; only 5 exceeded 70% |
| GPTZero, self-reported response (Oct. 2023) | Vendor claim, post-criticism update | Updated model still showed a 7.7% false-positive rate on the same TOEFL benchmark (1.1% if “possible AI content” is counted as a negative result) |
| Pangram Labs technical report | Independent-style vendor benchmark, reproducible methodology disclosed | 0% false positives on the TOEFL benchmark; 0% on 3,907 ESL essays (ELLIPSE dataset); 0.09% on 5,600 essays (ICNALE dataset) |
| Turnitin, published FAQ | Vendor claim | ~98% accuracy, under 1% false positives — specifically on documents where more than 20% of the text is flagged, validated against 700,000 pre-ChatGPT academic papers |
| Van Vlasselaer et al., International Journal for Educational Integrity (June 2026) | Peer-reviewed, independent | Zero false positives on fully human-written papers, but a 0% detection rate on papers that were 100% AI-generated in the same test set |
Two things follow from putting these side by side. First, Turnitin’s headline figure is real but narrow: it describes performance on a specific document type (more than 20% flagged content) validated against a specific historical corpus, not a general false-positive rate across all writing. Turnitin itself keeps scores in the 1–19% range hidden from instructors behind an asterisk, stating internally that the confidence in that band is too low to publish as a number — which is a tacit admission that the tool’s reliability is not uniform across its own scoring range.
Second, the Pangram result matters more than it might look. It’s not proof that non-native-writer bias is unfixable — Pangram’s own published methodology reports zero false positives on exactly the TOEFL benchmark that produced Liang et al.’s 61.3% figure across other tools. That’s a genuine information gap most coverage of this topic misses: the bias documented in 2023 is a property of specific detector architectures, not an unavoidable law of how perplexity-based detection has to work. Whether Pangram’s own numbers hold up under third-party replication at the same scale as the Weber-Wulff test is, as of this writing, not independently confirmed — which is itself worth flagging rather than treating a single vendor’s self-reported benchmark as settled fact.
The Van Vlasselaer finding is the odd one out, and it’s worth sitting with rather than smoothing over: a detector tuned hard enough to avoid false-flagging human writing can end up missing fully AI-generated text entirely. That’s consistent with reports that Turnitin deliberately trades detection sensitivity for a lower false-positive rate — the tool is asymmetrically tuned toward not accusing a human, at the cost of letting more AI-generated text through undetected. That’s a reasonable trade-off for a tool with disciplinary consequences attached. It also means “the AI detector didn’t flag it” is weaker evidence of human authorship than most instructors currently assume.
The institutional pivot: 2023 to 2026
The response to this evidence has moved in one direction for three years, and it accelerated sharply through 2025 and into 2026.
By mid-2026, a tracker of institutional policy changes counted more than 50 universities that had disabled or materially restricted AI-writing detection, while continuing to use traditional text-matching (plagiarism) checks, which rest on a fundamentally different and more verifiable mechanism — matching submitted text against a database of existing documents, rather than statistically inferring authorship. That distinction is the one piece of nuance worth holding onto: dropping AI-detection scoring is not the same as dropping plagiarism detection, and coverage that conflates the two is misleading readers about what these schools actually did.
The specific, sourced moves: the University of Waterloo disabled Turnitin’s AI-writing indicator in September 2025 after internal testing reportedly returned a 100%-AI score on text the university confirmed was human-written. Johns Hopkins moved to an advisory-only policy in early 2025, where a detector result can open a conversation with a student but cannot by itself support a formal charge. The University of Cape Town discontinued AI detection from October 2025 as part of a broader shift toward evaluating the writing process rather than policing the final product. Australian Catholic University, which logged roughly 6,000 alleged AI-cheating cases in 2024 — about 90% of all its academic integrity referrals that year, with roughly a quarter dismissed after investigation — disabled the tool in March 2025. Curtin University confirmed it would disable Turnitin’s AI-writing feature across all campuses from January 1, 2026, while keeping originality/text-matching checks active, framing the decision around “trust and clarity” in assessment.
What the Newby ruling actually decided
Here’s the correction most secondary coverage of Newby v. Adelphi skips: the court did not rule that Turnitin’s detector is inaccurate, and it did not make a factual finding about whether Newby’s paper was AI-generated. Justice Marber’s decision turned on process, not on detector science. The opinion focused on the university’s conduct — specifically that the same administrator who issued the original finding also presided over Newby’s appeal, and that the disciplinary process proceeded on the strength of a single automated score despite Newby’s denial, his tutoring-program documentation, and two other detectors classifying the same paper as human-written.
That’s a narrower ruling than “a court proved AI detectors don’t work,” but it’s arguably more consequential for how institutions should actually behave: it establishes that treating a single detector score as sufficient grounds for a disciplinary finding — without independent review, without weighing conflicting evidence, without separating the fact-finder from the appeals officer — can itself be an Article 78 “arbitrary and capricious” due-process failure, independent of whether the underlying score was right or wrong. Institutions that already require corroborating evidence before acting on a detector flag were, in effect, already compliant with the standard this case sets. Institutions that treat a score as self-sufficient evidence now have a specific, citable precedent working against that practice.
A working framework, not a verdict
The single biggest failure mode across every case surveyed here — Newby, the UC Davis and Grand Valley State cases documented earlier, the roughly 25% of ACU’s cases dismissed on review — is the same one: a detector score was treated as a conclusion instead of a lead.
For institutions and instructors:
- Use a detector score as a trigger for further review, never as the sole basis for a finding — this is also Turnitin’s own stated position, and it’s now backed by legal precedent, not just vendor caution.
- Require corroborating evidence before any disciplinary action: draft/revision history (Google Docs version history, Word track-changes metadata, cited research notes), a conversation with the student, or comparison against that student’s earlier, verified writing samples.
- Separate the person who makes the original finding from the person who hears the appeal. This single structural fix addresses the specific defect the Newby court identified.
- Treat AI-writing scores and plagiarism (text-matching) scores as different tools with different reliability profiles — several of the institutions above kept one while dropping the other, which is a defensible middle position rather than an all-or-nothing choice.
For writers, especially non-native English speakers and students:
- Keep draft history. It is, case after case, the evidence that actually moved outcomes — not a persuasive denial, but a Google Docs edit trail or a saved earlier draft.
- Understand that plain, direct, low-variance sentence structure is exactly what these tools are statistically primed to flag — this isn’t a personal failing in your writing, it’s a known property of how perplexity-based scoring works.
- If flagged, ask specifically what evidence beyond the score was used, and whether the reviewing party is the same person who will hear an appeal.
What’s still unresolved
No major detector vendor publishes a regularly updated, independently audited accuracy benchmark broken out by writer population, alongside its marketing accuracy figure. That gap — not a single new number — is the actual open question here. Turnitin’s 98%/under-1% figure and Pangram’s 0%-on-TOEFL figure are both real, both narrowly true, and both measured under conditions the companies themselves chose. Until a third party runs a standardized, population-stratified test across multiple current detectors on a recurring basis — the way Weber-Wulff did once, in 2023, for 14 tools — every accuracy claim in this space, including the ones in this article, should be read as conditional on the specific test that produced it.
Suggested title tag: AI Plagiarism Detection Accuracy: Why False Positives Flag Original Writing (2026 Guide) Suggested meta description: Peer-reviewed evidence on AI-detector false positives, the mechanism behind them, and the 2026 Newby v. Adelphi ruling — plus what educators and writers should actually do. Suggested URL slug: /ai-plagiarism-detection-false-positives/
External sources referenced:
- Liang, W. et al., “GPT detectors are biased against non-native English writers,” Patterns, 2023 — https://pubmed.ncbi.nlm.nih.gov/37521038/
- Pangram Labs Technical Report — https://arxiv.org/pdf/2402.14873
- CASRAI, “AI Detector Accuracy: The False-Positive Evidence” — https://casrai.org/guides/ai-detection-accuracy-higher-education
- CASRAI, “AI Detector False Positives: 2026 Update” — https://casrai.org/news/ai-detector-false-positive-controversies-2026
- Insider Higher Ed, “Adelphi Student Wins AI Plagiarism Lawsuit” — https://www.insidehighered.com/news/quick-takes/2026/02/11/adelphi-student-wins-ai-plagiarism-lawsuit
- Is It AI, “Newby v Adelphi: What a Court Needed Beyond a 100% AI Score” — https://isitai.co.uk/blog/what-a-court-needed-beyond-a-100-percent-ai-score