Detectors Punish Honest AI Editing More Than They Catch Actual Cheating. Here's the Study.

Quick answer

A new Notre Dame working paper, presented at the ACM AI Leadership Summit, found something backwards: a student who honestly uses AI to polish their own writing gets flagged 64 to 80 percent of the time, while a student who runs a fully AI-written paper through a humanizer tool gets caught less than 4 percent of the time. The researchers' own conclusion, stated plainly: "Honest AI-editing results in a higher sanction risk than humanizer-assisted evasion." That's not a minor calibration issue. It's the exact opposite of what a detection policy is supposed to reward.

What the study actually tested

Jonathan Karr Jr., Grigorii Khvatskii, Ting Hua, and Nitesh V. Chawla, all at the University of Notre Dame, built a controlled pipeline to isolate a question most detector coverage skips: not "does the detector work," but "does it treat degrees of AI involvement fairly." They pulled 642 published English-language abstracts from OpenAlex across four fields — chemistry, computer science, political science, and theology — split between a pre-ChatGPT window (2013 to 2015) and a recent window (2023 to 2025).

Each original abstract was run through three separate AI rewrite conditions using Gemini: a light edit that only touched the abstract text, a fuller edit that used the whole article for context, and a complete AI-written rewrite from scratch. Every version — original and all three AI variants — was then scored by two commercial detectors, Pangram (version 3.2) and GPTZero, at the standard 0.50 threshold vendors themselves use.

The numbers that don't add up the way policy assumes

Then came the second half of the design: every AI-generated variant went through Undetectable AI, a commercial humanizer, before getting rescored.

Nobody could have used AI to write the pre-ChatGPT abstracts, so that group works as a clean baseline. Both detectors returned a 0% false-positive rate there, exactly what you'd want.

Light AI editing changed everything. Lightly-edited abstracts from 2013-15 got flagged 64.4% of the time by Pangram and 37.6% of the time by GPTZero. Push the same test into 2023-25 and the numbers climb further, 80.1% for Pangram, 48.5% for GPTZero. Remember what these documents actually are: a person wrote the real academic argument, then asked AI to tighten a few sentences, which is exactly the kind of AI use most university policies say is fine.

Flip to the fully AI-generated variants once the humanizer touched them, and the story reverses completely. Detection fell below 4% on both tools, an FNR over 96%, so text an AI wrote start to finish got waved through as human more often than not, nearly every time. Pangram's own average score on that humanized text landed at 0.03 to 0.04, squarely inside "clearly human" territory.

Line the two results up and the gap is jarring: a student honestly disclosing their AI editing gets caught eight to twenty times more often than someone who ran a full AI draft through a ten-dollar humanizer.

Why this happens

The paper's linguistic analysis explains the mechanism, and it isn't a bug so much as a byproduct of what these detectors actually measure. Both Pangram and GPTZero scores correlate with vocabulary features like long-token ratio and Academic Word List density (Spearman ρ around 0.30 to 0.35, p < 0.001) — in plain terms, the more formal and precise the vocabulary, the higher the AI score, regardless of who wrote it. AI editing tends to push text toward exactly that kind of polished, word-list-heavy phrasing, which is also what strong academic writing already looks like. Non-STEM prose (political science, theology) showed flag rates far above STEM prose (chemistry, computer science) at p < 0.001, because dense technical and numeric writing doesn't trigger those same vocabulary cues either way.

Humanizer tools work by doing the opposite: they specifically strip out the long-token and academic-vocabulary patterns detectors have learned to associate with AI, while adding back the irregular sentence rhythm that reads as "human." The paper found long-token ratio, word-list density, and lexical diversity all fell sharply after humanization (p < 0.001 across every measure) — the humanizer isn't making AI text more honest, it's specifically sanding off the features detectors check for.

WIRED's September 2026 profile of Pangram independently confirmed the Notre Dame numbers directly from the paper: light AI editing flagged 64 to 80 percent of the time, dropping to under 4 percent once a humanizer touched the text. The same profile also surfaced a related but separate miss rate, this one from Pangram's own disclosed data rather than the Notre Dame study: human essays heavily rewritten by AI still get called human-written 41.37 percent of the time. Different test, same direction — detectors miss edited text far more than their headline accuracy numbers suggest, whichever side of "edited" you're looking at.

The policy problem this creates

The researchers frame this bluntly as a catch-22 for any institution that deploys a detector with a fixed threshold: enforce strictly and you punish students and multilingual writers who used AI exactly the way the syllabus told them to, or loosen the threshold and you let genuinely AI-written, humanizer-evaded submissions through untouched. There's no single number that avoids both failure modes at once — the paper's own threshold sweep (0.40 to 0.60) shows raising the cutoff barely moves the humanizer-evasion miss rate while still catching most honest light edits.

Their recommendation isn't "stop using detectors." It's that a flag should never be standalone evidence. Detector output has to be paired with something else — drafting history, version records, a conversation with the student — before any sanction, precisely because the tool can't currently tell "this person followed the AI policy" from "this person's writing happens to sound thorough" and reliably distinguishes neither from a humanized full AI draft.

What this means if you're on either side of a flag

If you're an educator or an integrity office: a single score crossing a threshold is not proof of anything specific about degree of AI involvement, and this study is a direct empirical reason why, not just a hedge. The same brand of asymmetry shows up in our own coverage of what a detection score is actually measuring — a probability about surface features, not a verdict about authorship.

If you edited your own writing with AI and got flagged: this research is the closest thing to independent confirmation that light, disclosed editing is one of the hardest categories for current detectors to score fairly — see our companion piece on detecting the difference between writing with AI and editing with AI for what a mixed-authorship score can and can't tell you.

If you're weighing whether "humanizing" text is worth the risk: don't read this study as a green light. It documents what happens to a detector's score, not what happens when an instructor or editor personally suspects something and looks closer — see our full breakdown of whether humanizing actually beats detection for the fuller picture, including why humanized text tends to read worse.

FAQ

Does this mean AI detectors don't work? AI detectors are not perfect. They did not fail completely. In a recent study, 100% of ordinary human writing went through unscathed. When text was entirely generated by AI before any editing, the detectors caught it reliably. Still, every system has a predictable flaw. The shortcoming lies in the scoring of text that mixes human and AI content. A humanizer can exploit this weakness. Even sincere editing cannot avoid that limitation.

Which detectors were tested? Pangram (version 3.2) underwent testing, as did GPTZero. The team also employed GPT-5 Nano to supplement the baseline. They presented real comparative data from the direct testing. The work does not claim one tool is superior. Both Pangram and GPTZero share a particular vulnerability, according to the findings.

What should a school or publication actually do with this? Treat a detector flag as a single input, not as proof of wrongdoing on its own — this is the paper's main recommendation. Before issuing a penalty, the flag must be weighed against drafting history or other supporting evidence. The study finds that flag rates stay elevated in non-STEM writing, regardless of policy.

See where your own edited draft actually lands

Every AI detector faces this same light-edit-versus-full-rewrite blind spot, including ours. Run your own AI-assisted draft through our detector and see the segment-level breakdown instead of guessing from one number, or read how we score submissions before trusting any single percentage.