Why AI Detector Accuracy Has an Expiration Date

Quick answer

Here's the uncomfortable part about that headline accuracy number: a detector that nails 99% this month can fall apart next month without a single line of its code changing. NeurIPS 2026 test-time research makes that point with two numbers pulled from the same commercial tool, Pangram. Against ordinary AI-generated text, it caught 96.0% of it. Once researchers deliberately reworded that same text to dodge detection, the catch rate collapsed to 27.0%. A second gap opened the moment OpenAI shipped GPT 5.4 in March 2026, when Pangram's recall on that model's fresh output sank to just 34%, even while it held near-perfect scores on older models it already knew. Recall only climbed back after a May 2026 update closed the gap. Neither drop means Pangram is a weak detector. What both show is that an accuracy figure is a snapshot of one moment, and every detector shares the same underlying blind spot: whatever kind of writing it hasn't met yet.

The three ways a detector's accuracy decays

What ties the whole problem together is a paper built around one plain observation. Cornell Tech's Kevin Ren and Nikhil Garg, working with MIT's Manish Raghavan, framed their NeurIPS 2026 paper, "Hitting a Moving Target," around the fact that every deployed AI-text detector gets trained once, on one fixed batch of labeled human and AI writing, then has to keep working in a world that doesn't hold still. Their contribution is naming three distinct pressures that each pull a detector's real accuracy away from its lab-tested number, on three separate timelines.

The first is adversarial humanization. Paraphrasing tools and "humanizer" services exist specifically to rewrite AI output until it no longer matches the patterns a detector learned to flag. This is a moving target by design: as detectors adapt, humanizers adapt back. The paper's own test is blunt about the stakes. Against text nobody tried to disguise, Pangram detected 96.0%. Against text the researchers deliberately engineered to evade it, that fell to 27.0%.

The second is new model releases. A detector trained on GPT-4 or GPT-4o output has learned the statistical fingerprints of those specific models: their word choices, their sentence rhythms, their habits. A brand-new model doesn't automatically share those fingerprints. When GPT 5.4 shipped in March 2026, the paper found Pangram flagged only 34% of its output as AI or AI-assisted, a steep drop from the near-perfect detection it still showed on older models that same week. Pangram closed most of that gap with an update two months later, but for that stretch, a detector widely described as one of the most accurate on the market was missing two out of every three documents from the newest major model.

The third is the slowest-moving and the hardest to see coming: human writing itself is changing. The researchers tested a classifier trained on 2010-era human text against writing from 2020, years before ChatGPT existed, and found it already misclassified 15.1% of 2020 science-abstract sentences as AI-generated. Nothing in those 2020 abstracts was machine-written. The classifier's idea of "how humans write" had simply gone stale, because writing style shifts over a decade regardless of AI. Every year that passes puts more distance between a detector's human-writing baseline and how people actually write now, especially once some of what people read and absorb is itself AI-influenced.

When "the wrong model" turned into an accidental demonstration

What makes this case stand out is that the mistake itself ended up proving the paper's point. In June 2026, a team at Vrije Universiteit Brussel — Van Vlasselaer, Van Droogenbroeck, and Spruyt, writing in the International Journal for Educational Integrity — ran four detectors, Turnitin, Copyleaks, GPTZero, and Pangram, against text they believed had come from GPT-4o. Three of the four numbers that came back were startling: Turnitin, Copyleaks, and GPTZero each scored 0% on the fully AI-written category. Not a low number. Zero.

The explanation arrived months later. A correction the same journal ran on October 6, 2026 revealed that the testing interface had labeled the model "GPT-4o," but the text had actually come from ChatGPT's Deep Research mode, running on an o3-based model underneath that label. So the three zero-scoring detectors hadn't failed at detecting AI writing broadly; they'd simply never trained on output from that specific underlying model, and had no way to recognize it. Once the correction clarified what had really produced the text, the zero scores stopped looking like a detection failure and started looking exactly like the blind spot the Ren/Raghavan/Garg paper describes: a detector only knows the last model it trained against, and a familiar label on an interface is no guarantee it's the model the detector was actually built to catch.

Only Pangram, the one detector in the study trained with what researchers call "mirror" and hard-negative-mining methods — deliberately training on new model outputs mapped against matched human text, rather than a fixed historical dataset — caught a meaningful share of the Deep Research output. That's not a coincidence. It's the practical difference between a detector that treats its training data as a one-time snapshot and one built with the expectation that new models will keep arriving. The same four-detector study also found a sharper version of this problem on mixed, AI-edited text — see our piece on whether detectors can tell editing from writing for the full breakdown.

What a decaying accuracy number means for how you read a score

None of this means detection doesn't work. Both studies found that, against writing style it was actually trained on, a modern detector can get extremely close to perfect — 99%-plus accuracy isn't a fiction, it's a real, achievable number under the right conditions. The problem is treating that number as permanent. A detector's accuracy is tied to a training snapshot, and every month that passes without retraining, new models ship, paraphrasing tools improve, and human writing itself keeps drifting, all of which quietly erode the number a marketing page quoted six months ago.

That has two practical consequences. First, a single detection score on a brand-new or heavily rewritten piece of text deserves more scrutiny than one on ordinary, unremarkable writing, precisely because novel and adversarial text is where every detector's accuracy degrades fastest and least predictably. Second, "which detector is most accurate" isn't a question with a permanent answer. It's a question with a timestamp.

A detector maker's response to that reality matters more than any single accuracy claim. Ask whether a tool is retrained against current models on a defined cadence, whether it's been tested against text generated after its last update, and whether its published numbers come with a date attached. A detector score is a useful signal about a specific piece of text, checked against current models, read alongside context, and never the final word on its own. Try a passage through TheChecker.AI's free demo to see what a current, sentence-level read looks like rather than relying on a headline percentage that may be describing last year's models.

FAQ

Does this mean AI detectors are unreliable and not worth using? Not from what either study found. Both show near-perfect accuracy once a detector is tested against writing that resembles what it was actually trained on — the limitation is narrower than "detectors don't work." An accuracy number simply carries a shelf life, and it erodes fastest right where you'd expect: brand-new models, paraphrased text, and writing published after a detector's last update, not ordinary text from models it already knows well.

How often do detectors need retraining to stay accurate? There's no fixed industry schedule for this. What the research does show is a clear risk window: right after a major release, like GPT 5.4 in March 2026, a detector's recall can fall sharply and stay down until the detector gets updated against the new model's output. In that case recovery took roughly two months.

If a detector misses one document, does that mean the detector is broken? Not necessarily. A single miss can mean the text came from a newer model, was lightly paraphrased, or simply falls outside what that detector was trained on — not that the detector has stopped working in general. For a related angle on why even a 99%-accurate tool still misses paraphrased text at a meaningful rate, see our piece on detector recall under paraphrasing.

Can a detector that was accurate yesterday be wrong today? Yes, without the detector itself changing at all — purely because the world it's scoring changed. A new model release, a new paraphrasing tool, or simply writing style drifting over time can all shift what "accurate" means for the exact same, unmodified detector. See our explainer on why a single detector score shouldn't decide a case for how to weigh a score responsibly given that uncertainty.