Two Professors Proved AI Writing Without a Detector

Quick answer

Two Dartmouth professors just showed what a close read can prove that a detector score can't. When Provost Santiago Schnell's Washington Post column scored 100% AI-written on Pangram, the resulting fight became mostly about whether the detector was right. Professors Michael Herron and Andrew Levin tried a different test: they read the column itself, checked its citations against the actual studies it referenced, and found specific content errors that are common in AI-generated text and hard to explain in a piece written by a genuine subject-matter expert. A detector describes a statistical pattern in finished text. A content read checks whether the claims inside that text hold up. They're different tools, and the Dartmouth case shows what each one catches that the other misses.

The column, the score, and the argument that started

In August 2026, Schnell published a Washington Post guest essay arguing that universities need to separate what students can do unaided from what they can only do with AI's help. Weeks later, Semafor reported that the AI detector Pangram scored the essay, along with nine other guest columns published in major outlets that month, as mostly or fully AI-generated. The Dartmouth, the student newspaper, ran its own check: Pangram 4.0 put Schnell's column at 100% AI-written and found a median score of 96% across nine of his pieces published in 2026, against a clean 0% on five papers he wrote before ChatGPT existed. We covered the detector-score side of this story in detail here, including the part where Schnell's own emailed explanation of his writing process also scored 100% AI.

Schnell didn't deny using AI. He said he developed the arguments and research himself and used ChatGPT only for "language refinement, copyediting, improving clarity and organization." That's a real, common distinction, and it's exactly the kind of dispute a percentage score can't settle on its own: a detector measures the finished prose, not the process that produced it.

What two professors did instead of trusting the score

Herron, a professor of quantitative social science, and Levin, a professor of economics, wrote in a guest column for The Dartmouth that "definitive conclusions on this issue cannot hinge on proprietary software tools." So they set the detector score aside and read the column the way a peer reviewer reads a manuscript: checking every factual claim against its actual source. What they found wasn't a stylistic tell. It was a pattern of specific, checkable errors.

A citation that didn't match the study it cited. Schnell's column described a study by researcher André Barcaui as "a randomized study of 120 students." The number 120 does appear in that study's abstract, as the size of the group originally recruited. But the paper's own results section makes clear that only 85 of those 120 participants actually completed the retention test the study's conclusions are built on, and the reported statistics (including a reported t-value with 83 degrees of freedom) only make sense against that smaller number. Citing the abstract's headline number instead of the number the actual results rest on is a known AI failure mode: language models tend to weight an article's opening summary heavily and skim past the methodology deeper in the text, a pattern researchers call "lost in the middle." Herron and Levin argued a genuine expert working with that paper would have caught the gap between the recruited sample and the completed sample immediately.

A missing caveat the study's own author flagged. Barcaui's paper explicitly notes that its participants were "recruited through convenience sampling from a large Brazilian university" — a limitation on how far the results generalize. Schnell's column repeats the study's headline finding with no mention of that caveat. Leaving out an author's own stated limitation isn't a style choice; it's the kind of detail an AI summary skips unless specifically asked to include it.

An outdated literature review. Schnell's column cited research from 2025 on AI and student learning but made no mention of a working paper posted in December 2025 or a peer-reviewed article published in April 2026 — both directly relevant and both already available by the time the column ran. Missing older results can happen to anyone. Missing more recent, more relevant work on the exact topic you're writing about is a pattern more consistent with a tool generating summaries from older training data than an expert who tracks their own field.

Uncredited paraphrase of Dartmouth's own press release. The column's one specific example of AI use on campus — comparing a student's summary of a text against an AI-generated one — closely paraphrases language from a Dartmouth news release about the same program, without citing it. A human author describing their own institution's program would typically credit that institution's own announcement. An AI assistant pulling from source material it was fed, without being told to attribute it, wouldn't.

A logical jump with no connecting argument. The professors pointed to a passage where Schnell describes shifting from thinking about AI as a cheating problem to thinking about it as a "capability revelation" question, with no explanation connecting the two ideas. They called it close to a non sequitur: the kind of smooth-sounding but logically disconnected transition that shows up when a model strings plausible sentences together without actually working through the argument between them.

None of these five findings depends on a detector at all. Every one of them is checkable by anyone willing to pull up the cited sources and read them.

Why this matters more than the detector score did

A detection score is a statistical judgment about surface patterns: word choice, sentence rhythm, predictability. It's useful precisely because it doesn't require a human to read every word of a disputed document before flagging it for a closer look — which is exactly the role we design our own detector to play. But a score alone can't tell you whether the argument inside the text actually holds up, and it can't catch an error that would be obvious to any domain expert but invisible to a model trained on surface-level patterns.

Herron and Levin's method catches a different, arguably more damning class of problem: content that reads smoothly but gets the underlying facts wrong in ways a genuine expert in the field almost certainly wouldn't. We've written before about the gap between detecting that AI touched a document and verifying that the claims inside it are true — Wiki Education found the same split when it checked AI-flagged Wikipedia edits and discovered most failures weren't fake citations, they were real citations that didn't actually say what the article claimed. The Dartmouth case is the same lesson from the opposite direction: a score said "AI," and only a line-by-line content check could say specifically why that mattered.

What this means if you're the one being accused, or the one accusing

If you're disputing a flag against your own writing, a detector score invites one kind of pushback: "the tool is wrong." A content-level critique invites a different, harder one: "here's the specific factual error, and here's the source that proves it." The second is much harder to wave away, because it doesn't ask anyone to trust a black-box percentage — it asks them to check a citation themselves. A single number has never been enough to settle a disputed case on its own, in a classroom or a courtroom. What ends disputes is someone actually checking the specifics: the citation, the sample size, the missing caveat, the uncredited sentence.

If you're the one writing, with or without AI help, the practical lesson is the same one that should guide any byline you put your name on: read your own sources closely enough that you'd catch it if a cited number quietly changed. A detector can flag a pattern in your prose. It takes an actual read, your own or a critic's, to catch an error buried in your facts.

See what a detection score actually tells you

A detector score is the fast first pass, not the full verdict — run a draft through our free AI detector to see where the pattern concentrates, then go check the specific claims the way Herron and Levin did before you treat any score as the final word.

FAQ

Did the professors prove Schnell's column was AI-written? They proved specific factual and structural problems in the column that are more consistent with AI-generated text than with an expert's own writing: a citation that used the wrong sample size, a missing caveat the cited study's own author stated, outdated references, uncredited paraphrase of a press release, and a disconnected logical transition. They were explicit that their method, unlike a detector, can't put a percentage on the whole piece — it can only point to specific passages and explain why each one looks the way AI-generated text tends to look.

Is a content-level read more reliable than an AI detector? They answer different questions. A detector scores statistical patterns across an entire document quickly, without anyone needing domain expertise in the subject matter. A content read requires someone who knows the field well enough to check whether a citation, a number, or a claim is actually accurate — which is slower, but catches errors a pattern-matching score has no way to see.

Why did Schnell's own email explaining his process also score as AI-written? Because he said he used the same process, drafting the explanation himself and then running it through AI for phrasing, that he used for the original column. A detector measures the finished text, not which document it came from or why it was written — if the process is similar, the score can come out similar too. We covered that part of the story in detail here.