Why AI Detectors Struggle Outside English (And What That Means for Your Score)

Quick answer

Most AI detectors, including the models powering commercial tools, are trained overwhelmingly on English text. When researchers test those same detectors on other languages, accuracy doesn't dip slightly, it falls off a cliff in some cases. The largest published multilingual benchmark found a detector fine-tuned only on English text lost 25.7% of its F1-score the moment it was tested on non-English languages it hadn't seen in training. A separate 2026 study testing English and Spanish side by side found a similar pattern: strong performance in English, a real and measurable drop in Spanish, even using the same model and the same training process. This isn't a reason to distrust detection outright. It's a reason to know exactly what a score is actually measuring before you read too much into it, especially if the text in question isn't in English.

The benchmark that measured the actual gap

MULTITuDE, built by researchers at the Kempelen Institute of Intelligent Technologies along with Penn State and University of Mississippi collaborators, is the largest published benchmark of its kind: 74,081 human-written and machine-generated texts across 11 languages (English, Spanish, Russian, Portuguese, Catalan, German, Dutch, Ukrainian, Czech, Arabic, and Chinese), generated by 8 different language models including GPT-3, GPT-4, and ChatGPT.

The headline number is stark. When the researchers fine-tuned a detector on English text only, then tested it on English, it scored an average F1 of 0.9292. Test that identical detector on languages it never saw during training, and the score dropped to 0.6903, a 25.7 percentage-point fall. That's not a rounding error. That's the difference between a detector you can rely on and one that's barely better than a coin flip for some language pairs.

The drop wasn't uniform, either. Detectors generalized better to languages that share an alphabet and grammatical structure with English (other Latin-script European languages) and generalized worse to languages with different scripts and structures, like Arabic and Chinese. The researchers also found something worth remembering before assuming any specific detector escapes this pattern: even GPTZero, tested as a zero-shot black-box baseline in the same benchmark, showed the same directional weakness moving from English into unfamiliar languages, just at different magnitudes depending on the language pair.

A second, independent study lands on the same finding

A 2026 paper presented at ICICS (the 17th International Conference on Information and Communication Systems, held in Jordan) ran a more targeted test: a RoBERTa-based detector trained and evaluated on the AuTexTification dataset, which contains both English and Spanish samples generated for exactly this kind of research.

The results: 82% accuracy and a 0.84 F1-score on the English test set. On the Spanish test set, using the same model and the same training pipeline, accuracy dropped to about 79% and the F1-score to 0.75. The researchers' own explanation points to Spanish's more complex morphology, verb conjugation system, and flexible word order, features that don't map cleanly onto patterns a model learned primarily from English data.

Two things stand out about this pair of studies. First, they were run independently, by different teams, using different datasets and different detection approaches, and they land on the same underlying conclusion: cross-lingual generalization is a real, measurable weakness in AI text detection, not a one-off finding from a single paper. Second, neither team frames this as detection being broken. Both frame it as a data problem. As the ICICS paper puts it directly: "the vast majority of our knowledge regarding AI detection technology is based in English, resulting in a serious lack of coverage for other languages." Training data determines what a model can reliably recognize, and most of the training data driving today's detectors is English.

Why this happens, mechanically

AI text detection largely works by measuring statistical properties of language, things like perplexity (how predictable a sequence of words is to a language model) and burstiness (how much sentence length and structure vary across a piece of writing). Those measurements are calibrated against enormous quantities of real human writing, and the overwhelming majority of that training data, for almost every detector on the market, is in English.

That calibration doesn't transfer automatically. A perplexity baseline built from English prose doesn't describe what "normal" human writing looks like in Mandarin, Arabic, or Ukrainian, because those languages have entirely different rhythms, sentence structures, and predictability patterns at the word level. When a detector trained this way scores non-English text, it's applying an English yardstick to a different kind of ruler. The MULTITuDE researchers found that fine-tuning on multiple languages at once measurably improved cross-lingual generalization compared to English-only training, which supports the same conclusion from the other direction: the gap is about what the model has actually seen, not some fixed ceiling on how well detection can ever work outside English.

This connects to a pattern we've written about before in the context of non-native English writers being flagged more often by detectors trained mostly on native-English text. It's the same underlying mechanism showing up in two related but distinct forms: within English, detectors miscalibrate against writing that doesn't match the statistical profile of native speakers. Across languages, the same miscalibration happens at a larger scale, because the training data gap is even wider.

What this means if you're reading a score on non-English text

If a piece of writing isn't in English, or was translated from another language, treat any AI-detection score on it with more caution than you would an English-language result, not less. The research above shows measurable accuracy drops even for well-resourced languages like Spanish, and the drop-off tends to get worse for languages that are structurally further from English or have less representation in training data generally.

This is also a straightforward case where honesty about a product's limits matters more than a marketing claim. TheChecker.AI is built and optimized for English content, and we say that plainly rather than promise even-handed accuracy across languages we haven't rigorously tested. If you're checking English writing, whether it's a student essay, a professional draft, or your own work before you send it somewhere, that's exactly the use case our detector is built and measured for. If the text you're checking isn't in English, the honest answer, backed by the research above, is to weigh that score with real caution.

FAQ

Does this mean AI detectors don't work at all outside English? No. It means accuracy is measurably lower and more variable outside English, not that detection is useless. Some language pairs (closely related European languages, for instance) hold up better than others (languages with different scripts and grammatical structures). The right response is caution and lower confidence in a non-English score, not discarding detection entirely.

Why don't detectors just train equally on every language? Training data availability is the core constraint. English has, by a wide margin, the largest volume of both human-written and AI-generated text available for training and evaluation. Building equally robust detection for lower-resource languages requires collecting and labeling comparable datasets in each of those languages, which is a slower, more expensive process that the field is still working through.

Is this the same issue as detectors flagging non-native English writers? Related, but distinct. The non-native-writer problem happens within English: a detector miscalibrates against English written by someone whose first language isn't English, because their sentence patterns differ from the training data's assumed "typical" native-speaker baseline. The cross-lingual problem happens when the text itself is in a different language altogether. Both trace back to the same root cause: training data that doesn't represent the full range of how people actually write.

Should I trust a detector's stated accuracy number if I don't know what language it was tested on? Ask before you assume. A single headline accuracy figure, like the ones we cover in our guide to how accurate AI detection actually is, is almost always measured on English test data unless a vendor specifically states otherwise. The same caution about population-level numbers versus your specific document applies with even more force when the language changes.