AI detectors systematically misjudge writing by non-native English speakers. In peer-reviewed research published in Patterns, seven widely used detectors wrongly flagged an average of 61.22% of human-written TOEFL essays as AI-generated, while misclassifying native-speaker essays only 5.19% of the time. The cause is built into how detection works.
Key takeaways
- The finding comes from Liang, Yuksekgonul, Mao, Wu and Zou, published in the Cell Press journal Patterns in 2023.
- Seven detectors were tested on 91 human-written TOEFL essays and 88 US eighth-grade essays. Average false-positive rates were 61.22% and 5.19% respectively.
- 89 of the 91 TOEFL essays, or 97.8%, were flagged by at least one detector. All seven agreed on 18 of them.
- When the researchers enriched the vocabulary of the same essays, the false-positive rate fell to 11.77%, which confirms the cause is linguistic variety rather than cheating.
- The authors explicitly caution against using detectors in evaluative or educational settings, particularly with non-native English speakers.
- For TEFL, this is not an edge case. It describes the entire student population.
What the study actually did
The design is simple and that is what makes it hard to argue with. The researchers took 91 TOEFL essays written by non-native English speakers, sourced from a Chinese educational forum, and 88 essays written by US eighth-graders from the Hewlett Foundation’s Automated Student Assessment Prize dataset. Every essay was human-written. None involved AI.
They ran both sets through seven widely used GPT detectors and compared the false-positive rates.
On the US eighth-grade essays, written by native speakers and younger writers at that, the detectors were close to perfect, with an average false-positive rate of 5.19%. On the TOEFL essays the average was 61.22%. Nearly every non-native essay, 89 out of 91, was flagged by at least one detector.
The numbers side by side
| Essay set | Written by | Average false-positive rate |
|---|---|---|
| US eighth-grade essays | Native English speakers | 5.19% |
| TOEFL essays, as written | Non-native English speakers | 61.22% |
| TOEFL essays, vocabulary enriched | Same non-native writers | 11.77% |
Liang, Yuksekgonul, Mao, Wu and Zou, GPT detectors are biased against non-native English writers, Patterns, 2023. Averages across seven detectors on human-written essays.
Why it happens
Detectors do not look for traces of a machine. They measure how predictable text is, usually through perplexity, which captures how surprising each word choice is given what came before.
Generated text tends to be smooth and predictable, because models favour likely words. So detectors treat low perplexity as a signal of AI authorship.
The problem is that low perplexity is also the signature of someone writing in a second language. A learner works from a smaller active vocabulary, reaches for sentence patterns practised in class, and avoids the idiomatic risks a confident native speaker takes for granted. The result is careful, regular, predictable prose. That is good language learning. To a perplexity-based detector it is indistinguishable from machine output.
The study confirms the mechanism directly. The essays that all seven detectors unanimously flagged had significantly lower perplexity than the rest, at a p-value of 9.74E-05. And when the researchers asked a language model to enrich the word choices in those same human-written essays, the false-positive rate collapsed from 61.22% to 11.77%. Nothing about authorship changed. Only the vocabulary did.
What this means if you teach English
In most subjects, detector bias against non-native speakers is a fairness problem affecting a minority of a class. In TEFL it affects everyone you teach.
Three consequences follow.
A detector score tells you almost nothing about your students. If a tool flags 61% of honest work from this population, a positive result carries very little information. You would be acting on something barely better than chance.
The students most likely to be accused are the least able to contest it. A learner writing in a second language, often studying abroad, sometimes on a visa tied to their course, is poorly placed to challenge a confident-looking percentage from a teacher or an institution.
Progress makes it worse, not better. The learner producing clean, controlled, accurate prose, the one doing exactly what you taught, generates the lowest perplexity in the class. The better they get at the target forms, the more machine-like they read.
What the researchers recommend
The paper does not hedge. It cautions strongly against using GPT detectors in evaluative or educational settings, and singles out contexts with significant numbers of non-native English speakers as the place where the risk of unwarranted consequences is highest.
That is an unusually direct recommendation for an academic paper, and it is worth quoting to an institution still treating detector output as evidence.
What to do instead
The alternatives are the same ones that work for any class, and they happen to be fairer. Set one unaided in-class writing task early so you have a baseline in that learner’s own voice. Ask for outlines and drafts, or use document version history, so you can see writing happen over time. Build tasks around a specific class discussion or text that would be hard to generate convincingly. Then, if something still looks off, have a two-minute conversation about how they approached it.
Our pillar guide to AI detectors for teachers covers that sequence in detail, and our report on the future of AI in English language teaching sets this alongside the wider evidence on AI in the classroom.
Read next
See the pillar guide to AI detectors for teachers, our breakdown of what AI detector teachers actually use, and the honest comparison of AI detectors.
Frequently asked questions
Are AI detectors biased against non-native English speakers?
Yes, and it is documented in peer-reviewed research. Liang and colleagues, writing in Patterns in 2023, found seven widely used detectors falsely flagged an average of 61.22% of human-written TOEFL essays by non-native speakers, against 5.19% for native-speaker essays.
Why do AI detectors flag ESL writing as AI-generated?
Because they measure predictability rather than authorship. Learners use a smaller active vocabulary and familiar sentence patterns, producing low-perplexity text. Detectors treat low perplexity as a signal of machine generation, so careful second-language writing scores as AI.
Does better English make the problem go away?
Partly, and in a way that proves the point. When researchers enriched the vocabulary of the same human-written essays, the false-positive rate fell from 61.22% to 11.77%. Authorship never changed. Only the linguistic variety did.
Which detectors were tested in the study?
Seven widely used GPT detectors were evaluated. The paper reports averages across all seven rather than singling out individual products, and all seven agreed in wrongly flagging 18 of the 91 TOEFL essays.
Can I still use a detector if I teach English learners?
The study’s authors advise against using detectors in evaluative settings, particularly with non-native speakers. If you run one, treat a flag as a reason to look at drafts and talk to the student, never as evidence, and never as grounds for an allegation.
Is there a detector that handles ESL writing fairly?
None has demonstrated it. No major vendor publishes false-positive rates broken down by the writer’s first language, and the bias stems from the underlying perplexity-based method rather than any single product’s implementation.
Quick favour: how do you actually use AI in your teaching?
We are running the first real study of AI use among TEFL teachers. 12 questions, about 3 minutes, fully anonymous. Leave your email at the end and you will get the report before anyone else.
