AI Detector False Positives: What Teachers Need to Know

How Detectors Work

Some detectors use features such as text predictability (perplexity) and structural variation (burstiness), but modern products may use different models and additional signals. None can determine authorship with certainty from text alone.

The logic behind these metrics was reasonable when they were developed. Large language models, optimised to produce smooth and statistically probable text, tend to score low on both. Human writing, with its unexpected word choices and irregular sentence rhythms, tends to score higher. Detectors were trained on this distinction.

The problem is that the distinction has eroded in every direction at once.

Three Ways Detectors Fail

The polish penalty

When students revise and edit their work before submission, they iron out the irregularities that detectors expect from human writers. A logically structured, carefully edited essay reads statistically like machine output because it has been cleaned up to read well. The detector is not finding AI. It is finding editorial quality.

The article previously cited an unsupported claim of false-positive rates above 30% for all kinds of professional writing. That figure should not be used without an identifiable dataset and model version. A 2023 study of non-native English writing does show substantial errors under its specific testing conditions.

One student described the resulting paradox on a community forum: students are now being forced to intentionally degrade their own logical, human prose to satisfy a machine's statistical preference for messiness. "We are effectively punishing clarity," they wrote. r/ChatGPT, 2026

Discrimination against EAL students

Students writing in English as an Additional Language rely on more formalised grammatical structures and display less idiosyncratic vocabulary than native speakers. Their writing consistently triggers the low-perplexity patterns that detectors associate with machine generation.

In the 2023 study by Liang and colleagues, seven tested detectors averaged a 61.3% false-positive rate on 91 TOEFL essays by non-native writers. This result cannot be applied indiscriminately to all EAL students or to every updated commercial detector.

The humaniser arms race

AI humaniser tools take machine-generated text and restructure it to inject the burstiness and vocabulary variation that detectors look for. These tools analyse the original text for detection patterns and apply natural language processing to rewrite sentences, deliberately introducing the irregularities that signal human authorship to the algorithm.

Demonstrations of rewriting tools evading particular detectors can illustrate limitations, but a single anecdotal experiment cannot establish that all commercial detectors will return zero for every humanised submission. Model versions and text samples matter.

What the Humaniser Arms Race Means for Your Classroom

A high detector score may be a false positive, and a low one may miss AI-generated text. Scores should therefore be treated as uncertain signals that require independent context, not as confirmation of authorship or misconduct.

The institutions that moved earliest on this have been clear. UC San Diego Extended Studies deactivated the Turnitin detection module in April 2025. Cornell University and the University of Pittsburgh have both explicitly recommended against using detector scores as the basis for disciplinary action. Cornell University, 2026

The risk of proceeding with discipline based on a detector score is not abstract. Students have challenged academic integrity findings, produced their own counter-reports from alternative detectors showing 0% AI, and escalated complaints that have cost institutions far more time and credibility than the original incident would have if handled differently.

A detector score is not evidence. Treating it as evidence is a liability.

What to Use Instead

The three tools that actually work do not require software subscriptions and do not discriminate against any group of students.

Stylometric comparison. Compare the suspected submission directly against prior handwritten or in-class work from the same student. Look for shifts in vocabulary complexity, sentence structure, and punctuation habits. The evidence is the contrast between this submission and the student's own documented history. That contrast is something an administrator can review, a parent can examine, and a disciplinary process can evaluate. A detector score is not.

The oral defence. Ask the student to explain specific arguments from their submission in their own words. Ask them to define specific words they used. Ask them where they found a particular source. A student who wrote the work may be able to discuss its development, though anxiety, language proficiency and disabilities can affect oral responses. A conversation adds context but is not an infallible test.

The process trail. Request the Google Docs version history or Microsoft Word track changes for the submitted document. Genuine human writing has a revision trail: deletions, restructuring, progressive composition across multiple sessions. A 1500-word essay that appeared in a document in a single paste event with no subsequent editing has no process trail. That absence is reviewable evidence that any administrator can examine without relying on any algorithmic output.

For the full six-step framework for handling suspected AI submissions without detector evidence, see AI Didn't Cheat. Your Student Did.

FAQ