How accurate are AI detectors? What our test of 900 texts shows
Less accurate than their reputation. In our test of seven free detectors, the two best reached about 76% balanced accuracy. Desklib caught 58.3% of AI texts and wrongly flagged 6.3% of human ones; Fast-DetectGPT caught 68.1% and flagged 15.9%. Accuracy also swung widely with the kind of writing, the AI model and light editing.
Tested
Catch rate and false positive rate
A detector makes two kinds of mistake. It can miss AI text, and it can flag human text. The catch rate measures the first, the false positive rate the second. Most accuracy claims blend them into one number, which hides the trade-off.
TMR caught 83.3% of AI texts and flagged 83.3% of human texts, so it was no better than flagging everything. Desklib flagged only 6.3% of human texts but caught just 58.3% of AI texts. Fast-DetectGPT sat between them. You can see all seven on the leaderboard.
Why a small false positive rate still hurts
Imagine a class of 100 essays in which 10 were written with AI. A detector with Desklib’s rates would catch about 6 of the 10 AI essays. It would also flag about 6 of the 90 human essays, because 6.3% of 90 is close to 6. Half of the flagged essays would belong to students who did nothing wrong.
The fewer AI texts there really are, the larger the share of flags that land on honest students, so treat a flag from any detector we tested as a reason to ask about the work.
Accuracy depends on what you are checking
The kind of writing changed the results more than the choice between the two best detectors. Desklib flagged no human fiction or encyclopedia articles but 20.8% of human news reports. Fast-DetectGPT passed every human fiction passage and flagged 40.0% of human academic abstracts.
The AI model mattered as much. A majority of detectors caught 77.0% of Claude Haiku 5.5 texts and only 38.9% of Claude Fable 5.1 texts. Editing mattered too: after a humanizer rewrite, Desklib’s catch rate on Claude Opus 5.5 texts fell from 62.7% to 32.5%.
What to do with a detector result
Treat a high score as a reason to look at the writing process, such as drafts, version history or a conversation about the text. Treat a low score on edited text as weak evidence. And check which detector a tool uses: the OpenAI RoBERTa detector, which is still free to download and widely used, caught 9.5% of the AI texts in our test.
Questions people ask
What is a good accuracy for an AI detector?
It depends on what a false flag costs. For judging a person, the false positive rate matters most, and even Desklib, the most careful detector we tested, flagged 6.3% of human texts.
Are AI detectors more accurate on longer texts?
Some are. MAGE caught 20.6% of AI texts under 200 words and 57.6% of those with 300 words or more. Fast-DetectGPT and Desklib changed little with length.
Why do AI detector companies claim 99% accuracy?
We cannot check claims about detectors we have not tested. Accuracy figures depend heavily on the test texts, and our results show the same detector scoring very differently across kinds of writing and AI models.