Best AI detector for academic writing (tested)
Desklib is the safer AI detector for academic writing. On our arXiv abstracts it caught 50.0% of AI-written abstracts and flagged 2 of 25 human ones (8.0%). Fast-DetectGPT had a slightly higher balanced accuracy, 72.0% against 71.0%, but it flagged 10 of 25 human abstracts. Neither is accurate enough to accuse a student or researcher of using AI.
Tested
AI detectors ranked for academic writing
| # | Detector | Balanced accuracy | Catches AI text | Flags human text |
|---|---|---|---|---|
| 1 | Fast-DetectGPT (Qwen2.5-1.5B) | 72.0% | 84.0% | 40.0% |
| 2 | Desklib AI text detector | 71.0% | 50.0% | 8.0% |
| 3 | Fakespot AI text detector | 60.5% | 33.0% | 12.0% |
| 4 | OpenAI RoBERTa detector | 50.5% | 1.0% | 0.0% |
| 5 | MAGE | 50.0% | 12.0% | 12.0% |
| 6 | TMR (Target Mining RoBERTa) | 50.0% | 16.0% | 16.0% |
| 7 | E5-small LoRA detector | 42.5% | 21.0% | 36.0% |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| Fast-DetectGPT (Qwen2.5-1.5B) | 40.0% | 10 of 25 | 23.4 to 59.3% |
| Desklib AI text detector | 8.0% | 2 of 25 | 2.2 to 25.0% |
| Fakespot AI text detector | 12.0% | 3 of 25 | 4.2 to 30.0% |
| OpenAI RoBERTa detector | 0.0% | 0 of 25 | 0.0 to 13.3% |
| MAGE | 12.0% | 3 of 25 | 4.2 to 30.0% |
| TMR (Target Mining RoBERTa) | 16.0% | 4 of 25 | 6.4 to 34.7% |
| E5-small LoRA detector | 36.0% | 9 of 25 | 20.2 to 55.5% |
Why abstracts are hard for detectors
An arXiv abstract compresses a problem, a method and a result into one dense paragraph, in stock phrases such as “we propose” and “our results show”. That probably makes human abstracts predictable to a language model, and predictability is what some detectors read as a sign of AI. Fast-DetectGPT, which scores predictability directly, flagged 40.0% of human abstracts, far more than in any other genre we tested. E5-small LoRA flagged 36.0% of human abstracts and caught only 21.0% of AI ones, a balanced accuracy of 42.5%, which is worse than guessing.
AI abstracts are also hard to catch. Apart from Fast-DetectGPT, no detector caught more than half of them: Desklib caught 50.0%, Fakespot 33.0% and MAGE 12.0%. TMR, which flags almost every multi-paragraph text, caught just 16.0%, probably because most abstracts are a single paragraph (see how paragraph breaks change verdicts).
Desklib or Fast-DetectGPT for academic text?
On balanced accuracy the two are level, 72.0% for Fast-DetectGPT and 71.0% for Desklib, and Fast-DetectGPT got there with a threshold we chose on our calibration split, which the others did not have. They differ in the kind of mistake. Fast-DetectGPT caught 84.0% of AI abstracts and flagged 10 of 25 human ones. Desklib caught 50.0% and flagged 2 of 25. When a wrong flag hurts a person, as in grading or peer review, Desklib’s mistakes cost less.
One of Desklib’s two false positives was “On the Permanence of Vertices in Network Communities”, a 2014 network science abstract that five of the seven detectors flagged, Fast-DetectGPT included.
What this means for checking student work
Our human texts are research abstracts by working scientists. Student essays were not in the test and may score differently. With only 25 human abstracts the error bars are wide: Desklib’s 8.0% false positive rate has a 95% interval from 2.2% to 25.0%, and Fast-DetectGPT’s 40.0% runs from 23.4% to 59.3%. A flag from either tool is a reason to look at drafts, notes and sources. It cannot settle the question by itself. Results for all kinds of writing are on the leaderboard.
Questions people ask
Which AI detector works best on academic papers?
On our arXiv abstracts, Desklib, because it rarely flagged real abstracts (2 of 25) while catching 50.0% of AI ones. Fast-DetectGPT caught 84.0% of AI abstracts, but it also flagged 40.0% of the human abstracts.
Can a teacher rely on an AI detector for student essays?
We did not test student essays, only published research abstracts. Even on those, Desklib flagged 8.0% of human texts, and with 25 abstracts the 95% interval for that rate reaches 25.0%. That is too uncertain to base an accusation on.
Is Fast-DetectGPT good for academic writing?
It catches the most AI abstracts, 84.0%, but it flagged 10 of 25 human abstracts that were all written before ChatGPT. Its threshold was also tuned on our calibration texts, an advantage the other detectors did not get.