Fast-DetectGPT accuracy test on 900 texts
Fast-DetectGPT caught 68.1% of AI-written texts in our test, the highest catch rate of any detector that passed most human writing. It also flagged 15.9% of human texts as AI, and 40.0% of human academic abstracts, so its verdict on scientific writing is close to a coin toss.
Tested
Verdict: The best catch rate among the usable detectors, with a false positive rate that rules it out as evidence, especially for academic writing.
Fast-DetectGPT (Qwen2.5-1.5B) test results
| Rank | 1 of 7 |
|---|---|
| Balanced accuracy | 76.1% |
| Catches AI text | 68.1% (343 of 504; 95% interval 63.9 to 72.0%) |
| Flags human text | 15.9% (20 of 126; 95% interval 10.5 to 23.2%) |
| Catches humanized AI text | 47.6% (60 of 126) |
| AUROC | 0.815 |
| Speed on an Apple M4 Pro | 0.99 seconds per 1,000 words |
| Type | Zero-shot (no training data) |
| Threshold | 0.4619 (calibration split (maximum of true positive rate minus false positive rate)) |
| Licence | MIT (method), Apache-2.0 (model) |
| Model | Qwen/Qwen2.5-1.5B |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| Claude Fable 5.1 | 46.0% | 58 of 126 | 37.6 to 54.7% |
| Claude Haiku 5.5 | 88.9% | 112 of 126 | 82.2 to 93.3% |
| Claude Opus 5.5 | 63.5% | 80 of 126 | 54.8 to 71.4% |
| Claude Sonnet 5.5 | 73.8% | 93 of 126 | 65.5 to 80.7% |
| Humanized Claude Opus text | 47.6% | 60 of 126 | 39.1 to 56.3% |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| Academic abstracts | 40.0% | 10 of 25 | 23.4 to 59.3% |
| Q&A answers | 21.4% | 6 of 28 | 10.2 to 39.5% |
| Encyclopedia articles | 8.0% | 2 of 25 | 2.2 to 25.0% |
| Fiction | 0.0% | 0 of 24 | 0.0 to 13.8% |
| News reports | 8.3% | 2 of 24 | 2.3 to 25.8% |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| under 200 words | 74.2% | 72 of 97 | 64.7 to 81.9% |
| 200 to 299 words | 65.1% | 162 of 249 | 59.0 to 70.7% |
| 300 words or more | 69.0% | 109 of 158 | 61.4 to 75.7% |
How Fast-DetectGPT works
Fast-DetectGPT comes from a paper by Bao and colleagues at ICLR 2024. It needs no detector training. A language model reads the text and measures how much more predictable each word is than the words the model would have chosen itself. AI text tends to stick to the model’s preferred choices, so it scores high. We used Qwen2.5-1.5B, a model with 1.5 billion parameters, because it runs on a laptop.
The method comes without a threshold for this model, so we chose one on the 144 calibration texts, which are kept out of every reported number. The other detectors used their authors’ threshold. That gave Fast-DetectGPT a small advantage, and it still flagged 15.9% of human texts.
Where it does well and where it fails
It caught Claude Haiku 5.5 best, at 88.9%, and Claude Fable 5.1 worst, at 46.0%. News was its strongest genre: it caught 93.8% of AI news articles and flagged only 8.3% of human ones.
Academic abstracts went the other way. It caught 84.0% of AI abstracts, but it also flagged 40.0% of the human abstracts, all written before ChatGPT existed. Q&A answers were weak in both directions, with 43.8% of AI answers caught and 21.4% of human answers flagged. Fiction was the bright spot for false positives: no human fiction passage was flagged.
Editing and length
A light rewrite with a humanizer cut its catch rate on Claude Opus 5.5 texts from 63.5% to 47.6%. Text length made little difference. It caught 74.2% of AI texts under 200 words and 69.0% of those with 300 words or more.
It was the slowest detector we tested, at 0.99 seconds per 1,000 words on an Apple M4 Pro, which still means a typical essay takes about a second.
Human writing Fast-DetectGPT (Qwen2.5-1.5B) flagged as AI
It is extra payments that you apply to the principal. Here is an example. Let's say that you have a mortgage with a monthly payment of $1000 per month (principal and interest), which you pay on the 1st of the month. Now, let's say that, in addition to your $1000 regular payment, you have an extra $1000 that you would like to pay to reduce your debt. So after you...
Last Sunday evening at the Beverly Hilton Hotel in Beverly Hills, California were the 67th Annual Golden Globe Awards hosted by English comedian Ricky Gervais. The choice of Gervais as host was a move to increase viewership which declined to 15 million viewers last year from 27 million in 2004. Canadian director James Cameron—who previously won in 1997 for his film Titanic—and his high-grossing 3D epic Avatar seemed to dominate...
Despite the prevalence of community detection algorithms, relatively less work has been done on understanding whether a network is indeed modular and how resilient the community structure is under perturbations. To address this issue, we propose a new vertex-based metric called "permanence", that can quantitatively give an estimate of the community-like structure of the network. The central idea of permanence is based on the observation that the strength of membership...
Questions people ask
Is Fast-DetectGPT accurate?
Partly. It caught 68.1% of AI texts and flagged 15.9% of human texts in our test, which is among the best results here and still far from reliable.
Does Fast-DetectGPT need training data?
No. It is a zero-shot method. It asks a language model how predictable each word was and compares that with what the model itself would have written. We ran it with Qwen2.5-1.5B.
Why does Fast-DetectGPT flag academic abstracts?
Abstracts are written in a dense, predictable register, and predictable wording is exactly what the method treats as a sign of AI. It flagged 10 of 25 human abstracts in our test set.