Fakespot AI text detector accuracy test
The Fakespot AI text detector caught 66.3% of AI-written texts in our test but flagged 34.9% of human texts as AI. News writing fared worst, with 70.8% of human news reports called AI-written, all of them published years before ChatGPT.
Tested
Verdict: Catches a fair share of AI text, but it flags about a third of human writing and most human news, so its verdicts cannot be trusted alone.
Fakespot AI text detector test results
| Rank | 3 of 7 |
|---|---|
| Balanced accuracy | 65.7% |
| Catches AI text | 66.3% (334 of 504; 95% interval 62.0 to 70.3%) |
| Flags human text | 34.9% (44 of 126; 95% interval 27.2 to 43.6%) |
| Catches humanized AI text | 47.6% (60 of 126) |
| AUROC | 0.741 |
| Speed on an Apple M4 Pro | 0.07 seconds per 1,000 words |
| Type | Trained classifier |
| Threshold | 0.5 (default probability cut-off) |
| Licence | Apache-2.0 |
| Model | fakespot-ai/roberta-base-ai-text-detection-v1 |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| Claude Fable 5.1 | 50.0% | 63 of 126 | 41.4 to 58.6% |
| Claude Haiku 5.5 | 81.0% | 102 of 126 | 73.2 to 86.9% |
| Claude Opus 5.5 | 65.1% | 82 of 126 | 56.4 to 72.8% |
| Claude Sonnet 5.5 | 69.0% | 87 of 126 | 60.5 to 76.5% |
| Humanized Claude Opus text | 47.6% | 60 of 126 | 39.1 to 56.3% |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| Academic abstracts | 12.0% | 3 of 25 | 4.2 to 30.0% |
| Q&A answers | 39.3% | 11 of 28 | 23.6 to 57.6% |
| Encyclopedia articles | 40.0% | 10 of 25 | 23.4 to 59.3% |
| Fiction | 12.5% | 3 of 24 | 4.3 to 31.0% |
| News reports | 70.8% | 17 of 24 | 50.8 to 85.1% |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| under 200 words | 43.3% | 42 of 97 | 33.9 to 53.2% |
| 200 to 299 words | 68.7% | 171 of 249 | 62.7 to 74.1% |
| 300 words or more | 76.6% | 121 of 158 | 69.4 to 82.5% |
What it is
This RoBERTa-base model comes from Fakespot, the team at Mozilla that built tools for spotting fake reviews. Its model card ships a text cleaning function, which we applied before scoring, and we used the standard probability threshold of 0.5.
Results
Fakespot caught 66.3% of AI texts. It did best on encyclopedia articles and news, catching 95.0% and 100.0% of the AI versions, and worst on academic abstracts, at 33.0%. Among the models, it caught 81.0% of Claude Haiku 5.5 texts and 50.0% of Claude Fable 5.1 texts.
It also flagged human writing in the same genres where it catches the most AI text, calling 70.8% of human news reports and 40.0% of human encyclopedia articles AI-written. It seems to react to the style of a genre more than to who wrote the text.
Length and rewrites
Its catch rate climbed with length, from 43.3% for AI texts under 200 words to 76.6% for those of 300 words or more. False positives also rose with length: it flagged 57.1% of human texts of 300 words or more. Longer texts give it more of whatever it reacts to, in both directions.
After a humanizer rewrite, it caught 47.6% of Claude Opus 5.5 texts, down from 65.1%.
Human writing Fakespot AI text detector flagged as AI
It is extra payments that you apply to the principal. Here is an example. Let's say that you have a mortgage with a monthly payment of $1000 per month (principal and interest), which you pay on the 1st of the month. Now, let's say that, in addition to your $1000 regular payment, you have an extra $1000 that you would like to pay to reduce your debt. So after you...
Last Sunday evening at the Beverly Hilton Hotel in Beverly Hills, California were the 67th Annual Golden Globe Awards hosted by English comedian Ricky Gervais. The choice of Gervais as host was a move to increase viewership which declined to 15 million viewers last year from 27 million in 2004. Canadian director James Cameron—who previously won in 1997 for his film Titanic—and his high-grossing 3D epic Avatar seemed to dominate...
Despite the prevalence of community detection algorithms, relatively less work has been done on understanding whether a network is indeed modular and how resilient the community structure is under perturbations. To address this issue, we propose a new vertex-based metric called "permanence", that can quantitatively give an estimate of the community-like structure of the network. The central idea of permanence is based on the observation that the strength of membership...
Questions people ask
Is the Fakespot AI detector accurate?
Not reliably. It caught 66.3% of AI texts and flagged 34.9% of human texts in our test, for a balanced accuracy of 65.7%.
Why does Fakespot flag news articles?
We cannot see inside the model. Human news reports were its most flagged genre, at 70.8%, while it flagged only 12.0% of human academic abstracts and 12.5% of fiction, which suggests it reacts to the style of a genre more than to who wrote the text.
Is the Fakespot detector fast?
Yes. It took 0.07 seconds per 1,000 words on an Apple M4 Pro, among the fastest in the test.