Fakespot vs TMR, which AI detector is more accurate?
Fakespot is the more accurate of the two, with a balanced accuracy of 65.7% against 50.0% for TMR. TMR caught more AI text, 83.3% against 66.3%, but it flagged 83.3% of human texts too, against 34.9% for Fakespot. Most of TMR's verdicts follow paragraph breaks, so its high catch rate means little.
Tested
Fakespot AI text detector vs TMR (Target Mining RoBERTa): the numbers side by side
| Measure | Fakespot AI text detector | TMR (Target Mining RoBERTa) |
|---|---|---|
| Rank on the leaderboard | 3 of 7 | 7 of 7 |
| Balanced accuracy | 65.7% | 50.0% |
| Catches AI text | 66.3% | 83.3% |
| Flags human text | 34.9% | 83.3% |
| Catches humanized AI text | 47.6% | 81.0% |
| AUROC | 0.741 | 0.605 |
| Flags famous public-domain texts | 43.3% | 93.3% |
| Catches Claude Fable 5.1 | 50.0% | 88.1% |
| Catches Claude Haiku 5.5 | 81.0% | 81.0% |
| Catches Claude Opus 5.5 | 65.1% | 83.3% |
| Catches Claude Sonnet 5.5 | 69.0% | 81.0% |
| Flags human text: Academic abstracts | 12.0% | 16.0% |
| Flags human text: Q&A answers | 39.3% | 100.0% |
| Flags human text: Encyclopedia articles | 40.0% | 100.0% |
| Flags human text: Fiction | 12.5% | 100.0% |
| Flags human text: News reports | 70.8% | 100.0% |
| Seconds per 1,000 words | 0.07 | 0.09 |
| Type | Trained classifier | Trained classifier |
| Licence | Apache-2.0 | MIT |
Two detectors that flag too much
Both of these RoBERTa-base classifiers call a lot of human writing AI, but the errors come from different places. Fakespot’s false positives follow the kind of writing: it flagged 70.8% of human news reports and 40.0% of encyclopedia articles, yet only 12.0% of academic abstracts and 12.5% of fiction. TMR’s follow the layout. It flagged every human news report, encyclopedia article, Q&A answer and fiction passage, and passed most abstracts, which are usually a single paragraph.
The famous texts show how far apart they are. TMR flagged 28 of 30 public-domain classics and Fakespot 13. There was no classic that Fakespot flagged and TMR passed.
What happens without line breaks
We scored every test text a second time as one paragraph, with the line breaks replaced by spaces. Fakespot gave exactly the same verdicts, because its input cleaning already removes line breaks. TMR changed completely. On the 101 human texts with more than one paragraph, it went from 100.0% flagged to 25.7%, and its catch rate on AI text dropped to 38.7%. The paragraph break guide has the results for all seven detectors.
So the two can be compared fairly in their single-paragraph form too. There TMR flagged fewer human texts, 23.8% against 34.9% for Fakespot, but Fakespot caught far more AI text.
Claude models and humanized text
TMR’s catch rates barely moved between models, from 81.0% on Claude Haiku 5.5 and Sonnet 5.5 to 88.1% on Claude Fable 5.1. Most of the other detectors found Fable the hardest model to catch. Fakespot caught 50.0% of Fable texts and 81.0% of Haiku texts. Humanized Opus text made almost no difference to TMR, at 81.0%, while Fakespot fell from 65.1% to 47.6%. The likely reason is that a rewrite changes the wording, while TMR reacts mainly to paragraph breaks.
If you must pick one
Fakespot. Its scores do separate AI from human text to a degree, with an AUROC of 0.741 against 0.605 for TMR, and it was fast at 0.07 seconds per 1,000 words, with TMR at 0.09. Still, it flagged 34.9% of human writing, so it suits a first pass over a pile of texts where a person checks every flag. TMR as published should not be used at all, since its verdict mostly tells you whether a text has paragraphs.
Questions people ask
Why does TMR catch more AI text than Fakespot?
Mostly because it flags almost any text with paragraph breaks. With the breaks removed, TMR caught 38.7% of AI texts, while Fakespot caught 66.3% either way.
Which flags fewer human texts, Fakespot or TMR?
Fakespot, with 44 of 126 human texts flagged against 105 for TMR. On academic abstracts they were close, at 12.0% for Fakespot and 16.0% for TMR.
Can I make TMR usable by removing line breaks?
It helps with false positives, which fell to 23.8%, but its catch rate fell to 38.7%. Fakespot catches more AI text than that.