Fast-DetectGPT vs OpenAI RoBERTa detector, which is more accurate on Claude text?
Fast-DetectGPT is far more accurate on current AI text. It caught 68.1% of AI texts against 9.5% for the OpenAI RoBERTa detector, and 63.5% of Claude Opus 5.5 texts against 2.4%. The OpenAI detector flagged fewer human texts, 4.8% against 15.9%, but that comes from calling almost everything human. Fast-DetectGPT's threshold was tuned on our calibration split.
Tested
Fast-DetectGPT (Qwen2.5-1.5B) vs OpenAI RoBERTa detector: the numbers side by side
| Measure | Fast-DetectGPT (Qwen2.5-1.5B) | OpenAI RoBERTa detector |
|---|---|---|
| Rank on the leaderboard | 1 of 7 | 6 of 7 |
| Balanced accuracy | 76.1% | 52.4% |
| Catches AI text | 68.1% | 9.5% |
| Flags human text | 15.9% | 4.8% |
| Catches humanized AI text | 47.6% | 0.0% |
| AUROC | 0.815 | 0.579 |
| Flags famous public-domain texts | 56.7% | 3.3% |
| Catches Claude Fable 5.1 | 46.0% | 3.2% |
| Catches Claude Haiku 5.5 | 88.9% | 20.6% |
| Catches Claude Opus 5.5 | 63.5% | 2.4% |
| Catches Claude Sonnet 5.5 | 73.8% | 11.9% |
| Flags human text: Academic abstracts | 40.0% | 0.0% |
| Flags human text: Q&A answers | 21.4% | 10.7% |
| Flags human text: Encyclopedia articles | 8.0% | 4.0% |
| Flags human text: Fiction | 0.0% | 4.2% |
| Flags human text: News reports | 8.3% | 4.2% |
| Seconds per 1,000 words | 0.99 | 0.07 |
| Type | Zero-shot | Trained classifier |
| Licence | MIT (method), Apache-2.0 (model) | MIT |
Catch rates for each Claude model
On the leaderboard, Fast-DetectGPT ranked first and the OpenAI RoBERTa detector, released in 2019 to spot GPT-2 output, ranked sixth of seven. The gap is clearest model by model.
Fast-DetectGPT caught 88.9% of Claude Haiku 5.5 texts, 73.8% of Sonnet 5.5, 63.5% of Opus 5.5 and 46.0% of Fable 5.1. The OpenAI RoBERTa detector caught 20.6%, 11.9%, 2.4% and 3.2% of the same sets. Its order differs too: it found Opus the hardest, while Fable was hardest for Fast-DetectGPT and for most of the other detectors. The per-model pages, such as Can AI detectors catch Claude Opus 5.5?, compare all seven.
Light editing widens the gap. After a humanizer rewrite of the Opus texts, Fast-DetectGPT still caught 47.6%, while the OpenAI detector flagged none of them.
What Fast-DetectGPT’s catches cost
Fast-DetectGPT flagged 20 of 126 human texts, the OpenAI detector 6. Most of Fast-DetectGPT’s errors were on academic abstracts, 40.0% of which it flagged, and Q&A answers, at 21.4%. The OpenAI detector flagged no abstracts at all.
That comparison flatters the older model. Its AUROC was 0.579, close to the 0.5 of guessing, against 0.815 for Fast-DetectGPT. Its scores barely separate human from AI text, so it passes human writing for the same reason it passes AI writing. On 30 famous classics the pattern repeats: it flagged only Genesis 1, which every detector flagged, while Fast-DetectGPT flagged 17.
Fast-DetectGPT had help the others did not. We set its threshold on our calibration split, because the method has none for Qwen2.5-1.5B. The OpenAI detector used its default of 0.5.
When to use which
For any text that might come from a current chatbot, the OpenAI RoBERTa detector is not worth running. A “human” verdict from it carries almost no information, and its speed of 0.07 seconds per 1,000 words does not make up for that. Fast-DetectGPT is the stronger tool here, though at 15.9% false positives, and 40.0% on academic abstracts, a flag from it is only a reason to look closer. Readers who mainly check scientific writing should look at Fast-DetectGPT vs MAGE for how that genre trips up detectors.
Questions people ask
Can the OpenAI RoBERTa detector detect Claude?
Barely. Its best result was 20.6% of Claude Haiku 5.5 texts, and it caught 3.2% of Claude Fable 5.1 texts. Fast-DetectGPT caught 88.9% and 46.0% of the same sets.
Does the OpenAI detector ever beat Fast-DetectGPT?
On false positives, yes. It flagged none of the 25 human academic abstracts, where Fast-DetectGPT flagged 10. But it caught only 1.0% of AI abstracts.
Which is faster?
The OpenAI RoBERTa detector, at 0.07 seconds per 1,000 words. Fast-DetectGPT took 0.99 seconds, the slowest in our test.