OpenAI RoBERTa detector accuracy on modern AI text
OpenAI's RoBERTa detector caught 9.5% of the AI-written texts in our test and only 2.4% of texts from Claude Opus 5.5. It rarely flags human writing (4.8%), but that is because it calls almost everything human. It was built in 2019 to spot GPT-2 output and does not work on today's models.
Tested
Verdict: Obsolete. It almost never flags anything, so a pass from this detector tells you nothing about modern AI text.
OpenAI RoBERTa detector test results
| Rank | 6 of 7 |
|---|---|
| Balanced accuracy | 52.4% |
| Catches AI text | 9.5% (48 of 504; 95% interval 7.3 to 12.4%) |
| Flags human text | 4.8% (6 of 126; 95% interval 2.2 to 10.0%) |
| Catches humanized AI text | 0.0% (0 of 126) |
| AUROC | 0.579 |
| Speed on an Apple M4 Pro | 0.07 seconds per 1,000 words |
| Type | Trained classifier |
| Threshold | 0.5 (default probability cut-off) |
| Licence | MIT |
| Model | openai-community/roberta-base-openai-detector |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| Claude Fable 5.1 | 3.2% | 4 of 126 | 1.2 to 7.9% |
| Claude Haiku 5.5 | 20.6% | 26 of 126 | 14.5 to 28.5% |
| Claude Opus 5.5 | 2.4% | 3 of 126 | 0.8 to 6.8% |
| Claude Sonnet 5.5 | 11.9% | 15 of 126 | 7.3 to 18.7% |
| Humanized Claude Opus text | 0.0% | 0 of 126 | 0.0 to 3.0% |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| Academic abstracts | 0.0% | 0 of 25 | 0.0 to 13.3% |
| Q&A answers | 10.7% | 3 of 28 | 3.7 to 27.2% |
| Encyclopedia articles | 4.0% | 1 of 25 | 0.7 to 19.5% |
| Fiction | 4.2% | 1 of 24 | 0.7 to 20.2% |
| News reports | 4.2% | 1 of 24 | 0.7 to 20.2% |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| under 200 words | 7.2% | 7 of 97 | 3.5 to 14.2% |
| 200 to 299 words | 11.6% | 29 of 249 | 8.2 to 16.2% |
| 300 words or more | 7.6% | 12 of 158 | 4.4 to 12.8% |
What it is
OpenAI released this RoBERTa-base model in 2019 alongside GPT-2, to show that text from that model could be detected. It is one of the most downloaded detectors on Hugging Face, so people still use it. We used the standard threshold of 0.5.
Results
It caught 9.5% of AI texts overall. Fiction was its best genre at 25.0%; it caught none of the AI-written forum answers. Not one humanized text was flagged.
Its false positive rate of 4.8% looks good, but a detector that calls almost everything human will always score well on human text. Its AUROC of 0.579 shows how little its scores separate the two.
If you find this detector behind a free online tool, treat a “human” result as meaningless for text written by current chatbots.
Human writing OpenAI RoBERTa detector flagged as AI
It is extra payments that you apply to the principal. Here is an example. Let's say that you have a mortgage with a monthly payment of $1000 per month (principal and interest), which you pay on the 1st of the month. Now, let's say that, in addition to your $1000 regular payment, you have an extra $1000 that you would like to pay to reduce your debt. So after you...
Unlike many other countries, US airports do not have any form of physical immigration controls when you are departing the country on an international flight. In fact, in most airports there isn't even a concept of an "International" terminal/gate, with the same gates frequently being used for international flights and domestic flights. As a result of this they can't enforce the concept of a 'transit' passenger - once you're in...
A 16-year-old boy has been arrested after he hijacked the Washington Township, New Jersey Wal-Mart's public announcement system to make racial comments. The boy was arrested Friday night and then released into his parents' custody. He was charged with harassment as a bias crime. The boy, whose name was not released, faces up to a year in a youth detention center if convicted. Police say that they are not sure...
Questions people ask
Does OpenAI have an AI text detector?
OpenAI released this RoBERTa detector in 2019 for GPT-2 text. It also launched a separate AI text classifier in January 2023 and withdrew it in July 2023 because of its low accuracy. The 2019 model is still downloadable, and it caught 9.5% of AI texts in our test.
Can the OpenAI RoBERTa detector catch Claude?
Hardly. It caught 2.4% of Claude Opus 5.5 texts, 11.9% of Sonnet 5.5, 20.6% of Haiku 5.5 and 3.2% of Fable 5.1.