MAGE vs OpenAI RoBERTa detector, which AI detector is more accurate?
MAGE is the more useful of the two, on catch rate and on AUROC. It caught 46.4% of AI texts against 9.5% for the OpenAI RoBERTa detector, with an AUROC of 0.666 against 0.579. The OpenAI detector flagged fewer human texts, 4.8% against 19.0%, because it rarely flags anything. MAGE still misses most AI text.
Tested
MAGE vs OpenAI RoBERTa detector: the numbers side by side
| Measure | MAGE | OpenAI RoBERTa detector |
|---|---|---|
| Rank on the leaderboard | 4 of 7 | 6 of 7 |
| Balanced accuracy | 63.7% | 52.4% |
| Catches AI text | 46.4% | 9.5% |
| Flags human text | 19.0% | 4.8% |
| Catches humanized AI text | 23.8% | 0.0% |
| AUROC | 0.666 | 0.579 |
| Flags famous public-domain texts | 20.0% | 3.3% |
| Catches Claude Fable 5.1 | 28.6% | 3.2% |
| Catches Claude Haiku 5.5 | 66.7% | 20.6% |
| Catches Claude Opus 5.5 | 45.2% | 2.4% |
| Catches Claude Sonnet 5.5 | 45.2% | 11.9% |
| Flags human text: Academic abstracts | 12.0% | 0.0% |
| Flags human text: Q&A answers | 7.1% | 10.7% |
| Flags human text: Encyclopedia articles | 40.0% | 4.0% |
| Flags human text: Fiction | 4.2% | 4.2% |
| Flags human text: News reports | 33.3% | 4.2% |
| Seconds per 1,000 words | 0.39 | 0.07 |
| Type | Trained classifier | Trained classifier |
| Licence | Apache-2.0 | MIT |
How cautious each one is
MAGE and the OpenAI detector had the lowest catch rates in our test, and they come from different eras. The OpenAI RoBERTa detector was released in 2019 to spot GPT-2 output. MAGE is a Longformer model from a 2024 research paper, trained on text from many language models released before it.
The OpenAI detector caught 9.5% of our AI texts and flagged 4.8% of human ones. MAGE caught 46.4% and flagged 19.0%. Neither is impressive, but MAGE’s scores carry more information: its AUROC was 0.666, against 0.579 for the OpenAI model, and its balanced accuracy was 63.7% against 52.4%. The newer training data is the likely reason MAGE does better on Claude text. It caught 45.2% of Claude Opus 5.5 texts, where the OpenAI detector caught 2.4%.
Where each one does find AI text
MAGE’s catches are concentrated in a few genres. It caught 91.7% of AI news articles and 68.0% of AI encyclopedia articles, but only 12.0% of AI academic abstracts. The OpenAI detector’s best genre was fiction, at 25.0%, and it caught no AI Q&A answers at all. MAGE also needs length. It caught 57.6% of AI texts of 300 words or more but 20.6% of those under 200 words, where the OpenAI detector caught 7.2%.
The same split shows on human writing. MAGE flagged 40.0% of human encyclopedia articles, so its strongest genres for catching AI are also where it accuses humans most. The OpenAI detector’s false positives were thin everywhere, highest on Q&A answers at 10.7%.
On 30 famous public-domain texts, the OpenAI detector flagged only Genesis 1. MAGE flagged 6, among them the US Constitution, the Declaration of Independence and The Communist Manifesto, and no novel openings.
Is either one good enough?
Not for deciding whether someone used AI. MAGE misses more than half of AI text and flags 19.0% of human writing, and the OpenAI detector misses almost everything. Both are quick, at 0.39 seconds per 1,000 words for MAGE and 0.07 for the OpenAI model, but speed does not help a detector that misses this much. If you want a cautious detector, Desklib flagged 6.3% of human texts while catching 58.3% of AI texts. Fakespot vs MAGE shows how MAGE fares against a detector that leans the other way.
Questions people ask
Which is better at catching Claude Fable 5.1?
MAGE, though neither does well. MAGE caught 28.6% of Claude Fable 5.1 texts and the OpenAI RoBERTa detector 3.2%.
Where does MAGE flag human writing?
Mostly on encyclopedia articles and news. It flagged 40.0% of human encyclopedia articles and 33.3% of human news reports, but only 4.2% of human fiction.
Do these detectors catch humanized text?
MAGE caught 23.8% of humanized Claude Opus 5.5 texts. The OpenAI RoBERTa detector caught none.