MAGE vs TMR, which AI detector is more accurate?
MAGE is the more accurate detector, with a balanced accuracy of 63.7% against 50.0% for TMR and far fewer false positives, 19.0% of human texts against 83.3%. TMR's higher catch rate, 83.3% against 46.4%, comes from flagging nearly every multi-paragraph text. Even scored without line breaks, TMR trailed MAGE on both measures.
Tested
MAGE vs TMR (Target Mining RoBERTa): the numbers side by side
| Measure | MAGE | TMR (Target Mining RoBERTa) |
|---|---|---|
| Rank on the leaderboard | 4 of 7 | 7 of 7 |
| Balanced accuracy | 63.7% | 50.0% |
| Catches AI text | 46.4% | 83.3% |
| Flags human text | 19.0% | 83.3% |
| Catches humanized AI text | 23.8% | 81.0% |
| AUROC | 0.666 | 0.605 |
| Flags famous public-domain texts | 20.0% | 93.3% |
| Catches Claude Fable 5.1 | 28.6% | 88.1% |
| Catches Claude Haiku 5.5 | 66.7% | 81.0% |
| Catches Claude Opus 5.5 | 45.2% | 83.3% |
| Catches Claude Sonnet 5.5 | 45.2% | 81.0% |
| Flags human text: Academic abstracts | 12.0% | 16.0% |
| Flags human text: Q&A answers | 7.1% | 100.0% |
| Flags human text: Encyclopedia articles | 40.0% | 100.0% |
| Flags human text: Fiction | 4.2% | 100.0% |
| Flags human text: News reports | 33.3% | 100.0% |
| Seconds per 1,000 words | 0.39 | 0.09 |
| Type | Trained classifier | Trained classifier |
| Licence | Apache-2.0 | MIT |
How each one goes wrong
MAGE and TMR go wrong in opposite directions. MAGE is reluctant to say “AI”: it caught fewer than half of our AI texts. TMR says “AI” to almost everything with more than one paragraph. It flagged all 101 multi-paragraph human texts in the test, every one written before ChatGPT.
That makes TMR’s numbers on AI text look strong. It caught 88.1% of Claude Fable 5.1 texts, the model most other detectors found hardest, against 28.6% for MAGE. After a humanizer rewrite of the Claude Opus 5.5 texts, TMR still caught 81.0% and MAGE 23.8%. With a false positive rate of 83.3%, those figures say little about AI writing.
Giving TMR a fair chance
TMR’s problem is formatting, so we scored every test text again with line breaks replaced by spaces (the paragraph break guide has the method). MAGE’s input cleaning already removes line breaks, and its results did not change. TMR’s did. It flagged 23.8% of human texts and caught 38.7% of AI texts.
Even in that form MAGE stays ahead on both counts, with 19.0% of human texts flagged and 46.4% of AI texts caught. So a TMR user who strips line breaks first still ends up with a worse detector than MAGE as published.
Kinds of writing
On encyclopedia articles, MAGE’s weakest genre for false positives, it flagged 40.0% of human texts; TMR flagged all of them. News followed the same pattern, 33.3% against 100.0%. Academic abstracts are the one genre where TMR mostly behaves, and there each detector flagged AI and human abstracts at the same low rate, which means neither can separate them at all.
On famous public-domain texts, TMR flagged 28 of 30 and MAGE 6, all six of them among TMR’s 28.
Who should use either
TMR as published suits nobody. MAGE is the better of the two, but at 46.4% of AI text caught and 19.0% of human text flagged, it is not good enough to accuse anyone. Readers who want a detector that catches more without TMR’s formatting problem can compare Fast-DetectGPT vs TMR.
Questions people ask
Is TMR better than MAGE at catching AI text?
Only on paper. TMR caught 83.3% of AI texts and MAGE 46.4%, but TMR also flagged 83.3% of human texts. As single paragraphs, TMR caught 38.7%.
Which is faster, MAGE or TMR?
TMR, at 0.09 seconds per 1,000 words against 0.39 for MAGE on an Apple M4 Pro.
Do both detectors struggle with academic writing?
Yes. MAGE caught 12.0% of AI abstracts and flagged 12.0% of human ones. TMR caught 16.0% and flagged 16.0%. Neither tells the two apart.