Best AI detector for fiction and creative writing (tested)
Fast-DetectGPT is the best AI detector for fiction in our test. It caught 62.5% of AI-written fiction passages and flagged none of the 24 human passages from public-domain novels, a balanced accuracy of 81.2%. Fakespot caught a little more AI fiction, 67.7%, but flagged 3 of 24 human passages. Most of the seven detectors missed most AI fiction.
Tested
AI detectors ranked for fiction
| # | Detector | Balanced accuracy | Catches AI text | Flags human text |
|---|---|---|---|---|
| 1 | Fast-DetectGPT (Qwen2.5-1.5B) | 81.3% | 62.5% | 0.0% |
| 2 | Fakespot AI text detector | 77.6% | 67.7% | 12.5% |
| 3 | MAGE | 69.8% | 43.8% | 4.2% |
| 4 | E5-small LoRA detector | 63.0% | 38.5% | 12.5% |
| 5 | Desklib AI text detector | 60.4% | 20.8% | 0.0% |
| 6 | OpenAI RoBERTa detector | 60.4% | 25.0% | 4.2% |
| 7 | TMR (Target Mining RoBERTa) | 50.0% | 100.0% | 100.0% |
Numbers behind this chart
| Item | Rate | Texts | 95% interval |
|---|---|---|---|
| Fast-DetectGPT (Qwen2.5-1.5B) | 0.0% | 0 of 24 | 0.0 to 13.8% |
| Fakespot AI text detector | 12.5% | 3 of 24 | 4.3 to 31.0% |
| MAGE | 4.2% | 1 of 24 | 0.7 to 20.2% |
| E5-small LoRA detector | 12.5% | 3 of 24 | 4.3 to 31.0% |
| Desklib AI text detector | 0.0% | 0 of 24 | 0.0 to 13.8% |
| OpenAI RoBERTa detector | 4.2% | 1 of 24 | 0.7 to 20.2% |
| TMR (Target Mining RoBERTa) | 100.0% | 24 of 24 | 86.2 to 100.0% |
How fiction changes the ranking
Desklib, which leads on encyclopedia articles and forum answers, falls to a 60.4% balanced accuracy here, level with the OpenAI RoBERTa detector, a 2019 model built for GPT-2. Fakespot, third in the overall ranking, comes second here at 77.6%. Most detectors flagged little or no human fiction. Fast-DetectGPT and Desklib flagged none of the 24 passages, and MAGE and the OpenAI RoBERTa detector flagged 1 each.
The AI side is where fiction gets hard. Desklib caught 20.8% of AI-written passages, E5-small LoRA 38.5% and MAGE 43.8%. A story is free to choose its own words, scenes and voice, while an encyclopedia entry or a news report pushes every writer toward the same phrasing, so a classifier probably has fewer fixed habits to latch onto. That is our best guess; the data shows only the low catch rates.
Why the human passages may be too easy
Our human fiction comes from novels on Project Gutenberg that are old enough to be in the public domain. Their vocabulary and sentence rhythms differ from what a modern chatbot usually writes, which probably makes them easier to tell apart from AI fiction than a contemporary novel would be. The zero false positives for Fast-DetectGPT also rest on 24 passages, so the 95% interval runs from 0.0% to 13.8%. A writer with a plain modern style should not assume the same result.
TMR is the exception in both directions. It caught all 96 AI passages and flagged all 24 human ones, a balanced accuracy of 50.0%. Fiction with dialogue is split into many short paragraphs, and TMR flags almost any text with more than one paragraph, as our paragraph break test shows.
Which detector to use on creative writing
Fast-DetectGPT gives the best balance, with the caveat that its threshold was tuned on our calibration split while the others used their authors’ settings. Fakespot catches a little more AI fiction at the cost of 3 of 24 human passages flagged (12.5%, with an interval from 4.3% to 31.0%). Neither caught more than 67.7% of AI fiction, so a pass from either says little about who wrote a story. All seven detectors are compared on the leaderboard.
Questions people ask
Can AI detectors detect AI-written stories?
Partly. Fakespot caught 67.7% of AI fiction passages and Fast-DetectGPT 62.5%. Desklib, which caught 81.0% of AI encyclopedia articles, caught only 20.8% of AI fiction.
Will an AI detector flag my novel as AI?
Fast-DetectGPT and Desklib flagged none of our 24 human passages, but the 95% interval for that result still runs to 13.8%. Our human fiction came from novels old enough to be in the public domain, and modern prose may score differently. TMR flagged every passage.
Why is Desklib so weak on fiction?
The data does not show why. It flagged no human fiction, so it is not simply overcautious, but most AI fiction scored below its threshold of 0.5. Fast-DetectGPT, which needs no training data, caught 62.5% of the same passages.