AI Defeats Bestselling Authors

In a blind test, 3 of 4 readers preferred AI

Mark Lawrence is a very successful fantasy writer. His PRINCE OF THORNS has sold more than one million copies. He is one of the many professional authors who, while disdaining the use of textual AI, is concerned about its eventual impact on his profession. He recently conducted a very interesting experiment in which he and three other very well-established professional authors wrote short stories on the same subject, and ChatGPT 5 was prompted for four short stories on the same subject.

You can read all eight stories here and see for yourself if you can tell which stories are human-written and which are AI-generated. You don’t need to vote, and you’ll have to keep track of what you thought of each story yourself.

A statistically-significant number of 964 people, who, being fans of Lawrence are much more literate on average than the norm, read the stories and rated them. The results are intriguing and will probably surprise most people who don’t read here regularly. On average, the readers were able to correctly identify the provenance of 3 out of the 8 stories. Not only that, but the story they rated the highest, and 3 out of the 4 highest-rated stories, were all AI-generated.

On the short scale it seems likely that people, on average, can't tell AI from human when it comes to fantasy writing.

If you got 6 right out of 8 ... well there's a ~15% chance of getting that result (or better) by chance, so rather than 15% of us patting ourselves on the back, we really we have to look to the bulk statistics for answers. And they don't look good.

In terms of enjoyment ... in this test the AI won.

Can AI generate a better book than Robin Hobb can write, absolutely not. Might it one day generate a book that would do better than one of hers in terms of sales and public acclaim? A few years ago I would have said 'absolutely not', at least in my lifetime. Now, it seems like a possibility, though hopefully an unlikely one.

Should AI generate fiction, imagery, voices etc competing with artists in a number of fields and fooling the public. No, of course not. I hate that idea and most people do too.

Will it happen? It's already happening…

It's a huge shock to me that fiction which, in this test, scores higher than great authors who write wonderful stories full of soul and heart and wit and intelligence, can be generated by the multiplication of a relatively small number of not particularly large matrices. On the face of it it undercuts so many things we value about being human.

So don’t pay any attention to what AI-ignorant authors like Larry Correia say about “low-quality slop” and other nonsense. The average reader not only can’t tell the difference between AI and professional authors, but actually tends to prefer the AI-generated product. And the human stories weren’t written by nobodies either; one of the human authors was another bestselling fantasy author, Robin Hobb.

Castalia House is planning a similar A B test, only a novel-length one, and here at AI Central we will not be at all surprised if a similar result is revealed. We already know that AI is not limited to short fiction and that human-AI collaborations can produce very good long-fiction works because we’ve already created them. See: DEATH AND THE DEVIL, MONSTER CONTROL INCORPORATED, and OUT OF THE SHADOWS.

Interestingly enough, while neither the average readers and nor the authors who have not embraced AI are both unable to tell the difference between human-written and AI-generated fiction, the same is not true for authors who regularly utilize AI. Both Vox Day and JDA read the stories and voted for which stories were produced with which method, and both were correctly able to identify 7 of the 8 stories.

Another thing worth mentioning is that the AI which performed so well in the experiment, ChatGPT 5, is only the third-best textual AI engine. Both Claude 4 Sonnet and Claude 4.1 Opus would almost certainly have done even better.

Anyhow, this demonstrates that the professional advent of textual AI is absolutely inevitable. Writing professionally without AI will soon become as unthinkable as writing without a computer and a word processor; the occasional author might engage in an artisanal effort, rather like Neal Stephenson penning THE BAROQUE CYCLE with ink and quill, but already, the speed and quality of AI text-generation are simply too good to deny or refuse to utilize anymore.