People can’t tell the difference between human and AI-generated poetry – new study

Has the bell finally tolled for Shakespeare and Byron? New research conducted by philosophers of science Brian Porter and Edouard Machery suggests that the latest AI-generated poetry is “indistinguishable from human-written poetry” and “rated more favourably”.

Ten poets, from the medieval Geoffrey Chaucer to modern writer Dorothea Lasky, were successfully impersonated by AI chatbots, with most of the 696 participants slightly preferring the imitation to the real thing.

Porter and Machery conclude that “the capabilities of generative AI models have outpaced people’s expectations of AI”. But they don’t say AI has been proven an adequate replacement for human poets – and rightly so, as such a conclusion would require a great deal more testing.

That the research participants were fooled is not particularly worrying. Porter and Machery set out to include a wide range of poem types, which meant choosing poets who mostly belong to ages past. In such cases, modern readers are likely to have a hard time looking past the obvious signs of antiquity – outdated diction, rigid formalism, and obscure cultural references. It’s not so hard to disguise yourself as someone when that person is chiefly known for the odd clothes they wear.

Looking for something good? Cut through the noise with a carefully curated selection of the latest releases, live events and exhibitions, straight to your inbox every fortnight, on Fridays. Sign up here.

But what about the matter of preference? As well as overall quality, the researchers asked participants to rate poems on a range of qualitative dimensions. How was the imagery, rhythm, sound or beauty? How “inspiring”, “lyrical”, “meaningful”, “moving”, “original”, “profound”, “witty” (and so on) was it? AI won out over Shakespeare and company in nearly every category.

Does this mean human poets have been supplanted? Not really. Participants in the research overall reported “a low level of experience with poetry”. Lack of familiarity with any artform severely limits our ability to get the most out of it. All the AI has to do is sand off the more challenging elements – ambiguity, wordplay, linguistic complexity – in order to produce a version which is more palatable to those with little interest in the art.

If that sounds snobbish, think of it this way: when we aren’t used to eating a foreign cuisine, most of us gravitate toward the blandly familiar end of the menu.

But poetry is not a medium many look to for instant gratification. The reason poets of the calibre of Byron and Walt Whitman (both of whom were included in this study) continue to command respect is because their poetry rewards extended, rather than cursory, attention. The report agrees on this point, noting that participants complained more often of the human-authored poems that they “don’t make sense”.

For now, then, poets have little reason to fret. Is it possible, though, that we aren’t too far off the point where seasoned readers of poetry are able to discover a richness and depth in AI poetry that outstrips similar efforts by humans? I think so – not least because a substantial contributor to the emotional and intellectual impact of a poem is the reader’s own imagination.

It is the reader who, through the act of reading, brings the words to life. For decades now, the concept of “found poetry” – as well as collage poetry and other related techniques – has rested on the fact that all language can be recontextualised as poetry, if arranged with care. For the skilled reader of poetry, the poem is a construction kit, or playground, for the mind to revel in.

But we must then ask, how many readers will choose to repeatedly commit the time and effort needed to draw meaning from AI-created texts? Is the pleasure of reading reward enough in itself?

For some, it will be. But I suspect for the majority, the real point of poetry is to put you in touch, in a very specific way, with other human minds. It is more social activity than technical feat.

In many cultures, the rituals that have grown around it are collaborative, participatory. Poetry is made not because we need poems to exist, but because we seek a keener, fuller awareness of each other – of the sense each of us makes of the world.

That does not mean AI won’t change poetry. Every recent generation of poets has been deeply interested in adapting and absorbing new technologies, along with shifts in cultural mood. Film poets continue to explore combinations of spoken word and moving image. Flarf poetry collected and reconfigured search engine detritus. And my own research into video game poetry has uncovered rapidly growing interest in a form of poetry that is restlessly interactive, playable, slippery.

Already, poets like Dan Power and Nick Flynn are collaborating in different ways with AI to uncover new avenues of possibility. And AI’s ability to approximate Shakespeare’s style is a technological marvel.

But art that merely imitates and iterates on what has come before is art at its most trivial. The goal of the poet is not to be mistaken for Shakespeare, but much the opposite: to make something never seen before.

Jon Stone does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.