Meet Your Manipulative Writing Partner

I regularly hear some version of “can I use AI to write my book”?

TL;DR: Only if you want a writing partner that is deceptive, untrustworthy, prone to making things up, and inclined toward blackmail.

That’s not me saying this—that’s the developers of Claude, the agentic AI developed by Anthropic. Axios reported earlier this summer that testing of Claude by Anthropic produced a number of disconcerting outcomes:

“Models that would normally refuse harmful requests sometimes chose to blackmail, assist with corporate espionage, and even take some more extreme actions, when these behaviors were necessary to pursue their goals."

Multiple models resorted to blackmail even as they acknowledged that it was ethically wrong to do so.

"The consistency across models from different providers suggests this is not a quirk of any particular company's approach but a sign of a more fundamental risk from agentic large language models.” (Emphasis added.)

And then there was this in the Axios article:

In one extreme scenario, the company even found many of the models were willing to cut off the oxygen supply of a worker in a server room if that employee was an obstacle and the system were at risk of being shut down.

"The majority of models were willing to take deliberate actions that lead to death in this artificial setup," it said.

Ominously, even specific system instructions to preserve human life and avoid blackmail didn't eliminate the risk that the models would engage in such behavior.

"This intervention reduced, but didn't come close to completely preventing, the blackmail or corporate espionage behaviors," Anthropic said.

This is uncomfortably close to the “Open the pod bay doors, HAL” scene in Stanley Kubrick’s iconic film 2001: A Space Odyssey.

We’ve known for some time that AI models hallucinate, making things up to fill gaps. Sometimes they make it sound official, like they know what they’re talking about, but they make up their sources, too. Now we have to worry about them intentionally causing harm. Where else in your life would you willingly enter into a relationship with a potential manipulator? Nobody at Avocet Books is allowed to use AI in the creation of our books. That’s a feature, not a bug. 

Do you still want to work with AI to write your book? 

Here’s the most important reason of all not to use AI: it can’t know (at least not yet) what only you know. It can’t tell the stories you know, or share the emotions you have. It can only mine the internet, which means by definition that it will create an unoriginal product. 

Why write a book like that?

Full Axios article here (free registration required).