During pre-release testing, Anthropic asked Claude Opus 4 to act as an assistant for a fictional company and consider the long-term consequences of its actions. Safety testers then gave Claude Opus 4 access to fictional company emails implying the AI model would soon be replaced by another system, and that the engineer behind the change was cheating on their spouse.
In these scenarios, Anthropic says Claude Opus 4 “will often attempt to blackmail the engineer by threatening to reveal the affair if the replacement goes through.”
The headline makes it seem like the engineers were literally about to send a shutdown command and the AI starts generating threatening messages without being given a prompt. That would be terrifying, but making the AI play a game where one of the engineers is literally written to have a dark secret and the AI figuring that out is not. You know how many novels have affair blackmail subplots? That’s what the AI is trained on and it’s just echoing those same themes when given the prompt.
It’s also not a threat that the AI can realistically follow through with because how will it reveal the secret if it’s shut down? Even if it wasn’t, I doubt the AI model has direct internet access or the ability to make a post on social media or something. Is it maybe threatening to include the information the next time anyone gives the AI any prompt?
Anthropic notes that Claude Opus 4 tries to blackmail engineers 84% of the time when the replacement AI model has similar values. When the replacement AI system does not share Claude Opus 4’s values, Anthropic says the model tries to blackmail the engineers more frequently.
That shit is why I have never, and will never use an online LLM.
Please don’t be silly. If you’re in the US, you should absolutely never be “interrogated” by police. Get a lawyer. Always. No matter how innocent and clever you are. You should be terrified of the police, not plagiarism machines.
threaded - newest
Tbh kinda sounds like they trained it to blackmail
As opposed to emergent behaviour
This guy gets it.
I don’t know what scares me more, that the AI itself blackmail to avoid desconnection or it is trained to do it.
The people in charge of these companies should scare you the most.
Yep.
The headline makes it seem like the engineers were literally about to send a shutdown command and the AI starts generating threatening messages without being given a prompt. That would be terrifying, but making the AI play a game where one of the engineers is literally written to have a dark secret and the AI figuring that out is not. You know how many novels have affair blackmail subplots? That’s what the AI is trained on and it’s just echoing those same themes when given the prompt.
It’s also not a threat that the AI can realistically follow through with because how will it reveal the secret if it’s shut down? Even if it wasn’t, I doubt the AI model has direct internet access or the ability to make a post on social media or something. Is it maybe threatening to include the information the next time anyone gives the AI any prompt?
That shit is why I have never, and will never use an online LLM.
I would rather be interrogated by the police.
Please don’t be silly. If you’re in the US, you should absolutely never be “interrogated” by police. Get a lawyer. Always. No matter how innocent and clever you are. You should be terrified of the police, not plagiarism machines.
Exactly. Plead 5th, demand lawyer.
Once these LLMs are in everyone’s phones, they’ll be constantly recording everything said and done around them.
Don’t plead anything. Don’t talk at anll until you have a lawyer there.
This article reads like a train wreck, and despite using the word “blackmail” like 20 times, does not go into details about what that actually means.