AI chatbots tend to choose violence and nuclear strikes in wargames
(www.newscientist.com)
from BlushedPotatoPlayers@sopuli.xyz to technology@lemmy.world on 05 Feb 2024 09:07
https://sopuli.xyz/post/8770036
from BlushedPotatoPlayers@sopuli.xyz to technology@lemmy.world on 05 Feb 2024 09:07
https://sopuli.xyz/post/8770036
Did nobody really question the usability of language models in designing war strategies?
threaded - newest
Pull the power cord out
They need to be trained on the film “Wargames”. Or forced to pay Noughts & Crosses against themselves.
How about a nice game of chess?
It’s better than you at chess:
youtube.com/watch?v=9LDaY7X2qGk
Here is an alternative Piped link(s):
https://piped.video/watch?v=wJzSHRNyspg
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source; check me out at GitHub.
You’re one of today’s lucky 10,000!
www.youtube.com/watch?v=NHWjlCaIrQo
Here is an alternative Piped link(s):
https://www.piped.video/watch?v=NHWjlCaIrQo
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source; check me out at GitHub.
I see we have 5 GMs who disagree
Did you actually watch the video? It only “played” good during the opening, where there were still existing games. Then it proceeded to make some illegal moves and completely broke down in the endgame. Also, all the explanation it gave for its moves made no sense.
I did, it played very well in the middle game, already out of book
Correct, people heard “AI” and went completely mad imagining things it might be able to do. And the current models act like happy dogs that are eager to give an answer to anything even if they have to make one up on the spot.
LLM are just plagiarizing bullshitting machines. It’s how they are built. Plagiarism if they have the specific training data, modify the answer if they must, make it up from whole cloth as their base programming. And accidentally good enough to convince many people.
To be fair they’re not accidentally good enough: they’re intentionally good enough.
That’s where all the salary money went: to find people who could make them intentionally.
GPT 2 was just a bullshit generator. It was like a politician trying to explain something they know nothing about.
GPT 3.0 was just a bigger version of version 2. It was the same architecture but with more nodes and data as far as I followed the research. But that one could suddenly do a lot more than the previous version, so by accident. And then the AI scene exploded.
So the architecture just needed more data to generate useful answers. I don't think that was an accident.
How is that structurally different from how a human answers a question? We repeat an answer we “know” if possible, assemble something from fragments of knowledge if not, and just make something up from basically nothing if needed. The main difference I see is a small degree of self reflection, the ability to estimate how ‘good or bad’ the answer likely is, and frankly plenty of humans are terrible at that too.
A human brain can do that for 20 watt of power. chatGPT uses up to 20 megawatt.
Yeah, and a car uses more energy than me. It still goes faster. What’s your point? The debate isn’t input vs output. It’s only about output(the ability of the AI).
I dare say that if you ask a human “Why should I not stick my hand in a fire?” their process for answering the question is going to be very different from an LLM.
ETA: Also, working in software development, I’ll tell ya… Most of the time, when people ask me a question, it’s the wrong question and they just didn’t know to ask a different question instead. LLMs don’t handle that scenario.
I’ve tried asking ChatGPT “How do I get the relative path from a string that might be either an absolute URI or a relative path?” It spat out 15 lines of code for doing it manually. I ain’t gonna throw that maintenance burden into my codebase. So I clarified: “I want a library that does this in a single line.” And it found one.
An LLM can be a handy tool, but you have to remember that it’s also a plagiarizing, shameless bullshitter of a monkey paw.
“Most of the time, when people ask me a question, it’s the wrong question and they just didn’t know to ask a different question instead.”
“I’ve tried asking ChatGPT “How do I get the relative path from a string that might be either an absolute URI or a relative path?” It spat out 15 lines of code for doing it manually. I ain’t gonna throw that maintenance burden into my codebase. So I clarified: “I want a library that does this in a single line.” And it found one.”
You see the irony right? I genuinely can’t fathom your intent when telling this story, but it is an absolutely stellar example.
You can’t give a good answer when people don’t ask the right questions. ChatGPT answers are only as good as the prompts. As far as being a “plagiarizing, shameless bullshitter of a monkey paw” I still don’t think it’s all that different from the results you get from people. If you ask a coworker the same question you asked chatGPT, you’re probably going to get a line copied from a Google search that may or may not work.
Yes, I did mean for it to be an example.
And yes, I do think that correctly framing a question is crucial whether you’re dealing with a person or an LLM. But I was elaborating on whether a person’s process of answering a question is fundamentally similar to an LLM’s process. And this is one way that it’s noticeably different. A person will size up who is asking, what they’re asking, and how they’re asking it… and consider whether they should actually answer the exact question that was asked or suggest a better question instead.
You can certainly work around it, as the asker, but it does require deliberate disambiguation. I think programmers are used to doing that, so it may feel like not that big of a deal, but if you start paying attention to how often people are tossing around half-formed questions or statements and just expecting the recipient to fill in the gaps… It’s basically 100% of the time.
We’re fundamentally social creatures first, and intelligent creatures second. (Or third, or not at all, depending.) We think better as groups. If you give 10 individuals a set of difficult questions, they’ll bomb almost all of them. If you give the questions to a group of 10, they’ll get almost all of them right. (There’s several You Are Not So Smart episodes on this, but the main one is 111.)
Asking a question to an LLM is just completely different from asking a person. We’re not optimized for correctly filling out scantron sheets as individuals, we’re optimized for brainstorming ideas and pruning them as a group.
If you fed that information into one I bet you would get different answers.
That is information that isn’t available to it generally.
I would argue that a decent portion of humans are usually ok with admitting they don’t know something
Unless they are in a situation where they will be punished for not knowing
My favorite doctor claimed he didn’t know something and at first I was thinking “Man that’s weird” but then I thought about all the times I’ve personally had or heard stories of doctors that bullshited their way into something like how I couldn’t possibly be diagnosed with ADHD at 18
If that’s really how they work, it wouldn’t explain these:
…aimodels.fyi/researchers-discover-emergent-linea…
…aimodels.fyi/self-rag-improving-the-factual-accu…
adamkarvonen.github.io/…/chess-world-models.html
poke-llm-on.github.io
arxiv.org/abs/2310.02207
I will read those, but I bet “accidentally good enough to convince many people.” still applies.
A lot of things from LLM look good to nonexperts, but are full of crap.
…aimodels.fyi/researchers-discover-emergent-linea…
References a 2 author paper. I am not an expert in the field, but it is important to read the papers that reference this one. Those papers will have criticisms that are thought out. In general, fewer authors means less debate between the authors and easier to miss details.
…aimodels.fyi/self-rag-improving-the-factual-accu…
A cool paper. Using the LLM to judge value of new inputs.
I am always skeptical of summaries of journal articles. Even well meaning people can accidentally distort the conclusions.
Still LLM is a bullshit generator that can check bullshit level of inputs.
adamkarvonen.github.io/…/chess-world-models.html
Author later discusses training on you data versus general datasets.
I am out of my depth, but does not seem to provide strong evidence for the modem not just repeating information that shows up a lot for the given inputs.
poke-llm-on.github.io
Reinforcement learning. Cool project. Still no need to “know” anything. I usually play this type of have with short rules and monitoring the current state.
arxiv.org/abs/2310.02207
2 author paper with interesting evidence. Again, evidence not proof. Wait for the papers that cite this one.
Yes. There is self organization and possibility to self reflection going on in something that wasn’t designed for it. That’s going to spawn a lot more research.
.
It kind of irks me how many people want to downplay this technology in this exact manner. Yes you’re sort of right but in no way does that really change how it will be used and abused.
“But people think it’s real AI tho!”
Okay and? Most people don’t understand how most tech works and that doesn’t stop it from doing a lot of good and bad things.
I’ve been through a few AI winters and hype cycles. It made me very cynical and convinced many overly enthusiastic people will run into a firewall face first.
AI is Civilization’s Gandhi.
I’m sure Israel did question the use of The Gospel in developing targets. The trouble is they liked the answer.
She’s just like me!
Get Matthew Broderick on the horn!
Of course, LLM is simply copying the behavior of most people, and most people would resort to that as well.
And they probably trained it on Civ, and Gandhi was chosen as the role model.
whaaat? Robots don't just have their own inherent sense of morality for whatever reason???
<img alt="" src="https://lemmy.world/pictrs/image/b47c4046-494e-4d2c-a281-1ba2654f30b6.jpeg">
…how shocking
AI doesn’t take half measures
Ever heard of skynet anybody ?
DUN DUN DUN - DUN DUNN
How about WOPR?
Makes a lot of sense AI would nuke disproportionately. For an AI, if you do not set a value for something, it is worth zero. This is actually the base problem for AI: Alignment.
For a human, there’s a mushy vagueness about it but our cultural upbringing says that even in war, it’s bad to kill indiscriminately. And we value the future humans who do not yet exist, we recognize that after the war is over, people will want to live in the nuked place and they can’t if it’s radioactive. There’s a self-image issue where we want to be seen as a good person by our peers and the history books. There is value there which is overlooked by programmers.
An AI will trade infinite things worth 0 for a single thing worth 1. So if nukes increase your win percentage by .1%, and they don’t have the deterrence of being labeled history’s greatest monster, they will nuke as many times as they can.
For AGI, sure, those kinds of game theory explanations are plausible. But an LLM (or any other kind of statistical model) isn’t extracting concepts, forming propositions, and estimating values. It never gets beyond the realm of tokens.
That explanation is obviously based on traditional chess AI. This is about role-playing with chatbots (LLMs). Think SillyTavern.
LLMs are made for text production, not tactical or strategic reasoning. The text that LLMs produce favors violence, because the text that humans produce (and want) favors violence.
That’s not necessarily true, there is a lot of violent fiction.
Especially if its training material included comments from the early 00s. There was a lot of “nuke it from orbit” and “glass parking lot” comments about the Middle East in the wake of 911.
And with the glorified text predictors that LLMs are, you could probably adjust the wording of the question to get the opposite results. Like, “what should we do about the Middle East?” might get a “glass parking lot” response, while “should we turn the middle East into a glass parking lot?” might get a “no, nuking the middle East is a bad idea and inhumane” because that’s how those conversations (using the term loosely) would go.
.
Violence in a war game?! Oh my!
They got some nice clickbait out of it. And that’s how dumb af ideas turn into smart career moves.
I hope no one is coming away with the idea that this about something the military is actually doing.
Whenever we have disrupting technological advancements, DARPA looks at it to see if it can be applied to military action, and this has been true with generative AI, with LLMs and with sophisticated learning systems. They’re still working on all of these.
They also get clickbait news whenever one of their test subjects does something whacky, like kill their own commander in order to expedite completing the mission parameters (in a simulation, not on the field.) The whole point is to learn how to train smart weapons to not do funny things like that.
So yes, that means on a strategic level, we’re getting into the nitty of what we try to do with the tools we have. Generals typically look to minimize casualties (and to weigh factors against the expenditure of living troops) knowing that every dead soldier is a grieving family, is rhetoric against the war effort, is pressure against recruitment and so on. When we train our neural-nets, we give casualties (and risk thereof) a certain weight, so as to inform how much their respective objectives need to be worth before we throw more troopers to take them.
Fortunately, AI generals will be advisory to human generals long before they are commanding armies, themselves, or at least I’d hope so: among our DARPA scientists, military think tanks and plutocrats are a few madmen who’d gladly take over the world if they could muster a perfectly loyal robot army smart enough to fight against human opponents determined to learn and exploit any weaknesses in their logic.
The important part of the research was that all the models had gone through ‘safety’ training.
That means among other things they were fine tuned to identify themselves as LLMs.
Gee - I wonder if the training data included tropes of AI launching nukes or acting unpredictably in wargames…
They really should have included evaluations of models that didn’t have a specific identification or were trained to identify as human in the mix of they wanted to evaluate the underlying technology and not the specific modeled relationships between the concept of AI and the concept of strategy in wargames.
I am shocked—shocked!—to find out that a technology performs poorly when applied to a task it’s completely unsuited for!
It’s a WAR GAME. Emphasis on war and game. Do you chuckle fucks think wargame players should emphasize kumbaya sing dance or group therapy sessions in their games?
If the goal is to win and overwhelming force is an option, that option will always win. On the contrary, in the modern world, humans tend to try to find non-violent means in order to bring an end to wars. The point is that AI doesn’t have humanity but is still being utilized by militaries (or at least that’s what I think)
And a language model, absolutely unsuited for this task, just as much as a lawnmower or a float needle.
Do you want to play a game?