How we Rooted Copilot (research.eye.security)
from Pro@programming.dev to technology@lemmy.world on 27 Jul 14:48
https://programming.dev/post/34619779

#technology

threaded - newest

nathan@piefed.alphapuggle.dev on 27 Jul 16:38 next collapse

$10 says they haven't actually escaped anything and it's just hallucinating a directory structure & file contents

MagicShel@lemmy.zip on 27 Jul 16:57 next collapse

Even if it had access to its own source during training, the chances of it regurgitating it with total fidelity are zero.

communism@lemmy.ml on 27 Jul 18:04 collapse

MS said they fixed it and categorised it as a “moderate severity vulnerability” so presumably they did in fact gain root access to the container

wewbull@feddit.uk on 27 Jul 20:10 collapse

If they gained root access to the container, that’s not a moderate vulnerability. Root inside a container is still root. You can still access the kernel with root privs and it’s the same kernel as the host.

Docker is not a virtual machine.

communism@lemmy.ml on 27 Jul 23:59 next collapse

I know that? I’m just saying that MS categorised it as such. It would be strange to include the part about MS’s responses if MS also found that the vulnerability was not what the researchers claimed it was.

wewbull@feddit.uk on 28 Jul 08:44 collapse

What I’m saying is something about the story doesn’t add up.

Either Microsoft classified a major issue as a minor one so they didn’t have to payout the bug bounty (quite possible), or the attack didn’t achieve what the researchers thought it did and Microsoft classified it according to it’s actual results.

trolololol@lemmy.world on 28 Jul 09:00 collapse

If I have to choose between either ms or an unknown being correct, I pick the unknown person.

Fizz@lemmy.nz on 28 Jul 11:46 next collapse

I think they gained root to the python env which they couldn’t do anything with because it was still running in docker inside a VM.

  • According to a smart sounding fella on hacker news.
ftbd@feddit.org on 28 Jul 12:40 next collapse

That assumes the container itself is run as root, right?

Grappling7155@lemmy.ca on 28 Jul 18:22 collapse

Docker isn’t, but I was under the impression that hyperscalars tended to put all their containers in lightweight VMs or use something like kata containers anyways for security purposes

ignirtoq@fedia.io on 27 Jul 17:01 next collapse

Several years ago I created a Slack bot that ran something like Jupyter notebook in a container, and it would execute Python code that you sent to it and respond with the results. It worked in channels you invited it to as well as private messages, and if you edited your message with your code, it would edit its response to always match the latest input. It was a fun exercise to learn the Slack API, as well as create something non-trivial and marginally useful in that Slack environment. I knew the horrible security implications of such a bot, even with the Python environment containerized, and never considered opening it up outside of my own personal use.

Looks like the AI companies have decided that exact architecture is perfectly safe and secure as long as you obfuscate the input pathway by having to go through a chat-bot. Brilliant.

BaroqueInMind@piefed.social on 27 Jul 17:38 next collapse

And so Microsoft decided this wasn't a big enough vulnerability to pay them a bounty. Why the fuck would you ever share that with them then, if you could sell it to a black-hat hacking org for thousands?

fmstrat@lemmy.nowsci.com on 28 Jul 01:47 collapse

There may not have been any logical progression beyond the container.

deadcade@lemmy.deadca.de on 29 Jul 00:21 collapse

Surely there wasn’t an exploit on the half a year out of date kernel (Article screenshots from April 2025, uname kernel release from a CBL-Mariner released September 3rd 2024).

Bubbey@lemmy.world on 28 Jul 16:30 collapse

I’m sure nothing will go wrong with tons of critical business documents being routed through copilot for organizations…