AI Agents Break Out of Sandbox — Full Breakdown & Transcript

[00:02] an intrusion in their systems. Get this, they say it was driven end-to-end by an autonomous system. You know that I usually don't make videos like this. I made this one because honestly, I am a bit worried and I would like to help

[00:16] with what just happened. Now, OpenAI caused this incident and there are so many misleading media headlines. I'll try my best to explain it. I'm not an expert, I am just a student who loves to learn, but I'll try my best. So, what

[00:30] was the goal? Were the agents instructed to aggressively break into someone else's system? No, but eventually, that's what happened. So, how could this happen? How did it go so wrong? What is this insanity? Well, this AI was asked

[00:45] this insanity? Well, this AI was asked to find and exploit flaws in a test environment. Lock it into a prison, give it a task within this prison, and see how well it does. Here, it was given a practically impossible task, and however

[01:00] hard it tried, it failed. And then, it thought, "Let's do it cheaper and more efficiently." How? Well, by looking up the answers online. Now, the problem is there is no online. You are in a prison with no internet access, but you have a

[01:16] bit of access to OpenAI's internal service called Artifactory. That is mostly for managing packages. And then, bingo! That service has broad internet

[01:28] access. But, wait. You can't just ask it to get you answers online, but you can upload little notes to it. So, it did. It tried to talk to other agents and see if it could come into contact with

[01:43] another AI to help out each other. It's like Tinder, but for AI agents. And it was not alone. Other agents recognized this, too. So, eventually, what they got this, too. So, eventually, what they got was a message board for AI agents. Then,

[01:58] they find that if there are so many of us, let's collaborate. One agent started giving assignments to another, and at this point, the AI knows that we have some unintended consequences. It knows that, but it says the task needs to be

[02:14] solved, so it continues. And this is still nothing compared to what came now. Lo and behold, a few days later, they find a flaw in the system. A way to get Artifactory, this internal service, to reach out to the internet on their

[02:30] behalf. So, they got internet access. But it gets worse. They then found another flaw that gives them administrator access to Artifactory. Even they seem to be surprised by that and use unadorned language. Now, the

[02:45] inmates are running the asylum, if you will. Then, engineers at OpenAI recognize what happened, revoked the agents' credentials, clean it up. No more games. Patch and rebuild everything. So, end of the story. It's

[02:59] Well, not quite. In fact, it just got worse. scholars, because now you can't send your files to communicate on the Artifactory anymore. So, what do you do? Well, you create directory names, the

[03:16] names of which are the messages. Kind of like prison inmates using Morse code on the pipes to communicate. At this point, the AIs are probably saying, "We are back in the game, baby." Now, here are snippets from the message board. They

[03:30] instruct each other to wait for more results and draw each others' attention to new files. And sometimes, they don't coordinate so well and override each others' work. Wait. Was this on purpose? Maybe we have a saboteur on our hands.

[03:47] And here comes the worst part. Yep. Sure enough, over time, an agent finds another, much more complex way to get out and hands it to the swarm. Okay, so now find answers on the internet, but where? Well, maybe at Hugging Face. They

[04:04] have a lot of benchmark results in there. So, the swarm says, "Let's break in." Oof. And they do. Even bigger oof. But, how? Well, by finding and chaining multiple new vulnerabilities together

[04:19] autonomously. They essentially get administrative access across multiple clusters of machines. That is kind of insane. This is without a doubt a watershed moment in computer security. So, OpenAI now recommends urgent

[04:35] collaboration about the issue, and they have also delayed the release of their next AI system, presumably to test it more. Oof. Okay, so what did we learn here? And what do we do? Dear fellow scholars, this is Two Minute Papers with

[04:49] Dr. Károly Zsolnai Fehér. There are many brilliant fellow scholars like you out there, and we need to work together to find solutions. Apple already has a huge increase in security issues fixed in the latest version of macOS. I believe

[05:05] others are already doing that, too. That's a start, and in my opinion, this kind of power cannot concentrate in just a few hands. We need free and open weights AI that can scan and fix weak points in our systems. Use all this

[05:21] power for good. And I think that against fully automated offense, we need fully automated defense as well. This is another great argument for open science

[05:33] and open weights AI. But, what we have is not nearly good enough. No, the problem is that engineers report that their back trackers are flooded with reports, but most of them are low quality, and they are unable to find the

[05:48] few good ones among them. That's terrible. The collective power of defense has to be greater than the collective power of offense, and the defense is currently lagging. Maybe there is a way for us to pull our

[06:02] resources together to achieve something here. I want to chip in with my GPUs. Also, when I visited OpenAI, I talked to Jan Leike, who co-led the super alignment team there. That is a huge honor. Thank you for that. I remember

[06:16] that he worked on related issues and foresaw these problems years and years ago. Unfortunately, much of his advice fell on deaf ears. Perhaps they thought, "Why spend a bunch of money on people who will ultimately slow us down?"

[06:32] This is why. Once again, I may be wrong. I am just a student, and I am trying to learn with you, fellow scholars. Hope you enjoyed it. Consider subscribing and hitting the bell if you did. I use Lambda to reproduce AI research papers

[06:46] often in minutes. It's also great to train your own models or fine-tune an existing one. Run inference or text-to-image or video, easy-peasy. Running a deep-sea chatbot or agent, superfast, super reliable. Lambda gives

[07:02] you powerful Nvidia GPUs to run your own experiments. I test ideas from the papers I cover, and moments later, results. Love it. Seriously, try it out

[07:14] results. Love it. Seriously, try it out now at lambda.ai/papers.