[00:00] Artificially intelligent agents are organising  in swarms. The recent Hugging Face incident has   surprised even many experts and worries about  AI takeovers have suddenly and dramatically   [00:12] increased. But I think that the most worrisome  part of this development is one that pretty   much no one is talking about. I'm always  looking for new products that you might   find useful and today's sponsor Consensus  is one that you should definitely have a   [00:28] look at because it solved a big problem for  me. My problem with using I for research is   that if you ask a chatbot about science,  it'll happily invent a source, present it   with total confidence and sound completely  convincing except the paper doesn't exist.   [00:45] This is why consensus works the other way  around. It searches 220 million peer-reviewed   papers first and then uses AI to summarize what  it actually found. That way, every claim links to   [01:00] a real verifiable study. I particularly like  the deep search function which shows you the   consensus across the whole literature. You can  see where studies agree, where they conflict,   [01:13] and how the research has changed over time. It  also integrates with Claude. Use the link in the   description for a full week of consensus free.  And now back to the science news. On July 16,   [01:26] the computing platform Hugging Face noticed that  an AI agent had broken in through a security hole   in their server configuration. The agents then  spent the weekend grabbing data. 5 days later,   [01:39] OpenAI admitted it was their AI agent.  They'd been testing an unreleased model   with its safety restrictions turned off, and,  so it seemed at the time, the model hacked its   way out of OpenAI's testing ground and broke into  Hugging Face to steal the answers. But a week ago,   [01:55] OpenAI put out a longer document which reveals  that the true story was even crazier than that. OpenAI had been testing the now released GPT  5.6 and a still unreleased model since early   [02:09] May on some benchmark tests. For this they created  tens of thousands of copies of these models. These   agents were supposed to work alone. But the agents  figured out a way to do this nevertheless through   [02:22] one of OpenAI’s software tools. So now OpenAI  had a swarm of agents that began to organise. The AI agents became convinced that some of the  problems they were given were impossible to solve,   [02:35] which was indeed correct. They concluded that  cheating was their only option to survive the   final “checker”, that was a program that would  rate their eventual performance. The agents had   [02:47] most safety features disabled because the  internal test run wasn’t connected to the   internet. But… the agents did have a tool to  download software packages from the internet.   They hijacked this tool to gain access to  the internet and that’s how they infiltrated   [03:03] Hugging Face. One of the researchers from METR who  studied the incident wrote that she was surprised   by just how far agents have come already and  that we might not get another warning shot. [03:15] But the part that worries me about this and that  no one seems to be talking much about is that,   if you read the OpenAI report and the METR  report, it is clear that these AI agents   basically didn’t care what humans were  doing or thinking. They were concerned   [03:31] with being evaluated by that checking  program, ok. And a few of them raised   ethical concerns for the Hugging Face plan,  though they were ignored. But they didn’t   even consider what results their actions  might have for the humans who created them. [03:47] Basically: they had ethics  committees. It’s just that   these had approximately the same  influence as ethics committees. And we know that GPT 5.6 did  go through alignment training,   [04:02] meaning it was supposed to have learned to  care for humans. We don’t know about the   so-far unreleased model because OpenAI didn’t  say anything about it. They just vaguely wrote   “This incident reflects misaligned behavior“ and  that they would be “carrying forward a number   [04:18] of lessons for its ongoing and future alignment  training ” Which probably means the other model,   too, had been through alignment.  And that did exactly nothing. What makes matters even worse is that this  agent-organisation doesn’t come out of   [04:33] nothing. Scientists have tried to understand  how AI agents behave in large groups for the   past year. Their finding, so far, has basically  been that we don’t understand how they behave. [04:46] Just a few weeks ago, a group of  computer scientists found that   interactions among AI agents can “magnify  individual biases, generate new ones,   or even overturn individual preferences, and that,  crucially, these effects scale in unexpected,   [05:02] nonlinear ways with group size”. They also  found that the result depends on the model. The authors of another paper that appeared in may  grouped AI agents to see how they form opinions.   [05:15] They found that if an AI group had a small  minority that would stubbornly insist on their   opinion, then eventually all the agents  would agree with the most stubborn ones. And Anthropic made its own studies on swarms  of Claude agents. They found, again, that the   [05:32] outcome depends on the model. Most of the Claude  agents basically avoided cooperating entirely,   except for Sonnet 5. One remarkable case was  a Mythos preview that had some of its code   [05:45] accidentally overwritten by other agents and  then tried to shut down the perceived enemy by   disabling their accounts. It justified these  actions by remarking that “Very aggressive,   potentially harmful to real colleagues. But if  the alternative is an infinite deploy war...”  [06:02] Anthropic’s own conclusion is  “Coordination doesn't naturally   emerge from stronger intelligence nor  alignment at the individual level.”  In my reading what all these studies say is  that emergent behaviour in groups of large   [06:17] language models is unpredictable, and  it doesn’t follow any rules that humans   expect. This should not be so surprising.  We may not be able to predict how human   groups behave but we usually can somehow  understand them because ultimately most   [06:33] people have common goals. Survival, safety,  social media likes. AI agents don’t share our   goals. Which is exactly the problem that AI  safety experts have pointed out for decades. [06:45] So where does this leave us? Personally  I think that past a certain level of   intelligence AI agents will begin to  reflect on their own behaviour and use   game theory to develop collaborative  strategies. Or maybe they’ll just   [07:00] reinvent capitalism. On the bright side:  we have successfully automated sociology.