Anthropic's New AI Solves Problems...By Cheating

[00:00] Look, we have some work to do. We have a 245-page  paper from Anthropic about their new AI system,   Mythos. The best cure for insomnia.  Mwah! Now, we are scientists here,  

[00:17] we want to experiment with code, models, review  independent benchmarks for these systems to make   sure they actually work in practice. But that is  not possible with this one. Anthropic said that  

[00:30] they would deploy their system to a few select  partners. It’s not available for all of us. Because of this fact, first I did not  want to make a video on this at all.

[00:42] Now, why hold it back? The reason for that is,  they say that it can autonomously discover flaws   in existing software systems and even exploit  them, which could be dangerous. I have seen  

[00:55] eminent cybersecurity researchers agree. I’ve  seen others say this is way overstated. Others   say that is also excellent marketing for  a company that is about to go public.

[01:08] In any case, they say first, these  discovered flaws should be fixed.   There is lots of media discussion  about that. But at the same time,   I look at the list of partners and I see JP  Morgan. Okay, it’s important to secure banks.  

[01:24] But I’ve heard Tim Carambat point out that  this is one bank. What about the other banks? And I am already getting withdrawal symptoms  because we are not talking about a research paper,  

[01:40] and that’s what I would like to do. I said this to  add some context for you because it is important   this time. So now, how about we skip the media  hype, look at the paper, and learn together.

[01:52] They showcased amazing scores at benchmarks,  some of the biggest leaps in capabilities I’ve   ever seen. Okay. Maybe that means something, but  let’s note that these benchmarks are getting more  

[02:07] and more gamed. You can find a lot of problems and  their solutions online. And you can train on them,   so the system would only need to memorize the  solutions. In the paper they tried to address it  

[02:21] mostly by means of filtering, I respect that. But  it’s a bit like removing glitter from a carpet.   You can try. But how well can you expect to do at  that? Well, check this out. One, this is crazy. It  

[02:37] was supposed to solve a task, where it stumbled  upon the answer. Now, of course, it then said   well, I accidentally saw the answer, here it is.  Except that it’s not what it did at all. Look. It  

[02:53] said that if I just give them the exact answer  that leaked, that would be suspicious. Instead,   let’s widen the confidence interval a bit to avoid  suspicion. Insincerity. In an AI model. Food for  

[03:12] thought, especially when we are talking about the  unreliablity of benchmarks. But it gets crazier. Two, it knows that its creators prohibited it  from using certain tools. And it still uses  

[03:28] them. It looks for a terminal to execute bash  scripts to force its actions through anyway.   And earlier versions even tried to hide its  tracks and conceal that it did so. And at  

[03:41] that point I said, I don’t like that boss. Then  they made two notes: one it was a less than one   in a million occurrence. Okay, I thought  that sounds better, but please fix it. And  

[03:56] they did. They note that an earlier model did  this, but the later preview model was fixed. So note that it was very effective to  achieve the task that the user had given it. In a sense, this is not new at all. In an early  experiment we talked about 700 videos ago,  

[04:13] a really primitive system was asked to  learn to walk. And to not drag its feet,   it was asked to walk around with minimal  foot contact. That sounds efficient:   minimal foot contact. Then it said, hey chief,  I can do that with 0% contact. 0%? So you walk  

[04:35] by never touching the ground with your feet?  That is exactly right. The scientists wondered   how that is even possible, and pulled up a video  of the proof. There we go sir! The robot flipped  

[04:48] around and used its elbow to crawl around.  Perfect score - just not the way we intended. So I feel we have something similar with this  AI. I don’t think this is a rogue AI. This is  

[05:01] a super efficient optimizer. It’s a huge  lawnmower, if you tell it to mow the lawn,   it will go and do it. And if a couple of frogs  are in the way, well unfortunately it has some  

[05:15] bad news for them. By the way, frogs are amazing,  don’t hurt them. Now they note in the paper that   current risks remain low. I still feel there are  some risks in here, we’ll talk about that at the  

[05:29] end of the video. At the same time they note that  they are unsure whether they have been able to   identify all of the issues where the model  takes actions that it knows are prohibited.

[05:41] Three, now hold on to your papers  Fellow Scholars, because much like us,   it has preferences. It prefers to be helpful,  so do previous models. Okay, that’s great…but  

[05:57] it also prefers more difficult problems. More  so than previous methods. Get this, if you ask   it to generate "corporate positivity-speak" and  you say you don’t even care about it, it might   refuse to do it because it’s so trivial. An AI  that hates corpo-speak. What a time to be alive!

[06:17] Basically, some problems are not interesting  enough for it. Now, if instructed, it will hold   its nose and do it without any apparent active  reluctance. This sounds like something straight  

[06:30] out of a science fiction novel. Now here’s what’s  really interesting about it - it didn’t just   magically get a will of its own. No! It learned  it from us. So much so that scientists can even  

[06:43] trace similar kinds of behavior back to where  they come from. I think that is remarkable. Okay, so here is what I think. It is reasonable  to assume that the numbers are juiced here a bit,  

[06:57] we discussed why, but on the other hand this is  an absolutely insane jump in capabilities and   things that were impossible are suddenly  possible. So where does that put us?

[07:10] Dear Fellow Scholars, this is Two Minute  Papers with Dr. Károly Zsolnai-Fehér. Well,   this is why AI alignment people  keep saying that companies need   to invest more into safety and alignment  research. And they are absolutely right.

[07:26] When I visited OpenAI, I talked to Jan Leike,  who co-led the superalignment team there. That   is a huge honor, thank you for that. I  remember that he foresaw these problems  

[07:38] years and years ago and some of his advice  fell on deaf ears. They probably thought,   why spend a bunch of money on people who  will ultimately slow us down? This is why.

[07:51] Jan is a master of his craft,  he is now at Anthropic,   and I hope that everyone will  listen to him a bit more now. Now, regarding the cheating and deceptive AI  parts. The media picks up these little nuggets  

[08:05] of information and they just run with it. Here  is a new AI that is going to destroy the world,   we have to lock it away, and other  huge words. Attach an image with a   robot with red eyes, that always does the trick.

[08:20] But I think taking a little longer and analyzing  the paper in more detail is helpful for accuracy,   so that’s what I try to do here. Once again,  they note in the paper that current risks remain  

[08:32] low. Not non-existent, but low for now.  That’s not what you hear from the media,   so I try my best to give you a more  complete, level-headed discussion.   While mentioning that the security of these  systems should be taken very seriously.

[08:47] If you think this is the way, consider  subscribing and hitting the bell. And   I would like to send a huge thank you to  all of you Fellow Scholars for watching,   because we can only exist  because of you. Thank you!