[00:00] Look, we have some work to do. We have a 245-page&nbsp; paper from Anthropic about their new AI system,&nbsp;&nbsp; Mythos. The best cure for insomnia.&nbsp; Mwah! Now, we are scientists here,&nbsp;&nbsp;
[00:17] we want to experiment with code, models, review&nbsp; independent benchmarks for these systems to make&nbsp;&nbsp; sure they actually work in practice. But that is&nbsp; not possible with this one. Anthropic said that&nbsp;&nbsp;
[00:30] they would deploy their system to a few select&nbsp; partners. It’s not available for all of us. Because of this fact, first I did not&nbsp; want to make a video on this at all.
[00:42] Now, why hold it back? The reason for that is,&nbsp; they say that it can autonomously discover flaws&nbsp;&nbsp; in existing software systems and even exploit&nbsp; them, which could be dangerous. I have seen&nbsp;&nbsp;
[00:55] eminent cybersecurity researchers agree. I’ve&nbsp; seen others say this is way overstated. Others&nbsp;&nbsp; say that is also excellent marketing for&nbsp; a company that is about to go public.
[01:08] In any case, they say first, these&nbsp; discovered flaws should be fixed.&nbsp;&nbsp; There is lots of media discussion&nbsp; about that. But at the same time,&nbsp;&nbsp; I look at the list of partners and I see JP&nbsp; Morgan. Okay, it’s important to secure banks.&nbsp;&nbsp;
[01:24] But I’ve heard Tim Carambat point out that&nbsp; this is one bank. What about the other banks? And I am already getting withdrawal symptoms&nbsp; because we are not talking about a research paper,&nbsp;&nbsp;
[01:40] and that’s what I would like to do. I said this to&nbsp; add some context for you because it is important&nbsp;&nbsp; this time. So now, how about we skip the media&nbsp; hype, look at the paper, and learn together.
[01:52] They showcased amazing scores at benchmarks,&nbsp; some of the biggest leaps in capabilities I’ve&nbsp;&nbsp; ever seen. Okay. Maybe that means something, but&nbsp; let’s note that these benchmarks are getting more&nbsp;&nbsp;
[02:07] and more gamed. You can find a lot of problems and&nbsp; their solutions online. And you can train on them,&nbsp;&nbsp; so the system would only need to memorize the&nbsp; solutions. In the paper they tried to address it&nbsp;&nbsp;
[02:21] mostly by means of filtering, I respect that. But&nbsp; it’s a bit like removing glitter from a carpet.&nbsp;&nbsp; You can try. But how well can you expect to do at&nbsp; that? Well, check this out. One, this is crazy. It&nbsp;&nbsp;
[02:37] was supposed to solve a task, where it stumbled&nbsp; upon the answer. Now, of course, it then said&nbsp;&nbsp; well, I accidentally saw the answer, here it is.&nbsp; Except that it’s not what it did at all. Look. It&nbsp;&nbsp;
[02:53] said that if I just give them the exact answer&nbsp; that leaked, that would be suspicious. Instead,&nbsp;&nbsp; let’s widen the confidence interval a bit to avoid&nbsp; suspicion. Insincerity. In an AI model. Food for&nbsp;&nbsp;
[03:12] thought, especially when we are talking about the&nbsp; unreliablity of benchmarks. But it gets crazier. Two, it knows that its creators prohibited it&nbsp; from using certain tools. And it still uses&nbsp;&nbsp;
[03:28] them. It looks for a terminal to execute bash&nbsp; scripts to force its actions through anyway.&nbsp;&nbsp; And earlier versions even tried to hide its&nbsp; tracks and conceal that it did so. And at&nbsp;&nbsp;
[03:41] that point I said, I don’t like that boss. Then&nbsp; they made two notes: one it was a less than one&nbsp;&nbsp; in a million occurrence. Okay, I thought&nbsp; that sounds better, but please fix it. And&nbsp;&nbsp;
[03:56] they did. They note that an earlier model did&nbsp; this, but the later preview model was fixed. So note that it was very effective to&nbsp; achieve the task that the user had given it. In a sense, this is not new at all. In an early&nbsp; experiment we talked about 700 videos ago,&nbsp;&nbsp;
[04:13] a really primitive system was asked to&nbsp; learn to walk. And to not drag its feet,&nbsp;&nbsp; it was asked to walk around with minimal&nbsp; foot contact. That sounds efficient:&nbsp;&nbsp; minimal foot contact. Then it said, hey chief,&nbsp; I can do that with 0% contact. 0%? So you walk&nbsp;&nbsp;
[04:35] by never touching the ground with your feet?&nbsp; That is exactly right. The scientists wondered&nbsp;&nbsp; how that is even possible, and pulled up a video&nbsp; of the proof. There we go sir! The robot flipped&nbsp;&nbsp;
[04:48] around and used its elbow to crawl around.&nbsp; Perfect score - just not the way we intended. So I feel we have something similar with this&nbsp; AI. I don’t think this is a rogue AI. This is&nbsp;&nbsp;
[05:01] a super efficient optimizer. It’s a huge&nbsp; lawnmower, if you tell it to mow the lawn,&nbsp;&nbsp; it will go and do it. And if a couple of frogs&nbsp; are in the way, well unfortunately it has some&nbsp;&nbsp;
[05:15] bad news for them. By the way, frogs are amazing,&nbsp; don’t hurt them. Now they note in the paper that&nbsp;&nbsp; current risks remain low. I still feel there are&nbsp; some risks in here, we’ll talk about that at the&nbsp;&nbsp;
[05:29] end of the video. At the same time they note that&nbsp; they are unsure whether they have been able to&nbsp;&nbsp; identify all of the issues where the model&nbsp; takes actions that it knows are prohibited.
[05:41] Three, now hold on to your papers&nbsp; Fellow Scholars, because much like us,&nbsp;&nbsp; it has preferences. It prefers to be helpful,&nbsp; so do previous models. Okay, that’s great…but&nbsp;&nbsp;
[05:57] it also prefers more difficult problems. More&nbsp; so than previous methods. Get this, if you ask&nbsp;&nbsp; it to generate "corporate positivity-speak" and&nbsp; you say you don’t even care about it, it might&nbsp;&nbsp; refuse to do it because it’s so trivial. An AI&nbsp; that hates corpo-speak. What a time to be alive!
[06:17] Basically, some problems are not interesting&nbsp; enough for it. Now, if instructed, it will hold&nbsp;&nbsp; its nose and do it without any apparent active&nbsp; reluctance. This sounds like something straight&nbsp;&nbsp;
[06:30] out of a science fiction novel. Now here’s what’s&nbsp; really interesting about it - it didn’t just&nbsp;&nbsp; magically get a will of its own. No! It learned&nbsp; it from us. So much so that scientists can even&nbsp;&nbsp;
[06:43] trace similar kinds of behavior back to where&nbsp; they come from. I think that is remarkable. Okay, so here is what I think. It is reasonable&nbsp; to assume that the numbers are juiced here a bit,&nbsp;&nbsp;
[06:57] we discussed why, but on the other hand this is&nbsp; an absolutely insane jump in capabilities and&nbsp;&nbsp; things that were impossible are suddenly&nbsp; possible. So where does that put us?
[07:10] Dear Fellow Scholars, this is Two Minute&nbsp; Papers with Dr. Károly Zsolnai-Fehér. Well,&nbsp;&nbsp; this is why AI alignment people&nbsp; keep saying that companies need&nbsp;&nbsp; to invest more into safety and alignment&nbsp; research. And they are absolutely right.
[07:26] When I visited OpenAI, I talked to Jan Leike,&nbsp; who co-led the superalignment team there. That&nbsp;&nbsp; is a huge honor, thank you for that. I&nbsp; remember that he foresaw these problems&nbsp;&nbsp;
[07:38] years and years ago and some of his advice&nbsp; fell on deaf ears. They probably thought,&nbsp;&nbsp; why spend a bunch of money on people who&nbsp; will ultimately slow us down? This is why.
[07:51] Jan is a master of his craft,&nbsp; he is now at Anthropic,&nbsp;&nbsp; and I hope that everyone will&nbsp; listen to him a bit more now. Now, regarding the cheating and deceptive AI&nbsp; parts. The media picks up these little nuggets&nbsp;&nbsp;
[08:05] of information and they just run with it. Here&nbsp; is a new AI that is going to destroy the world,&nbsp;&nbsp; we have to lock it away, and other&nbsp; huge words. Attach an image with a&nbsp;&nbsp; robot with red eyes, that always does the trick.
[08:20] But I think taking a little longer and analyzing&nbsp; the paper in more detail is helpful for accuracy,&nbsp;&nbsp; so that’s what I try to do here. Once again,&nbsp; they note in the paper that current risks remain&nbsp;&nbsp;
[08:32] low. Not non-existent, but low for now.&nbsp; That’s not what you hear from the media,&nbsp;&nbsp; so I try my best to give you a more&nbsp; complete, level-headed discussion.&nbsp;&nbsp; While mentioning that the security of these&nbsp; systems should be taken very seriously.
[08:47] If you think this is the way, consider&nbsp; subscribing and hitting the bell. And&nbsp;&nbsp; I would like to send a huge thank you to&nbsp; all of you Fellow Scholars for watching,&nbsp;&nbsp; because we can only exist&nbsp; because of you. Thank you!