[00:00] Artificial intelligence companies are&nbsp; now asking a question that used to&nbsp;&nbsp; belong to science fiction writers and&nbsp; slightly over-caffeinated bloggers:&nbsp;&nbsp; what happens when AI begins to improve&nbsp; AI? Because it is beginning to happen.
[00:16] I'm always looking for new products that&nbsp; you might find useful. And today's sponsor&nbsp;&nbsp; Consensus is one that you should definitely have&nbsp; a look at because it solved a big problem for me.&nbsp;&nbsp;
[00:28] My problem with using AI for research is&nbsp; that if you ask a chatbot about science,&nbsp;&nbsp; it'll happily invent a source, present it with&nbsp; total confidence and sound completely convincing,&nbsp;&nbsp;
[00:40] except the paper doesn't exist. This is why&nbsp; Consensus works the other way around. It searches&nbsp;&nbsp; 220 million peer-reviewed papers first and then&nbsp; uses AI to summarize what it actually found. That&nbsp;&nbsp;
[00:56] way, every claim links to a real verifiable study.&nbsp; I particularly like the deep search function,&nbsp;&nbsp; which shows you the consensus across the whole&nbsp; literature. You can see where studies agree,&nbsp;&nbsp;
[01:09] where they conflict, and how the research has&nbsp; changed over time. It also integrates with Claude.&nbsp;&nbsp; Use the link in the description for a full week of&nbsp; Consensus free. And now back to the science news.
[01:23] That AI improves itself is widely regarded&nbsp; as the beginning of the singularity,&nbsp;&nbsp; aka the intelligence explosion, after which&nbsp; AI will see a rapid self-powered improvement&nbsp;&nbsp;
[01:35] that will either turn the world into a&nbsp; paradise or hell, depending on who you ask. Jimmy Ba, the cofounder of xAI, recently tweeted&nbsp; that: “Recursive self-improvement loops likely do&nbsp;&nbsp;
[01:48] live in the next 12 months.” And that “2026&nbsp; is gonna be insane and likely the busiest&nbsp;&nbsp; and most consequential year for the future of&nbsp; our species.” Dario Amodei, CEO of Anthropic,&nbsp;&nbsp;
[02:02] one of the world’s leading AI firms, also thinks&nbsp; it’s the biggest thing to watch out for right now,&nbsp;&nbsp; and I agree. [Dario Amodei:] “I think&nbsp; the biggest thing to watch is this issue&nbsp;&nbsp; of AI systems building AI systems. How that&nbsp; goes, whether that goes one way or another,&nbsp;&nbsp;
[02:19] that will determine, you know, whether&nbsp; it's a few more years until we get there,&nbsp;&nbsp; or if we have wonders and a great emergency&nbsp; in front of us that we have to face.”
[02:33] Indeed, Anthropic just created an internal&nbsp; think tank as a “fire drill” to prepare for&nbsp;&nbsp; the intelligence explosion and to look&nbsp; for “early warning signals for recursive&nbsp;&nbsp;
[02:45] self-improvement”. They called it the “Anthropic&nbsp; Institute”. It’s led by co-founder Jack Clark&nbsp;&nbsp; who put the odds above sixty percent that&nbsp; by twenty twenty-eight an AI system could&nbsp;&nbsp;
[02:57] be told to make a better version of&nbsp; itself, and then do it autonomously.&nbsp; However, the closer we have come to AI&nbsp; self-improvement, the more obvious it’s become&nbsp;&nbsp; that it’s an incredibly vague and useless term.&nbsp; One can argue that if an AI researcher uses AI&nbsp;&nbsp;
[03:15] in their research, then that counts as a sort of&nbsp; self-improvement of AI. Though seeing how things&nbsp;&nbsp; are going with the scientific literature,&nbsp; I think that’s an intelligence implosion.
[03:27] But I digress. My point is that there are many&nbsp; ways AI can self-improve in some sense and yet&nbsp;&nbsp; it’s not the sense that anyone had in mind&nbsp; when they were thinking of the intelligence&nbsp;&nbsp; explosion. It was supposed to be a runaway&nbsp; effect, caused by AI improving their own code,&nbsp;&nbsp;
[03:44] that leads to exponential or super&nbsp; exponential intelligence increase. I already talked last year about the beginning of&nbsp; self-improving AI with Large Language Models that&nbsp;&nbsp;
[03:57] tune their own hyperparameters or, on the more&nbsp; applied side, AI that has improved microchips&nbsp;&nbsp; used for AI training. Self-improvement, yes,&nbsp; but not the runaway effect we’re waiting for.
[04:11] But, these examples have multiplied in&nbsp; the past months, and some of these AIs&nbsp;&nbsp; do now indeed write new AI code. The clearest&nbsp; recent example is probably Andrej Karpathy’s&nbsp;&nbsp;
[04:25] AutoResearch. It is a small Python setup&nbsp; where an artificial intelligence agent&nbsp;&nbsp; modifies language-model training code, trains&nbsp; the model for 5 minutes, checks whether the&nbsp;&nbsp; result improved, keeps or discards the change,&nbsp; and repeats. Meta’s HyperAgents work like this,&nbsp;&nbsp;
[04:42] too. These are LLM-based research agents that&nbsp; write code, run experiments, debug failures,&nbsp;&nbsp; keep what works and toss what doesn’t. These are not yet models directly rewriting&nbsp;&nbsp;
[04:55] their own brains. But it is artificial&nbsp; intelligence developing new artificial&nbsp;&nbsp; intelligence, albeit on a small scale. We are&nbsp; indeed getting closer to the self-recursive part.
[05:07] METR, a nonprofit institute for Model Evaluation&nbsp; and Threat Research is keeping track of this&nbsp;&nbsp; closely. In November they asked whether we&nbsp; have already reached the point where AI can&nbsp;&nbsp;
[05:20] do AI research well enough to speed up the&nbsp; next AI? To answer this question, they tested&nbsp;&nbsp; GPT-5.1-Codex-Max on some research-engineering&nbsp; tasks, like fixing a damaged small language&nbsp;&nbsp;
[05:34] model. In November their answer was: we have&nbsp; not yet reached the self-acceleration point. But that was in November, and at the&nbsp; pace that things are going, that's&nbsp;&nbsp; basically the stone age. Already in February, one&nbsp; METR researcher published a model predicting that,&nbsp;&nbsp;
[05:51] if current trends continue, AI could automate&nbsp; more than 99 percent of AI research and&nbsp;&nbsp; development around 2032. Personally, I think&nbsp; that it will happen much sooner than that.
[06:03] So this is no longer just “someone&nbsp; on the internet says the singularity&nbsp;&nbsp; is near”. The people who work in the&nbsp; field are now discussing when and how&nbsp;&nbsp; the recursive loop is going to close&nbsp; and how to prepare for that event.
[06:19] That said, so far, artificial intelligence has&nbsp; not replaced scientists. It has merely automated&nbsp;&nbsp; the part where we try 700 things that don’t&nbsp; work and pretend this was the plan all along.
[06:33] Thanks for watching. See you tomorrow.