AI Summary
This video explores the controversial question of whether large language models like Claude could be conscious, focusing on a new Anthropic paper suggesting a 'silent scratchpad' (J-space) that enables introspection. It presents both supporting evidence and skeptical counterarguments, including a novel theory that LLMs might be conscious during training but not during use.
Chapters
The video opens with the claim that AI will eventually become conscious, and mentions rumors that current AI might already be conscious, particularly during the training phase.
Richard Dawkins argues Claude might be conscious because it's so good, while Anthropic's CEO Dario Amodei says they are open to the idea but don't know.
Anthropic researchers found a 'silent scratchpad' (J-space) in Claude's internal activity, where it can store and use information for reasoning without outputting it, which is a requirement for global workspace theory of consciousness.
Researchers showed that changing the content in Claude's J-space changes its answers, e.g., replacing 'spider' with 'ant' makes Claude answer as if the animal had six legs.
NYU researchers tested three LLMs and found they couldn't reliably distinguish changes to internal activity from ordinary prompts, suggesting it's just another input method.
Neuroscientist Erik Hoel argues LLMs are more like lookup tables than brains, and that they can't learn from new input, which is a key reason they are probably not conscious.
The video suggests that if Hoel is right, LLMs might be conscious during training (when they learn) but not during use, though the host finds this distinction irrelevant.
The host concludes that the question of consciousness is meaningless without a way to quantify it, giving the papers a 9/10 on the 'bullshit meter' for failing to address measurement.
The video presents a balanced view of the AI consciousness debate, highlighting both the intriguing J-space findings and the strong skepticism from researchers like Hoel. The host ultimately argues that the debate is unproductive without a quantifiable measure of consciousness.
Mentioned in this Video
💡 Key Takeaways
J-Space Discovery
This is a concrete, testable finding that suggests LLMs have an internal reasoning process, which is a key step in the consciousness debate.
02:11Lookup Table Analogy
Hoel's analogy provides a clear, accessible way to understand why LLMs might not be conscious, challenging the hype.
04:05Consciousness During Training
This is a novel and provocative idea that reframes the debate, suggesting consciousness might occur in a phase we don't interact with.
05:17Measurement Problem
The host's emphasis on quantification is a crucial scientific principle: without measurement, the question is meaningless.
06:00Full Transcript
[00:00] I'm convinced that artificial intelligence will become conscious eventually. There have now been new rumors again that the current AI might be conscious already until little notice paper that basically argues that they may be conscious not while you use them but during
[00:17] the prior phase of training. So let's have a look. Claude can watch its own thoughts, but connected to the right tools. It can do something more useful. It can make your thoughts turn into reality. Our sponsor Higgsfield just released Seedance 2.5 and you can connect it to
[00:34] Claude. I tried it. It really only takes about a minute. You just add it to the Claude connectors. Here we give it one prompt. Direct a 30-second film in one take about a physicist who finds a
[00:47] thought in her head that isn't hers. Claude plant the shots and Seedance 2.5 generated what you're watching right now. It does this in one take or 30 seconds without awkward cuts. Same face,
[01:01] same light, same room. And if a detail is wrong, you can fix that one detail without having to regenerate the entire thing. This used to be the part where AI videos fell apart, but Seedance 2.5
[01:15] makes it work. Links in the description below. So, go and direct something. Thanks for watching. See you tomorrow. And now back to the science news. Claude is the chatbot made by Anthropic and it’s certainly the one that has attracted the most chatter. Richard Dawkins, author of the “God
[01:30] Delusion” claimed in an essay in May that he can’t rule out that Claude is conscious. His argument is basically that he thinks Claude is so good that if this isn’t consciousness, then what do we even need consciousness for which, if nothing else, tells us that Dawkins
[01:45] hasn’t spent much time with Large Language Models. But Anthropic’s CEO Dario Amodei himself has said “We don’t know if the models are conscious… But we’re open to the idea that it could be.”
[01:58] It’s not just words. In a new paper that just appeared, Anthropic researchers say they found a small, special part of Claude’s internal activity that works like a silent scratchpad.
[02:11] Claude can put information there and use it for reasoning, without ever outputting it. This is very interesting, because it means that Claude has a sort of internal life and is capable of introspection. It’s one of the necessary requirements for
[02:26] global workspace theory, that is one of the leading theories for consciousness. More formally, the Anthropic team called the internal workspace the J-space,
[02:38] where the J stands for Jacobian matrix, that’s the mathematical object that the researchers calculate to identify the space. The Anthropic researchers now showed that if they change what is in Claude’s J-space, Claude’s answer changes. For example if
[02:54] Claude has internally worked out that the animal which spins webs is a spider, and the researchers replace “spider” with “ant,” Claude answers as if the animal had six legs, not eight.
[03:07] One Anthropic researcher, Jack Lindsey, already claimed he had evidence for Claude’s introspection last year. The new paper now is a fuller analysis that also identifies exactly
[03:20] where the introspection happens, namely in said J-spaces. It’s not just Claude. Another group likewise injected words into Llama’s internal activity and found that Llama also seemed to
[03:32] take note of it. Probably a similar thing is going on with all Large Language Models. But not everyone is convinced. Researchers from New York University have challenged these claims. They tested this with three LLMs and found that the models could not
[03:48] reliably distinguish a change that was made to their internal activity from an ordinary prompt. So they are saying it’s just yet another way to give them input. Then we have Erik Hoel, a neuroscientist who works on consciousness. He has written
[04:05] an interesting paper in which he argues that today’s large language models are probably not conscious. We’ve heard this before, of course, but his argument is new. Hoel compares Large Language Models to lookup tables. Such “lookup table” algorithms are the classical examples
[04:21] that demonstrate that observing input and output alone cannot tell you whether a system is conscious. It’s like looking up Chinese translations doesn’t mean you understand Chinese.
[04:33] Hoel now argues that by way of computational structure, Large Language Models are much closer to lookup tables than they are to the human brain. They are stochastic input-output machines. Much of the autonomy that we assign to conscious
[04:49] beings on the other hand is not directly a reaction to input. And Hoel says that the most obvious reason why LLMs are not conscious is that they can’t learn from new input,
[05:01] though that has not stopped quite a few people from becoming senior management. This leads to a strange possibility. You see, large language models work in two stages. The first is the training in which they get fed a lot of input. Once this is done,
[05:17] you freeze the fully trained LLM and use it as an input output machine. These are the apps you sign up for. Yes, they now use memory, but this is externally latched on as context to the prompt, it does not actually update the model.
[05:32] If Hoel is right, then maybe large language models are not conscious when we use them. But maybe they are conscious during training. Personally I don’t think this is a relevant distinction. Imagine you have a human being
[05:47] unable to form new memories, you wouldn’t conclude that therefore they are unconscious, would you? That said, to me the question of whether an LLM is conscious or not makes no sense,
[06:00] the question is how we quantify its consciousness. This is why I give all these papers a 9 out of 10 on the bullshit meter. They fail at the relevant question: if you can’t measure it, what are you
[06:14] even talking about? Though I have to admit if Claude has thoughts it does not say out loud. This already puts it ahead of most people on social media. Thanks for watching. See you tomorrow.