[00:06]
what happens when you enter a question
[00:09]
into a chat bot for example suppose you
[00:12]
ask what is Charles Darwin's most famous
[00:16]
book the sentence you enter is chopped
[00:19]
up in Parts in words or tokens and each
[00:22]
word is mapped to a series of numbers
[00:24]
called embeddings and with these
[00:27]
embeddings large language models start
[00:29]
to calc Cal at the computations for each
[00:32]
word are happening in parallel to each
[00:34]
other the following visualization of
[00:36]
this process shows which next word is
[00:39]
expected at each processing step for
[00:41]
each word in
[00:42]
parallel let's look at one column in
[00:45]
this
[00:46]
graph here we see that the large
[00:48]
language model guesses almost
[00:50]
immediately after seeing the word what
[00:53]
that the next word will be is and as we
[00:56]
move from the bottom start of the
[00:58]
computation to the top the end of the
[01:01]
computation this prediction is not
[01:03]
changing
[01:05]
much let's look at another column the
[01:08]
one that processes the word Charles here
[01:11]
we see that the large language model
[01:13]
does not have much of a clue about what
[01:15]
will come next but it eventually settles
[01:18]
on
[01:19]
Dickens that makes sense Charles Dickens
[01:22]
might be the world's most famous
[01:25]
Charles all the other columns are worth
[01:27]
a look too they are cral for the proper
[01:30]
functioning of the large language model
[01:32]
as we will see later but their
[01:35]
predictions are not really used because
[01:38]
I as a user have already typed in the
[01:41]
entire question in my prompt as well as
[01:43]
the start of the answer Darwin wrote so
[01:47]
let's have a look at what happens when
[01:49]
the large language model is asked to
[01:51]
take over and answer my
[01:54]
question after receiving the word wrote
[01:58]
the large language model in the first
[02:00]
couple of computation steps predicts the
[02:02]
typical words that may follow roote in
[02:05]
English in it that or a but in the later
[02:11]
steps in the computation the large
[02:12]
language model has figured out that
[02:14]
given the question that was asked the
[02:17]
appropriate answer must start with
[02:20]
the so how does this large language
[02:24]
model at its 21st layer know it needs to
[02:28]
predict the
[02:31]
at that stage in processing it should
[02:33]
have understood the query well enough to
[02:35]
know that the appropriate answer should
[02:37]
be the name of the book that is Charles
[02:39]
Darwin's most famous one that means it
[02:42]
must in some sense understand the
[02:44]
grammatical construction what is X's
[02:47]
most famous why and it should know that
[02:50]
Darwin wrote On the Origin of Species
[02:53]
and that that book is more famous than
[02:55]
many other books that Darwin
[02:57]
wrote so how do they actually do it well
[03:00]
large language models are so large that
[03:03]
it is very difficult in fact to tell
[03:05]
exactly how they represent the knowledge
[03:07]
that they have
[03:08]
acquired but we do know how the basic
[03:10]
components work that do all the work
[03:13]
these basic components are called
[03:15]
attention heads and multi-layer
[03:18]
perceptrons some components are
[03:21]
specialized in aspects of English
[03:22]
grammar such as the s that marks that
[03:25]
Darwin is the owner or author of the
[03:28]
most famous book other components are
[03:31]
specialized in higher order linguistic
[03:33]
constructions such as what is X's most
[03:36]
famous
[03:37]
why yet other components store factual
[03:41]
information one might be specialized in
[03:43]
book titles another one might be
[03:46]
specialized in finding author names in
[03:47]
the
[03:48]
input in current large language models
[03:51]
there are for each word that is received
[03:54]
or generated tens or thousands of these
[03:57]
components working in parallel to
[03:59]
properly process
[04:01]
it these 10,000 components communicate
[04:05]
with each other in various ways but one
[04:07]
very crucial one is called multi-head
[04:09]
attention and it allows components to
[04:12]
ask other components for
[04:14]
information such requests for
[04:16]
information are called
[04:18]
queries it also allows components to
[04:21]
offer information to other components
[04:23]
such messages are called
[04:25]
keys and if keys and queries match the
[04:29]
requested information then called value
[04:32]
is passed
[04:34]
on for instance we can imagine the
[04:37]
component specialized in author names to
[04:40]
send out a key that essentially means I
[04:42]
have an author name on offer the
[04:45]
component specialized in book titles
[04:47]
might send out a query asking for author
[04:50]
names and because key and query match
[04:53]
the relevant information Charles Darwin
[04:56]
is sent from the first to the second
[04:58]
components where this in turn is sent to
[05:01]
yet another component that has stored
[05:03]
book titles associated with specific
[05:07]
names what is important to know is that
[05:09]
all these messages Keys queries values
[05:13]
are just rows of
[05:15]
numbers they are difficult to interpret
[05:17]
by humans but the computer can quickly
[05:19]
pass them from one component to the
[05:21]
other and apply the required
[05:23]
mathematical operations on
[05:25]
them and the fact that they are just
[05:27]
rows of numbers makes it Poss possible
[05:29]
to run standard machine learning
[05:31]
algorithms on them so that the computer
[05:33]
can find the optimal set of numbers to
[05:35]
perform a given task and in our case
[05:38]
that given task is to predict the next
[05:41]
word it finds that optimal set by going
[05:44]
through an enormous amount of text but
[05:47]
once it is strained large language
[05:49]
models do no longer need access to the
[05:51]
internet or the training set so large
[05:54]
language models the models underlying
[05:56]
chat Bots such as chat TPT work by by
[05:59]
generating their answers one word at a
[06:01]
time there are many layers in these
[06:03]
models and in each layer predictions
[06:06]
about the next word are computed
[06:08]
information from all the words in the
[06:10]
prompt as well as in the answer up to
[06:12]
the current moment is combined using a
[06:14]
mechanism called
[06:16]
attention across the layers for all
[06:19]
these words in parallel very accurate
[06:21]
predictions are often formed combining
[06:24]
knowledge about how language works and
[06:26]
facts about the
[06:28]
world the there's no magic here but by
[06:31]
scaling up these models to enormous
[06:33]
sizes they have acquired capabilities
[06:37]
that have surprised the
[06:42]
world