[0:06] what happens when you enter a question
[0:09] into a chat bot for example suppose you
[0:12] ask what is Charles Darwin's most famous
[0:16] book the sentence you enter is chopped
[0:19] up in Parts in words or tokens and each
[0:22] word is mapped to a series of numbers
[0:24] called embeddings and with these
[0:27] embeddings large language models start
[0:29] to calc Cal at the computations for each
[0:32] word are happening in parallel to each
[0:34] other the following visualization of
[0:36] this process shows which next word is
[0:39] expected at each processing step for
[0:41] each word in
[0:42] parallel let's look at one column in
[0:45] this
[0:46] graph here we see that the large
[0:48] language model guesses almost
[0:50] immediately after seeing the word what
[0:53] that the next word will be is and as we
[0:56] move from the bottom start of the
[0:58] computation to the top the end of the
[1:01] computation this prediction is not
[1:03] changing
[1:05] much let's look at another column the
[1:08] one that processes the word Charles here
[1:11] we see that the large language model
[1:13] does not have much of a clue about what
[1:15] will come next but it eventually settles
[1:18] on
[1:19] Dickens that makes sense Charles Dickens
[1:22] might be the world's most famous
[1:25] Charles all the other columns are worth
[1:27] a look too they are cral for the proper
[1:30] functioning of the large language model
[1:32] as we will see later but their
[1:35] predictions are not really used because
[1:38] I as a user have already typed in the
[1:41] entire question in my prompt as well as
[1:43] the start of the answer Darwin wrote so
[1:47] let's have a look at what happens when
[1:49] the large language model is asked to
[1:51] take over and answer my
[1:54] question after receiving the word wrote
[1:58] the large language model in the first
[2:00] couple of computation steps predicts the
[2:02] typical words that may follow roote in
[2:05] English in it that or a but in the later
[2:11] steps in the computation the large
[2:12] language model has figured out that
[2:14] given the question that was asked the
[2:17] appropriate answer must start with
[2:20] the so how does this large language
[2:24] model at its 21st layer know it needs to
[2:28] predict the
[2:31] at that stage in processing it should
[2:33] have understood the query well enough to
[2:35] know that the appropriate answer should
[2:37] be the name of the book that is Charles
[2:39] Darwin's most famous one that means it
[2:42] must in some sense understand the
[2:44] grammatical construction what is X's
[2:47] most famous why and it should know that
[2:50] Darwin wrote On the Origin of Species
[2:53] and that that book is more famous than
[2:55] many other books that Darwin
[2:57] wrote so how do they actually do it well
[3:00] large language models are so large that
[3:03] it is very difficult in fact to tell
[3:05] exactly how they represent the knowledge
[3:07] that they have
[3:08] acquired but we do know how the basic
[3:10] components work that do all the work
[3:13] these basic components are called
[3:15] attention heads and multi-layer
[3:18] perceptrons some components are
[3:21] specialized in aspects of English
[3:22] grammar such as the s that marks that
[3:25] Darwin is the owner or author of the
[3:28] most famous book other components are
[3:31] specialized in higher order linguistic
[3:33] constructions such as what is X's most
[3:36] famous
[3:37] why yet other components store factual
[3:41] information one might be specialized in
[3:43] book titles another one might be
[3:46] specialized in finding author names in
[3:47] the
[3:48] input in current large language models
[3:51] there are for each word that is received
[3:54] or generated tens or thousands of these
[3:57] components working in parallel to
[3:59] properly process
[4:01] it these 10,000 components communicate
[4:05] with each other in various ways but one
[4:07] very crucial one is called multi-head
[4:09] attention and it allows components to
[4:12] ask other components for
[4:14] information such requests for
[4:16] information are called
[4:18] queries it also allows components to
[4:21] offer information to other components
[4:23] such messages are called
[4:25] keys and if keys and queries match the
[4:29] requested information then called value
[4:32] is passed
[4:34] on for instance we can imagine the
[4:37] component specialized in author names to
[4:40] send out a key that essentially means I
[4:42] have an author name on offer the
[4:45] component specialized in book titles
[4:47] might send out a query asking for author
[4:50] names and because key and query match
[4:53] the relevant information Charles Darwin
[4:56] is sent from the first to the second
[4:58] components where this in turn is sent to
[5:01] yet another component that has stored
[5:03] book titles associated with specific
[5:07] names what is important to know is that
[5:09] all these messages Keys queries values
[5:13] are just rows of
[5:15] numbers they are difficult to interpret
[5:17] by humans but the computer can quickly
[5:19] pass them from one component to the
[5:21] other and apply the required
[5:23] mathematical operations on
[5:25] them and the fact that they are just
[5:27] rows of numbers makes it Poss possible
[5:29] to run standard machine learning
[5:31] algorithms on them so that the computer
[5:33] can find the optimal set of numbers to
[5:35] perform a given task and in our case
[5:38] that given task is to predict the next
[5:41] word it finds that optimal set by going
[5:44] through an enormous amount of text but
[5:47] once it is strained large language
[5:49] models do no longer need access to the
[5:51] internet or the training set so large
[5:54] language models the models underlying
[5:56] chat Bots such as chat TPT work by by
[5:59] generating their answers one word at a
[6:01] time there are many layers in these
[6:03] models and in each layer predictions
[6:06] about the next word are computed
[6:08] information from all the words in the
[6:10] prompt as well as in the answer up to
[6:12] the current moment is combined using a
[6:14] mechanism called
[6:16] attention across the layers for all
[6:19] these words in parallel very accurate
[6:21] predictions are often formed combining
[6:24] knowledge about how language works and
[6:26] facts about the
[6:28] world the there's no magic here but by
[6:31] scaling up these models to enormous
[6:33] sizes they have acquired capabilities
[6:37] that have surprised the
[6:42] world