---
title: '3. How do Large Language Models work?'
source: 'https://youtube.com/watch?v=eJlyPy_Q0h0'
video_id: 'eJlyPy_Q0h0'
date: 2026-06-14
duration_sec: 404
---

# 3. How do Large Language Models work?

> Source: [3. How do Large Language Models work?](https://youtube.com/watch?v=eJlyPy_Q0h0)

## Summary

This video explains how large language models (LLMs) process a question and generate an answer, using the example of asking about Charles Darwin's most famous book. It describes the steps from tokenization and embeddings to parallel computation, attention mechanisms, and the role of specialized components like attention heads and multi-layer perceptrons.

### Key Points

- **Input Processing** [0:06] — A question is chopped into tokens (words), each mapped to a series of numbers called embeddings.
- **Parallel Computation** [0:29] — LLMs compute predictions for each word in parallel, visualizing which next word is expected at each step.
- **Word Prediction Example** [0:45] — For the word 'what', the model quickly predicts 'is'; for 'Charles', it eventually settles on 'Dickens'.
- **Answer Generation** [1:47] — After the prompt, the model predicts typical words following 'wrote', then narrows to 'the' based on context.
- **Understanding Grammar and Facts** [2:31] — The model understands grammatical constructions like 'what is X's most famous Y' and knows that Darwin wrote 'On the Origin of Species'.
- **Basic Components** [3:10] — Attention heads and multi-layer perceptrons are the basic components; some specialize in grammar, others in facts.
- **Multi-Head Attention** [4:07] — Components communicate via queries, keys, and values—rows of numbers that allow information exchange.
- **Training and Optimization** [5:23] — Standard machine learning algorithms find optimal numbers to predict the next word by training on vast text.

### Conclusion

LLMs generate answers one word at a time, combining language knowledge and facts through attention mechanisms across many layers. Scaling up these models has led to surprising capabilities.

## Transcript

what happens when you enter a question
into a chat bot for example suppose you
ask what is Charles Darwin's most famous
book the sentence you enter is chopped
up in Parts in words or tokens and each
word is mapped to a series of numbers
called embeddings and with these
embeddings large language models start
to calc Cal at the computations for each
word are happening in parallel to each
other the following visualization of
this process shows which next word is
expected at each processing step for
each word in
parallel let's look at one column in
this
graph here we see that the large
language model guesses almost
immediately after seeing the word what
that the next word will be is and as we
move from the bottom start of the
computation to the top the end of the
computation this prediction is not
changing
much let's look at another column the
one that processes the word Charles here
we see that the large language model
does not have much of a clue about what
will come next but it eventually settles
on
Dickens that makes sense Charles Dickens
might be the world's most famous
Charles all the other columns are worth
a look too they are cral for the proper
functioning of the large language model
as we will see later but their
predictions are not really used because
I as a user have already typed in the
entire question in my prompt as well as
the start of the answer Darwin wrote so
let's have a look at what happens when
the large language model is asked to
take over and answer my
question after receiving the word wrote
the large language model in the first
couple of computation steps predicts the
typical words that may follow roote in
English in it that or a but in the later
steps in the computation the large
language model has figured out that
given the question that was asked the
appropriate answer must start with
the so how does this large language
model at its 21st layer know it needs to
predict the
at that stage in processing it should
have understood the query well enough to
know that the appropriate answer should
be the name of the book that is Charles
Darwin's most famous one that means it
must in some sense understand the
grammatical construction what is X's
most famous why and it should know that
Darwin wrote On the Origin of Species
and that that book is more famous than
many other books that Darwin
wrote so how do they actually do it well
large language models are so large that
it is very difficult in fact to tell
exactly how they represent the knowledge
that they have
acquired but we do know how the basic
components work that do all the work
these basic components are called
attention heads and multi-layer
perceptrons some components are
specialized in aspects of English
grammar such as the s that marks that
Darwin is the owner or author of the
most famous book other components are
specialized in higher order linguistic
constructions such as what is X's most
famous
why yet other components store factual
information one might be specialized in
book titles another one might be
specialized in finding author names in
the
input in current large language models
there are for each word that is received
or generated tens or thousands of these
components working in parallel to
properly process
it these 10,000 components communicate
with each other in various ways but one
very crucial one is called multi-head
attention and it allows components to
ask other components for
information such requests for
information are called
queries it also allows components to
offer information to other components
such messages are called
keys and if keys and queries match the
requested information then called value
is passed
on for instance we can imagine the
component specialized in author names to
send out a key that essentially means I
have an author name on offer the
component specialized in book titles
might send out a query asking for author
names and because key and query match
the relevant information Charles Darwin
is sent from the first to the second
components where this in turn is sent to
yet another component that has stored
book titles associated with specific
names what is important to know is that
all these messages Keys queries values
are just rows of
numbers they are difficult to interpret
by humans but the computer can quickly
pass them from one component to the
other and apply the required
mathematical operations on
them and the fact that they are just
rows of numbers makes it Poss possible
to run standard machine learning
algorithms on them so that the computer
can find the optimal set of numbers to
perform a given task and in our case
that given task is to predict the next
word it finds that optimal set by going
through an enormous amount of text but
once it is strained large language
models do no longer need access to the
internet or the training set so large
language models the models underlying
chat Bots such as chat TPT work by by
generating their answers one word at a
time there are many layers in these
models and in each layer predictions
about the next word are computed
information from all the words in the
prompt as well as in the answer up to
the current moment is combined using a
mechanism called
attention across the layers for all
these words in parallel very accurate
predictions are often formed combining
knowledge about how language works and
facts about the
world the there's no magic here but by
scaling up these models to enormous
sizes they have acquired capabilities
that have surprised the
world