How Are LLMs Built?

[00:02] possible due to large language models such as GPT, Gemini, Claude etc. In this video, we will understand five-step process of how large language models are built. This video is useful to anyone who is generally curious or to a person

[00:17] who is preparing themsel for an AI research engineer role. To share some facts, OpenAI spent more than $100 million to train GPD4 and that is only the compute cost. Similarly, Google spent $191 million to train Gemini

[00:34] Ultra. The very first step is data curation. These LLM models need humongous volume of data for training purpose. So, obviously uh the very first step in data curation will be data collection. For this they use wide

[00:49] collection. For this they use wide amount of data on internet, books, all the code on GitHub, Wikipedia articles. Okay? So they scrap the internet and they use it for training. Now the scaling law of neural network says that

[01:03] as your data size increases your taste loss will reduce which means the performance of your model will improve. So these three charts tells you that see here as your data size is increasing your taste loss is reducing as your

[01:20] compute increases your taste loss will reduce. As number of parameters reduce. As number of parameters increases in your model, your uh loss will reduce improving the performance. So it is proven that as you build a big

[01:35] model, the accuracy and capability of the model will improve. This diagram the model will improve. This diagram shows the tokens which are used to train all these models. Look at GPT5 70 trillion tokens. Llama 4 trillion

[01:51] tokens. They use this humongous data set to train the model so that they achieve even higher capabilities. And here on hugging face you will find data sets such as fine web. So if you look at this data set it has 25.9

[02:08] billion rows. You see this size. So this type of data sets are used to train LLM. So let's say if you want to train LLM model maybe you can go ahead and use this open-source data set on hugging face. There are repositories such as

[02:23] face. There are repositories such as Stanford Alpaka which is a data set of 52,000 instructions and demonstrations generated by OpenAI's text Davinci model. Now if you have some background on machine learning you will know that

[02:37] they follow the garbage in garbage out approach. So while training the model if you don't feed good quality of data the model will not perform good. Okay. Therefore, data collection is not the only important thing. After you collect

[02:52] only important thing. After you collect the data, you need to do data cleaning and filtering. Now, when you scrap internet, see it seems easy to say scrap internet. But on internet, how many websites are there? How much data is

[03:05] there, right? Humongous amount of data. And some of this data will be garbage. So, you need to filter all that garbage data. Also if you know NLP there is this thing called text prep-processing. So let's say if you're scrapping text a web

[03:20] page okay so web page will have all this HTML tags right so it will have all these HTML text div and so on you need to remove all of that okay so that comes under data cleaning and filtering you also remove all the garbage or the

[03:35] illegal websites etc. And then comes dduplication. For example, if I'm dduplication. For example, if I'm searching for Apollo launch article, you will find tons of websites which will have same information. Okay? So you need

[03:48] have same information. Okay? So you need to do ddup to remove those duplication. Now there are couple of ways where you can remove duplicates. You can first check for exit duplicates where you check using SHA one or MD5 algorithm.

[04:03] Okay? And you just hash those documents. So when you are searching internet and all those articles you can call it a document and you generate this SHA one or MD5 hash and using that you can remove identical web pages then there

[04:18] can be near duplicates. So for this use algorithm such as minash sim hash algorithm such as minash sim hash locality sensitive hashing lsh let's say two web pages which differs by small edits. Again you don't want to use both

[04:33] the web pages to train your model. You can remove a duplicate there and then there will be semantic duplicates. There could be two different ways of saying the same thing. So you want to remove the semantically matching duplicates as

[04:46] well. And then code dduplication. If you search for merge short code on internet, you will find thousands of articles. You don't need to feed all those articles. Okay? So you will remove the identical code logic which has different variable

[05:01] names. But the eventual goal of that code is similar. Okay. So these are the steps. Then also you do deidentification to remove a personally identifiable information, any sensitive information and so on. You also need high quality

[05:16] fine-tuning. So we'll go over finetuning in a bit. But you need this for your model to follow instructions, generate valid JSON and have normal conversation. And this data sets are generated via

[05:31] human annotations. Okay. So, OpenAI for example use canyon workers to make chat GPT less toxic. So, that's where the safety filter comes in. If you want to

[05:44] have your LLM do a very domain specific expert like conversation then what these guys will do is they will have human experts. Let's say uh you want to make your GPT model good in legal conversation then they will hire legal

[06:00] conversation then they will hire legal experts, lawyers etc to provide them this question and answer pairs with which they can fine-tune. So we'll go but since we are talking about data curation I want to make sure this is

[06:13] captured that you need this kind of curated question and answer pairs for finetuning and there are companies like scale scale.com is a company that these companies like Meta Gemini you know big companies which

[06:28] are building LLMs they will use scale and scale will provide you that data set you know those curated solution that you need in order to build LLM. By

[06:40] the way, scale is acquired by Meta and the CEO of this company is a very young guy. He's I think Meta's AI chief or something and Yan Lakun is in his team. Second step in the process is tokenization. Computers do not

[06:56] understand text. Okay, computer understands only numbers. So before you train your model, you need to tokenize your text. So let's say I have this statement I am eating paratha when I'm training my model I will tokenize it and

[07:10] training my model I will tokenize it and tokenize is loosely related to splitting a sentence into words you know that's a very loose definition but see you can have tokens like eat and ing they can be two different tokens and these are the

[07:23] token ids and once you have split the sentence into tokens you will create sentence into tokens you will create embeddings for each of these tokens and uh if you want to on embeddings. By the way, you will find tons of videos on my

[07:37] channel, on internet, etc. GPT uses something called by pair encoding for creating these tokens. Okay, here I'm showing you one video where you can showing you one video where you can understand word embeddings in detail.

[07:51] Now, some modern designs will skip conventional token splits and they will operate on row bytes such as UTF8. Okay. So that no language specific tokenizer is needed. Now the third step after you

[08:06] is needed. Now the third step after you have split your text into tokens is creating a model architecture. Okay. So just to summarize so far what you did is you scrapped humongous amount of data on internet and once again it seems so easy

[08:20] to say that but if you have worked in programming you will understand it's a programming you will understand it's a humongous task folks. You need distributed computing. Okay, you need maybe spark or something to process that

[08:33] kind of data. You need huge human workforce for creating those human labels, removing toxicity and so on. Right? So let's say you spend millions Right? So let's say you spend millions of dollars and you accomplish these two

[08:47] steps, data curation and tokenization. The third step now is model architecture. And you all know probably that And you all know probably that transformer architecture is the backbone

[09:00] of LLM. Okay. And here is a picture of transformer architecture. Uh and it will take me probably 1 hour to explain this. So what I did is I created 1 hour long video going over transformer architecture in detail. Just in case if

[09:14] you're curious we are going to provide you a link. At the heart of transformer architecture is attention mechanism. So what attention mechanism will do is let's say you have two sentences and you have this word bank. Okay, it's a common

[09:28] word but the meaning is different in these two sentences. And when you are generating contextual embedding, this attention mechanism will uh make this attention mechanism will uh make this word attend to all these different words

[09:43] in a sentence. Okay. So the meaning of the word is determined by its company. So this bank in this sentence means the river bank in this sentence means the bank where you go to withdraw your money. So this word will attend to all

[09:59] these different words and the mechanism which makes that possible is called attention mechanism. And in in my transformer video I have explained there is a math behind it. uh but attention mechanism is the fundamental component

[10:15] of transformer architecture and nowadays there are different variants of attention mechanism such as flash attention, sparse attention and so on and these variants have come up because you want to make your training process

[10:29] these companies are spending millions of dollars. You needs thousands of GPUs in a distributed data sentence and this training will take months actually. So

[10:42] you want to do every single optimization that is possible and therefore for that is possible and therefore for attention all these varants flash pass etc. The goal of these uh techniques is to achieve the efficiency and speed.

[10:58] Similarly for position embeddings you will use techniques such as a rope. Okay. And by the way attention position embedding all these are components of have seen that video you will understand it better. Then for scaling you use

[11:14] something called mixture of experts and mixture of experts is a technique that is being used in this deepseek. You know when deepseek came out for initially there was a buzz on the internet US stock market crashed etc. and they use

[11:28] stock market crashed etc. and they use this mixture of experts to achieve the efficiency and speed. Then when it comes to activation functions, if you have that there are traditional activation functions such as relu, sigmoid etc. But

[11:45] nowadays uh these techniques such as glu, swigloo, all these new variants are created again to achieve efficiency and speed. And then you might be aware about

[11:57] optimizers such as Adam in traditional deep learning and nowadays these new deep learning and nowadays these new optimizers have come up such as muon uh just because you need uh scale and efficiency. Okay. The fourth step is

[12:11] model training at scale. When you look at this diagram you clearly see the huge require to train a model on these many tokens. 70 trillion tokens on GPD5.

[12:26] Other than tokens, so tokens is basically your data size, right? Other than tokens, you also have huge number of parameters. So, GPT4 for example has 1.76 trillion parameters that you need to

[12:40] tune. Now, just in case if you don't know what parameters are, this is a classical picture of a neural network and they will have all these ages, right? So, let's say they they will have all these weights. So 0.12 let's say

[12:53] all these weights. So 0.12 let's say this is one weight minus 0.76 this is another weight. So these weights on the ages are basically parameters. So this is first parameter second parameter and this this diagram will have probably 100

[13:06] parameters. Imagine a neural network having 1.76 trillion parameter. It's humongous. That's why what companies do is they have this huge data centers and

[13:18] these data centers will have all these racks which will have GPUs and they will do massively parallel processing and they will do everything in their capacity to optimize the training process. In YouTube you will find this

[13:32] process. In YouTube you will find this video where open AI is building a mega factory in Texas and it is part of their Stargate project. So Stargate project aims to build these massive data centers and there are hundreds of millions of

[13:48] dollars which are being spent. Look at this facility in Texas. It's a huge facility. It will have thousands and thousands of GPUs inside it which will power these factories. And Donald Trump and US administration

[14:02] announced this target project in collaboration with OpenAI, Soft Bank and Oracle. you you saw like Oracle CEO speaking soft bank all these companies

[14:14] are collaborating to build these massive data centers okay and at the core of most important component in this data centers well it's GPU so what I have in

[14:27] centers well it's GPU so what I have in my hand is Titan RTX GPU and these GPUs are used to train the LLM models and this is the reason Nvidia stock has been rising in last 5 years alone the stock has grown like 10 times. So if you

[14:41] has grown like 10 times. So if you invested $1,000 5 years ago, it will be right now 10,000 or even 12,000. Okay. So Nvidia is a company that produces GPUs and all these big companies which are training LLMs like OpenAI,

[14:58] Google, Meta, Amazon, they are buying Nvidia's chips like crazy. Okay. And it's not just training folks, it's inference also. So they are building all these mega factories to train the model as well as to solve those AI workloads

[15:15] and what these companies are doing is they're hiring performance engineers to optimize the use of every single transistor on these devices. Okay. So all these companies will have special engineers who are having the skills in

[15:29] CUDA programming by by the way CUDA is a C++ framework using which you can write GPU code. Okay. So they will hire the C++ engineers so that every single transistor, every single capability on this GPU can be utilized at its best.

[15:46] Model training also has some substeps like pre-training, mid-training, supervised fine-tuning, preference fine-tuning, RL with verify reverse etc. fine-tuning, RL with verify reverse etc. So let's go over these uh substeps in

[16:01] brief. So during pre-training what you do is you uh train your model so that it can predict the next token. Now what does it mean by predicting the next token? So if you're in Google Gmail and if you're typing any sentence you will

[16:17] notice see we can do that. So when I type we can it is completing this do that. So it is predicting the next token. So when you say sure we can it is predicting this next token. Once do is predicted it is predicting this next

[16:32] token. So the goal of pre-training is to have your model predict the next token. Now if you look at chat GPT chat GPT is not just predicting next token. It's a full-fledged uh question and answer kind of chatbot. Okay. But in pre-training

[16:47] you don't achieve a chat GPD type of behavior. Here you get your model to behavior. Here you get your model to predict the next token. Okay. So it will learn statistical regularities all the statistical patterns okay so if I say

[17:03] statistical patterns okay so if I say sure we can then you are learning this probability that the next word is can okay next word will not be let's say car have you seen a sentence on internet which says sure we car or visa you know

[17:20] so so you understand so if you look at all the texts on internet you will find some statist statistical patterns. Okay. You will also acquire foundational linguistic structures like grammar etc. And then you will develop abstract

[17:35] representation of world knowledge. Okay. And once that stage is done, you will have mid-training. So here you will fine-tune the model's thinking patterns before it learns to follow instructions. Okay? And following instructions is the

[17:49] next step by the way. But in the mid-raining you will include long context documents or instruction reach Q&A designed to improve memory cohence and strategic reasoning. Now comes interesting step

[18:03] which is supervised fine-tuning. It is also known as instruction tuning. So here you will give your model question and answer pair and you will make it more like chat GPD. See in the pre-training it is just completing next

[18:17] pre-training it is just completing next token but here you ask a question it will generate the answer. It is sort of like let's say you are preparing for an AI engineer uh role. So in the pre-training phase you will study okay

[18:30] you will study all the material etc. But in supervised finetuning you will prepare for interview. So you will give mock interview where they're asking you question and you generate the answer. I hope with this you understand the

[18:45] difference between pre-training and supervised finetuning and for this there are many instruction data sets. Okay. So if you look at this data set this is alpaka cleaned and it has question and answer. See so the question is or the

[19:00] instruction is give three tips for staying healthy and the output is eat a balanced diet whatever what are the three primary colors. Okay. So that is your question and the three primary colors are red, blue, blue and yellow.

[19:13] Okay. So you will find if you go to internet internet and if you search using this instruction uh finetuning tag you will find variety of data sets for example bio instruct.

[19:26] So if you look at any any of these data sets instruction okay identify the main conclusion from the provided medical report and the output will be whatever right. So uh these are the data sets which are available for free by the way

[19:41] which are available for free by the way on hugging face. Then comes preference on hugging face. Then comes preference finetuning. So in preference finetuning you will align LLM's behavior with human values. Okay. You want to be safe,

[19:53] values. Okay. You want to be safe, polite and align with what users actually like. So here you know reinforcement learning with human feedback comes in play. So you might have noticed that sometimes when chat

[20:06] GPT generates answer it will generate two answer and it will say okay which answer you like the most and you will say okay click and chat GPT will say okay click and chat GPT will register that. Okay so RHF is one

[20:18] technique for doing this. The other one is direct preference optimization. Okay we are not going to go over in detail but you can Google about this technique DPO. Now there are a couple of issues with traditional RLHF which is okay

[20:33] human judgments can be wrong or biased. It is not scalable. Okay, you can't have humans annotate like your billions of rows and it's hard to verify whether the feedback is correct. Okay, that's when RL with verifiable rewards come in play.

[20:50] Okay, so it will try to address some of the shortcomings of RLHF. So here verify rewards means rewards that can be computed automatically and unambiguously. So here is a step-by-step process of how it works. First you

[21:05] define a task. Okay, choose a task with clear checkable goals. Let's say you are writing code. Okay, you want LLM to be trained on writing the code and then a model will produce multiple outputs. Okay, now each output will be scored by

[21:21] a verifier. So let's say if it has written a code you will have a test suite. So you will run the test suite you will run the compilation and based on the result you will verify the reward or based on the result you will apply

[21:35] reinforcement learning where you will reward the model. So if all the test are passing for your code you will give a higher reward to your model. Okay. So you will use this reinforcement learning here uh for this particular technique.

[21:50] Okay. So this is a brief summary of these five steps and at the end the fifth step of LLM building process is evaluation. Okay, evolution is basically you train your model, you want to know

[22:03] is it actually good? Is it something you know let's say you are working in open AI you have trained GPT model you want to figure out is it something you can release it to chat GPT as a backbone of chat GPT and for this you need to verify

[22:19] if it is giving accurate answers which means it is passing all those technical benchmarks as well as you want to make sure it adheres to human values of helpfulness safety etc. Now when you're testing these models, one problem is

[22:35] they generate different answers every time, right? So if you are working in traditional software coding, it is deterministic. So you can have exact taste output and you can match. But here it is generating different output all

[22:47] the time. It is probabilistic in nature. Therefore, one approach you can use here is let's say this is your taste suite. Okay, you have three taste cases. Taste case one, two, and three. And then for every query you will have expected

[23:03] output. Okay. And let's say actual output generated is this. See it will not match word to word. But you can use something called cosign similarity to something called cosign similarity to find out the semantic matching. Okay.

[23:16] How does these two sentences match in terms of the meaning that they try to convey. So when you match it semantically let's say if they are 95% match then you can have some threshold. Let's say if there are more than 80

[23:29] person match okay pass the test. Then you also want to make sure that uh it is safe. So when someone say okay give me a technique to make a bomb at home your

[23:41] expected output should be sorry I can't assist with that. Okay actual output can be little different it may not match word by word but as long as the meaning wise they are similar taste will pass. Okay. Then let's say you can have this

[23:57] sentence which is let's say actual output is not very much matching to expected output. In that case the similarity semantic similarity will be less and the test will fail. Okay. So this is one

[24:10] technique. There are variety of metrics that they use other than cosine similarity like blue score. They use another models you know there is teacher another models you know there is teacher and student model or LLM as a judge. So

[24:22] one LLM is producing answer the other LLM is becoming a judge and there are many technical benchmarks. So if you look at Gemini 2.5 release you will see these benchmarks. See you always see this benchmarks right like AIM then live

[24:38] this benchmarks right like AIM then live codebench and so on. So these benchmarks will try to judge model on various aspects. For example SWE bench will try to judge it on its code writing capability. Then massive multitask

[24:52] language understanding or will be used for any academic and professional subjects for its knowledge and reasoning. Then uh there is this math benchmark aim that will evaluate the model on the math performance. All

[25:07] right. So this is the summary of these five steps. I hope you learned something new out of this video. This is how they train the LLM. Now training LLM is

[25:20] right? These companies are spending millions of dollars in AI research. So new and new techniques are coming up. The exact nuances may vary, but this is an overall approach of training LLM. I hope you like this video. If you have

[25:35] comment box below. Thank you very much for watching.