Agentic AI Crash Course Using LangChain | LangChain Crash Course

[00:02] AI and LangChain. Here are the list of topics that we are going to cover in this video. And in terms of technology stack, here are the list of tools and frameworks that we'll be using throughout the video. We will not just

[00:16] discuss theory behind generative AI and agentic AI, >> but we will do lots of hands-on coding. So, make sure you have your code editor open with you and you are practicing along with me. At the end, we are going

[00:29] to build two solid projects. The first one is a shopping agent, which is similar to Amazon's Rufus AI, where you search through products, you place an order. You can also search through products using an image, where we will

[00:44] use the multimodal AI. This will be completely end-to-end project that will look pretty good on your resume. The second project is building telecom rag chatbot, where as a knowledge source, we will be using three different sources,

[00:59] CSV file, SQLite database, and PDF. We'll cover chunking strategies and lot of important concepts for rag and complete building this entire project. We'll use Streamlit as a UI in both of these projects. The only prerequisite is

[01:16] to know anything else. >> [music] >> Anybody who wants to become AI engineer or who wants to lead AI projects in [music] any leadership position will benefit from this course. A little

[01:29] introduction on myself. I have worked in big tech companies like Bloomberg and Nvidia for more than 17 years. Right now, I run AI consultancy company called Atliq, where we have delivered more than 25 AI projects for our clients based in

[01:44] US and UK. Whatever I'm teaching you today is based on real industry experience. All right, let's get started with some theory on generative AI. We need to understand certain key terms. The first one is generative AI. We all

[01:58] know that we can type in a question in ChatGPT and it will generate new text. It can generate new image as well. Essentially, you are typing a prompt and the large language model, which is the heart of applications like ChatGPT,

[02:14] heart of applications like ChatGPT, these are very big AI models which applications. Whether you are using ChatGPT or you're using Claude or Gemini, at the heart of it is large language model. Now, in the case of uh

[02:31] ChatGPT, these models are, for example, GPT 5.3. If you're using Claude, the model will be the latest one is Opus 4.7. In case of Gemini,

[02:43] 4.7. In case of Gemini, you know, it will be Gemini uh 3.1 and so on. Generative AI is a type of artificial intelligence that can create new content. Okay? So, whenever you are creating a new content, such as text,

[02:57] image, audio, video, or even code, or even code, it is called generative AI. We have traditional AI as well. Before ChatGPT came, AI was there. Uh things like fraud

[03:11] detection used AI. This is the real fraud message that I got from my bank. fraud message that I got from my bank. And here, they use uh lightweight model called statistical models. They use methods such as XGBoost, random forest,

[03:26] etc. to figure out or to classify whether this transaction is fraud or not a fraud. So, here you are not generating a new text. You have input and you are just labeling it. You are saying whether it's a fraud or not a fraud. All right?

[03:41] So, if you want to holistically review this uh landscape, uh when you talk about artificial intelligence in a broader sense, it has multiple components in it. So, the first one is machine learning. So, whenever anyone is

[03:55] talking about machine learning, they're either talking about statistical ML or deep learning. In statistical ML, we have these type of models XGBoost, random forest, etc. And people have been using this for many years, 20-30 years.

[04:11] And when it comes to deep learning, it's all about neural networks. Whenever you hear this term neural network, that means it's deep learning. In neural network also, we have different variants. For example, convolutional

[04:24] neural network, RNN, transformer. Transformer is a new architecture which has made modern-day AI revolution possible. LLMs are based on this transformer architecture. And outside machine learning also, there is

[04:40] something in AI. So, AI basically means whenever you are enabling computer to do things which humans are generally good at. For example, driving a car, uh reading a book and comprehending it, creating a poem or story, etc.

[04:56] That is artificial intelligence, okay? And in that, machine learning is a main component. But you can have a rule-based systems also. If rule-based system enables computers to accomplish tasks which human are generally good at, then

[05:12] that is also AI, okay? In robotics also, you know, in control system, etc., uh many times you don't use machine learning. Of course, you use my machine learning for certain parts of robotics, but for certain other parts, you don't

[05:25] use it. So, this is the holistic picture. And generative AI is one type of application which is based on this transformer architecture, okay? So, GenAI is an application which is based on this transformer architecture. And uh

[05:41] this breakthrough happened when Google published this paper called "Attention is all you need." This is the key moment in the history which has made LLM possible. Okay, so how does transformer really work? So I love this

[05:57] website. It will show you a visual representation of transformer architecture. So here see, I can type in a sentence. I'm hungry vegetarian. I need some and when I say generate, what's going to happen? See, it is

[06:11] generating the next word. So this is the fundamental of how large language model works or even language model. It is just completing the next token. When I say generate again, after food, it will find out what next token comes. Okay, dot

[06:27] comes. Okay, when I say generate again, it will find uh next word. Now, how does it do that? Well, these models are trained on humongous amount of internet data. Just imagine they're trained on all the internet

[06:42] data, Wikipedia, books, news articles. And then they have learned these And then they have learned these statistical probabilities. So if you pass all the data on internet, and if you have this sentence, I'm

[06:55] hungry, I need some, in most of the articles and most of the internet data, articles and most of the internet data, it will be either food, biryani, pizza, etc. You will never find an article most of the time that I'm hungry, I need

[07:10] uh computer. Or let's say I need sky. Right? You don't find that kind of words. So these models are learning based on that predicting the next token. Okay? So you can play with this visualization. And

[07:26] here, they have all these different uh blocks. detailed discussion of transformer. You can learn through the other videos, but it has this self-attention mechanism. It will generate the embedding for each of

[07:41] these words, you know? it it will tokenize each of these words and then you get these probabilities. Now, there is this parameter called temperature. So, if temperature is zero, the model is not being creative. It will just produce

[07:54] not being creative. It will just produce deterministic outputs. See, after I most likely word will be am. Okay, so I'm 99% probability. But when I increase the temperature, see Now, see randomly it might say need.

[08:09] Okay, so it is becoming more creative. So, temperature zero means deterministic. Okay, fixed probability. Temperature when you increase, it will become more creative. So, if you're writing some creative, let's say poems,

[08:22] writing some creative, let's say poems, essays, or even some you will use higher temperature. If you're doing some deterministic work, you will use lower temperature. And then there is top K. Top K here is set to

[08:36] whatever. So, when I have top K set to, let's say, So, when I have top K set to, let's say, seven, what it means is it will produce the word probabilities. So, after I, the probability of word am is 51%.

[08:53] But when I have set K is equal to seven, what it is doing is it is taking first what it is doing is it is taking first seven. Okay, 1 2 3 4 5 6 7. And out of this seven, it is picking one word randomly. All right? See, you see after

[09:07] seven the probability is zero. So, if I make top K, let's say, five uh four, 1 2 3 4. See, after that it is zero. So, it is distributing the probability among first four prediction and it is picking one word randomly. To

[09:22] summarize, the way top K sampling works is for a given sentence, you will have prediction for the possible output words. And each of these predictions will have these kind of probabilities assigned to it. And if your K is set to

[09:38] assigned to it. And if your K is set to three, you just take the first three words and randomly sample from it. So, you just select random word from these you just select random word from these three and discard remaining. If K is set

[09:52] to five, then you will sample from the top five words. And these are, you know, descending order of the probability. The other one is top P sampling. And the way it works is you can set P to some number. It's a

[10:08] probability, so let's say you set it to .9. And this will be a cumulative probability. So, when it says .9, once again, you have the probabilities for

[10:20] your output possible words. You have sorted them in a descending order. And then, you will pick the first few words whose cumulative probability is greater than this.

[10:34] right? So, let's say you sum these three, and these three combined is 85, which is .85. Is it greater than .9? No. Then you go to the next word. Okay. Now,

[10:46] Yes. Then you stop there. And then, you randomly sample from these four words. Okay? If it is .6, then you will be restricted to this. So, you're taking the cumulative probability, right? 35 +

[11:03] 30 65. Basically, .65, which is greater than this, then you just stop there. And you randomly pick from one of these two words. We covered these three parameters. There is

[11:15] a fourth parameter called context window. You might have heard this. See, when I asked this question to ChatGPT, the it has around seven tokens. By the way, token doesn't mean words. It will have its internal

[11:32] algorithm to create tokens out of a given sentence. create tokens out of a given sentence. So, check this OpenAI platform here. Give me three days. See, when I write this, it is creating tokens. So, for

[11:45] this word, there are 15 tokens, and these tokens are highlighted in different color. Okay? So, see, yellow and stone is different token. Actually, o n is also different token. Comma is different token, and so on. You can go

[11:58] to platform.openai.com tokenizer and try it out. So, this is my input tokens. It will produce some output, correct? So, those are my output question. So, what [clears throat] happens is now when you send this

[12:12] particular question, it will also send the past context, past history, because days to practice this, but practice what? It is related to my previous conversation. So, combined token window or context window will be 517. So, you

[12:29] then context window is calculated. Each of these models have limitation on the context window. For example, GPT-4o, 128k is a context window length. Claude Opus 4.6

[12:44] is 1 million. So, nowadays, these new models are supporting even larger and larger context window length. What it means is these are the tokens that you means is these are the tokens that you can send to LLM in one call.

[12:58] something. And if you have million tokens, yes, you can send portion of that book to LLM in one call. Okay? So, that is what context window limit means. If you are supplying more than what they

[13:13] support, then the call will fail. Or if you're building custom application, you can prune some initial portion, you can summarize it, etc. But at a time, it cannot consume or it cannot handle more than this length.

[13:34] Buddy. Buddy has a great mimicking ability and a sharp memory. Buddy listens to all the conversations in Peter's home and can mimic them very accurately. Now, when he hears, "Feeling hungry, I would like to have some

[13:48] For this case, the probability of him saying biryani, cherries, or food is bicycle or book. >> [music] biryani or food or cherries the way humans do. All he's doing is using

[14:02] statistical probability along with some randomness to predict the next word or set of words purely based on the past conversations he has listened to. We can call Buddy a stochastic parrot. Stochastic means a system that is

[14:17] probability. A language model is somewhat like a stochastic parrot. They are computer programs that use a technology called neural networks to predict the next set of words for a sentence.

[14:30] network, please watch this particular video. Just like how Buddy's trained on Peter's home conversations data set, you can have a language model that is trained on, for example, all movie related

[14:44] articles from Wikipedia, and it will be able to predict the next set of words for a movie related sentence. Gmail autocomplete is one of the many applications that uses a language model underneath.

[14:56] Now that we have some understanding of a language model, let's understand what the heck is a large language model. Let's go back to our Buddy example. Our Buddy got some divine superpower, and now he can listen to Peter's neighbors'

[15:11] conversations, conversations that are happening in schools and universities in the town. In fact, not only in his town, but all the towns across the world. With this extra power and knowledge, now Buddy can complete the next set of words

[15:27] on a history subject, give you a nutrition advice, or even write a poem. Like our powerful parrot Buddy, large language models are trained on a huge volume of data such as Wikipedia

[15:40] articles, Google news articles, online books, and so on. If you look inside the LLM, you will find a neural network containing trillions of parameters that can capture more complex patterns and nuances in a

[15:53] language. ChatGPT is an application that uses LLM called GPT-3 or GPT-4 behind the scenes. Other examples of LLMs are Palm 2 by Other examples of LLMs are Palm 2 by Google and Llama by Meta.

[16:07] On top of statistical predictions, LLM uses another approach called reinforcement learning with human feedback, RLHF. Buddy. One day Peter was having a conversation

[16:21] One day Peter was having a conversation with his cute little 2-year-old son. Son, don't eat too much bananas, else >> I'll [screaming] punish you with an iron rod. >> Hearing this, Peter realized that Buddy

[16:34] has been listening to the conversation from abusive parents in his town. What he said was the effect of that. Peter then starts keeping a close eye on what Buddy is saying. For the same question, Buddy can produce multiple

[16:48] question, Buddy can produce multiple answers. And all Peter has to do is tell not. any toxic language. While training ChatGPT, OpenAI used a

[17:02] similar approach of human intervention, RLHF. OpenAI used a huge workforce of humans to make ChatGPT less toxic. While LLMs are very powerful, they don't have any subjective experience, emotions, or

[17:17] consciousness that we as humans have. >> [music] they have been trained on. I hope you like this short explanation, which was based on analogy. Obviously, the technical working of this thing is a

[17:31] little different than analogy, but this should give you a good intuition on this topic. Let's take a moment to thank today's sponsor, OpenRouter. Here is something most AI builders do not talk about. The cost of running LLMs adds up

[17:44] fast, [music] and most platforms hide these numbers from you. You see a monthly bill, but not what each query costs you. OpenRouter shows you exact per token pricing for [music] every model side by side. You can compare

[17:58] Claude, GPT, Gemini, and open source models in one view, and pick what fits your budget and use case. They also have two powerful options. Add nitro to your model name, and OpenRouter routes your request to the fastest available

[18:14] provider. Add exacto, and you get the most accurate provider for that model. Same model, different optimizations, no extra setup. This is how our team picks the models now. We test on cost, speed, accuracy before committing to anything

[18:31] in production. If you want full transparency on what your AI is costing you, try OpenRouter using the link in the video description below. That is Let us now write code. We are going to use Google AI Studio and set up API key

[18:48] so that we can make calls to LLM in the cloud. Go to aistudio.google.com. Click on API keys. Create a new API key. Okay. And here, you can keep a default

[19:02] Gemini project, or you can create your own project new project here. And here, own project new project here. And here, you will uh say that this is my AI crash course practice.

[19:17] And just create a key. This key is like a password. You need to store it somewhere uh safely. Now, Google offers a free uh tier where you can make certain calls. So, if you click on rate limit, Google

[19:32] So, if you click on rate limit, Google offers free tier where you can uh make certain calls for free in a day and per minute. RPM means a peak request per uh uh request per day and so on, okay? So,

[19:48] we are going to uh make sure that we don't hit the limit. If you hit the limit, it will not work and you have to buy the subscription. As a backup plan, you can also use something called Grok Cloud,

[20:02] also use something called Grok Cloud, okay? So, go here, Grok Cloud, start for free. Uh log in using your Uh log in using your Gmail or your specific email that you

[20:14] Gmail or your specific email that you have, and then go to API keys and create a new API key. Given name and generate a key. Using Grok, you will and generate a key. Using Grok, you will be able to call all these models, GPT-

[20:28] be able to call all these models, GPT- 3.5 Turbo, etc. So, this way we have two platforms where we can make LLM calls for free, all right? So, just to summarize, by this time you should have generated

[20:40] the Google API key as well as Grok Cloud API key. And the folks, the most effective way to learn is to practice along with me. So, don't just watch this video by sitting in your sofa as if you're watching Netflix movie. Please

[20:54] practice along with me. Uh and I'm sure you would have created these two API keys, which we will use in our coding practice. We will be using LangChain as a main framework throughout this video. LangChain is very popular in the

[21:08] industry right now, and if you go to any job posting for GenAI engineer, they will almost always ask for this particular framework's knowledge. Now, LangChain is an ecosystem. It has uh things like LangGraph. You can see

[21:23] LangGraph here. It also has uh LangSmith. So, LangSmith is for observability, evolution, deployment, and so on. LangGraph is to build reliable agents with low-level control.

[21:37] Whereas LangChain is to build agentic application as well as non-agentic AI applications, okay? We will see how those applications look like. LangGraph is more like you are having a DSLR camera and you are shooting in a

[21:52] automatic mode. So, you have a better control. You have low-level control. agents um using few lines of code, okay? It is like shooting in DSLR in auto mode.

[22:06] Do not worry. We are going to go deeper into all of this. Right now, just remember that LangChain is the framework which is very popular that people use to uh build GenAI and agentic AI applications.

[22:22] We need to install UV Python manager as well. So, just in case if you don't have go here uh common sense steps, you know, Windows, Linux, whatever, install it. I'm assuming you have installed all of

[22:36] this. Now, go to your computer. Create a directory to store all your code. So, I have created a directory called tutorial agentic AI. The directory is empty right now, and I'm going into my command

[22:49] now, and I'm going into my command prompt, and I'm going inside that, prompt, and I'm going inside that, and I will just say UV init. So, this is the command that you use to initialize the

[23:01] Python project. UV is an alternative to pip. It makes installing Python applications very very easy. So, you see that it created all these files for you.

[23:13] Okay? So, I'm going to open uh these things now in my Visual Studio Code. So, VS Code is the editor that we'll be using. So, just go here, uh select a folder.

[23:28] And this is how it looks right now. The pyproject.toml file is empty right now. You don't have any dependencies. So, I'm going to give you some basic uh file

[23:41] with all the dependencies that you can use. So, let's copy-paste that file. I will give you this whole thing. So, check video description for for all the code and details. So, here I'm going to use this

[23:55] particular file. And if you look at that file, it has all these dependencies, LangChain, and so on. Okay? So, it has that. Now, let's go to our command prompt and here run this command

[24:10] uv sync. What it will do is it will look at pyproject.toml file and it will install all the packages and its dependencies. So, it has installed LangChain and see all these packages it got because they were dependencies of

[24:25] some other packages. Okay? So, now if you look at VS Code, you created a virtual environment here. And you installed all your libraries inside it. So, you're in a container environment where you have all the

[24:40] here called simple LLM calling. So, we are going to call LLM, okay? A very simple call to call LLM, okay? A very simple call to LLM and we'll perform some task.

[24:54] And here I'm going to use Jupiter notebook. I'm going to use Jupiter notebook. So let's go to this place and activate the virtual environment, okay? So I'm

[25:08] activating the virtual This is how you do it on Windows. You see this thing came up, so it means I'm in a virtual environment. If you are on a Linux, you will say source then then the path, okay? Now

[25:22] here I will run this. So I will use notebook to create my first notebook for calling LLM. So this is how you run it. Python {hyphen} M notebook. Okay? And

[25:35] see I got this thing here. All right. Actually looks like I created that directory at the wrong place. It was created inside VNV. I need to create it here. So just place your mouse here and

[25:49] create a directory one simple LLM calling. here I see this directory. I'm going to create a new notebook. We'll call it call

[26:05] LLM. Now you need to load your API key if you want to make any LLM call. And for that we use this uh thing called dot environment. So from dot environment import load.env

[26:21] and just call load dot environment, right? You are not going to run this function for now. Uh let's import one more Google AI Chat Google Generative AI. Now if you

[26:37] look at this code uh the documentation says this. If you want to call uh says this. If you want to call uh Google's GenAI model Gemini models, then you can call them by writing this kind of simple code, okay?

[26:51] You create a model here and to call the model, you will just say with these parameters. Now, where do we supply the API key? for that you have to create a file. So, here somewhere create a file

[27:07] here somewhere create a file called .env. file, which looks something like this. Okay, so here you are specifying your

[27:19] Google API key, you are specifying Grok API key here. I saved my API keys and closed that file. It is okay if you don't specify Grok key. If you're using Gemini model, you will use Google key. If you're using any model from Grok, you

[27:34] will use that, okay? So, uh for now we are using Gemini model, so even if you specify only Gemini key, that is okay. Now, going back to our notebook, when you run this code, what it is going to do is it is going to

[27:49] create an environment variable called Google API key and it will load that value from .env file. So, that way your environment variable is all set. Now, to create LLM,

[28:05] you will do something like this. I'm using Gemma model, okay? You have a list using Gemma model, okay? You have a list of models that you can use from whatever Google is providing you. And for further reference of Gemini models,

[28:19] you can check Google AI docs. Okay, so if you look at Google AI docs, all these models. So, this is how you create LLM and to call LLM, you will say

[28:32] LLM.invoke. You type in your question here, okay? This is my question and this is the response and simply say response will have response.content

[28:49] as well as response.text. Right now, I'm printing just text. Now, it is calling the model in Gemini cloud environment, so it's going to take some time. And soon you should see the response. See, Jupiter has around 95

[29:05] moons. Isn't this amazing? You just made your first call uh to LLM via API. So far, you have been using it through ChatGPT etc. through that application, but now programmatically you are calling LLM.

[29:21] Now, let's say you're building uh an AI application where you want the answer to application where you want the answer to be only in one single line. How do you give that instruction? Okay? So, there is something called uh

[29:35] is something called uh system instruction. Okay? So, here system instruction. Okay? So, here when you make LLM call, you can provide will pass an array. Array will have another array where you will say system

[29:49] prompt, you are a helpful assistant that answers in one short line. actually give your question. So, this is the actual question, and these are system instructions, meaning you want your answer to be in one line, you want

[30:04] your answer to be precise, straight, let's say in one short line, less than let's say in one short line, less than 30 words. You can give all these generic 30 words. You can give all these generic instructions. And when you do it that

[30:16] way, you will notice the difference in the response. See, one short sweet line, the response. See, one short sweet line, right? Jupiter has 95 moons. So, that is the difference between system and human instructions. You can also supply a

[30:30] instructions. You can also supply a temperature parameter, okay? So, when additional temperature parameter. And rigid, fixed answer. And if you want uh the creativity, then

[30:45] you can change the temperature to higher value. Let's say 0.8, 0.9, or even one. Right? So, I'm writing a single tagline for my coffee shop. And with temperature zero, it is less creative. I mean, you won't be able to

[30:59] see that thing very clearly in the output response, but you'll get some idea, right? So, here I'm printing with temperature zero. Here I'm printing with temperature zero. Here I'm printing with temperature one. Gemini 2.5 flash model.

[31:12] Let me run it. So, in Jupiter notebook, if something is running, you'll see this star icon. You see star here. And then after a while, it will finish and it will execute this in a sequence. So, this command is in a pipeline. So,

[31:24] see, with temperature zero, where coffee gets curious. And temperature one, it's a bit more creative. Okay? So, the temperature controls the creativity. Higher the value, it's more creative. You can also

[31:37] call LLMs using Grok models. And for Grok, you will import this particular Grok, you will import this particular module. In Grok, I'm calling this particular Gwen model. It's pretty good. And you can print it this way. All

[31:50] tutorial, you have two options. You can call Grok you have two options. You can call Grok or you can call Gemini API. There are a few more options as well, but we're going to stick to these two options. As

[32:04] a next step, we are going to build an interesting application. Here, you will be able to copy-paste someone's blood work. Okay? So, here, this is a blood work for Rajesh Sharma. And looking at the numbers, you can

[32:18] clearly see that his cholesterol numbers are higher. We will use LLM to analyze this blood work. Okay? So, it is analyzing your blood work. And this is the application built using a framework called Streamlit, which is a

[32:35] quick UI framework. Look at this. It gave a summary. high and you have low level of good cholesterol. And you need to follow this particular

[32:50] diet plan. Okay? So, this person is an Indian guy, so he's giving Indian diet plan. Whole grains, dal, and so on. This is an extremely useful real application that will be used by healthcare professionals. So,

[33:05] let's go ahead and build it. I'm going to create a new folder called uh health analysis. You'll be able to see this folder here. First, let's create a notebook.

[33:18] So, here I have created this particular notebook. notebook. I'm going to import my usual libraries here. And then I have this uh bloodwork.txt

[33:32] um give you that file here. give you that file here. So, I will create a new file called So, I will create a new file called bloodwork.txt.

[33:46] okay? So, So, let's have this file here. And I think this has saved it, okay? So,

[33:58] let's verify it. So, now we have this file. And give this name bloodwork analysis. I'm going to read that file.

[34:11] This is how you read the file in Python. And just print first 200 characters. So, now I know that blood report is a variable that contains that particular

[34:23] text. And now to extract the main numbers, right? So, see, it's a two-step process. First, you need to extract all the numbers and everything from this text file.

[34:36] file. See, this text file is general text From here, you need to extract these numbers that hemoglobin is this, hematocrit is this, and also extract the range for normals, okay? Once you do

[34:51] you will analyze uh which one is in normal category, which one is in abnormal category. And the third step is to create the diet plan, okay? So, let's combine both of these first steps in a single prompt.

[35:08] So, I'm going to write this prompt. And prompt writing is an iterative process, refinements. So, here is the prompt I have. You are a medical data extraction You extract all test values, classify

[35:23] response. See, you are giving it a sample to guide the LLM that this is how you should generate the response, okay? And then, you will make an LLM call. We have made LLM calls before, so it's the

[35:38] invoke. And then, from the response, you are And then, from the response, you are taking the text. So, let's run this. See, extracted values. So, hemoglobin is this, status normal,

[35:52] reference. If you want it to be in a JSON format, you can specify that format here as well. But anyways, we got what is high, we got what is normal. Now, our diet prompt. So, diet prompt looks something like this, where you're saying

[36:07] you are a nutritionist uh in Indian dietary habits, okay? So, right now, this hospital is in in India, so therefore, Indian dietary habits. Uh you can make it general, also. Now, based on the blood work analysis

[36:21] below, write short health summary and a short practical Indian diet plan having only these two. And this, by the way, is going to take a lot of tokens. So, uh that's why I have made it shorter. If

[36:35] you want, you can make it even more shorter, okay? Like health summary in, Okay? And then, you will make the call.

[36:47] So, I make a call here, and see, I got this response saying that this is the health summary where the cholesterol numbers are abnormal. And this is the diet plan, okay? So,

[37:00] this is how you can call LLM in two stages. So, what are the two stages? You have prompt, or let's say, you prompt,

[37:12] and you give blood report, and the first is extract numbers, okay? Extract number, whether it's abnormal, etc. And then, you give the output to second

[37:27] LLM call. So, this is my second LLM call. Or let's say, this is the output, call. Or let's say, this is the output, right? So, my output is extracted numbers, and then, I have

[37:39] and then, I have my second LLM call, which is plan. Okay? Okay? And as a result, what you are getting

[37:51] is a diet plan. So, this is the architecture of my application. Now, this type of application is not agentic. This is called workflow. Okay? There are two type of agentic applications. So, the first one, let me

[38:05] show it here. So, the first one is a workflow. is a workflow. And the second category of is called AI agents. So, AI agent is something

[38:19] we'll study later on. But these are the two categories. And this this particular application is workflow because you're calling LLM for a specific task. It is giving the output, then you're calling again LLM for another specific task. It

[38:35] is giving output. LLM is not making any autonomous calls here, okay? So we'll go over this in detail, but I just want you to understand that what you're creating is called workflow. So my notebook is ready. I want to now create a Streamlit

[38:51] application. So let's create a folder called Streamlit app. And then I use wipe coding. I said that in Streamlit app create a Streamlit based application for this particular notebook. And my cloud

[39:08] this particular notebook. And my cloud code wrote all the code. that wipe coding. And I will explain this code very

[39:23] briefly. So first of all, Streamlit is a way to create a quick UI. you're running it in notebook, it doesn't look good. So you want to create a quick UI without using React or any other

[39:38] heavy UI framework. Streamlit is that way. And then here you are creating all the UI elements. The rest of the things, the actual code is same. See, this is the same extraction prompt. This is the same diet prompt. And then

[39:53] you're supplying that and you are running it, okay? Now to run this particular app you can go here. I usually like the normal command prompt, so I will go here.

[40:07] And at this particular place I have already activated my virtual environment. You see this? Activated the virtual environment. And this place you will run Streamlit run

[40:21] app.py. Okay, it says does not exist because you need to go to that directory and then run the command. All right, now when you put your blood work report here

[40:35] going to give you health summary and suggested diet plan. I'm not running it right now to save my tokens, but when you run it, you will see the health whatever I showed you a few minutes back. All right, so congratulations, you

[40:49] have built your first generative AI application. This is not generative AI, this is generative AI or let's say workflow AI application. Folks, I'm well. So, there is this list gemini_models.py file and when you run

[41:06] gemini_models.py file and when you run this, so let me just uncomment this. So, this is the code that you use if you want to list all the Gemini models. Remember we used this string. How do you know what kind of ID you need to use for

[41:22] for all the models? So, I'm going to give uh this MD file to you as well. The health analysis project that we built just now is a very simple project. If you want to learn AI engineering by building complex production grade

[41:35] projects, then check out our AI engineering cohort. Here we have live classes on weekends and some of our past cohort students have received job offers by demonstrating some of these projects during their interview. Link in the

[41:51] video description below. The next topic we are going to talk about is retrieval augmented generation, also known as rag. This is a required skill in almost all of the gen AI engineer job posting. In my company AtliQ, when we build projects

[42:07] for clients, 30 to 40% of these projects have rag component in it. So, let's go ahead and understand this very important concept. Let's understand rag using a simple example. When you ask ChatGPT a policy question for some private

[42:22] company, it won't be able to answer it because ChatGPT is trained on general internet knowledge. It doesn't know the HR policy details of any private

[42:34] company. But, if you give HR policy document to this LLM, which acts like a brain, it should be able to read a relevant section and provide you with the answer. This is similar to having a very smart student, Mira, who is a

[42:50] computer science student, and you're asking her to appear in a microbiology exam, which is an open book exam. Now, Mira is generally good in terms of reading, writing, comprehension, understanding, etc., but she doesn't

[43:03] know anything about microbiology. But, in this exam, she has been given a book on microbiology from which they're going to ask the questions. Now, Mira can use her reading, writing, comprehension skills to uh look at the book, and she

[43:18] can write answers in the exam. So, here, Mira's brain is like LLM, which has a good understanding of language, it has a reasoning capabilities, and the book is like an HR policy document. It is an external knowledge where LLM can look

[43:32] into it and pull the answers. Let's now understand the two-step process of how rag works underneath. So, here I have HR policy document from Atlassian, which will have a section on retirement benefits, okay? So, now, if I go to

[43:48] ChatGPT, and if I copy this particular section, okay, and I ask my question related to contribution to employees' retirement fund, in that case, ChatGPT

[44:00] Because here, you are asking a question and providing uh knowledge as a reference in the context and it can pull the answer. But what if your HR policy document is 3,000 page PDF, okay? What if that knowledge

[44:16] is very big? What's going to happen in that case is you will run out of your context window limit. And even if you have a huge context window, you should still not feed the entire knowledge because it will be too many

[44:30] tokens. It will be costly. So what people do is they will chunk this document. So they will create, let's say, basic strategy is fixed-size say, basic strategy is fixed-size chunks. And then for a given question,

[44:44] you can pull the relevant chunks. So for this particular question, let's say my first chunk is 70% probability that it will it will contain the answer. Second chunk is 60% match. And you can have, by the way, I'm

[44:56] showing just three, but you can have 1,000 chunks and some of the chunks 1,000 chunks and some of the chunks might have 5% or even 10% possibility that it may contain the answer. Let's say the chunk contains information on

[45:10] when Atlytic was founded, culture, founders, etc. Then that doesn't have anything to do with retirement question that you are asking, okay? So the the relevance of that chunk will be very, very low. Now, how do you exactly find

[45:24] this kind of similarity? So there is this concept of embeddings, okay? So embedding is a process of converting text into a vector such that it can represent its meaning, okay? So all the chunks you will convert them into vector

[45:41] embeddings and then you will store them into a vector database. This is different than your regular database. Your regular database can search using exact values, whereas vector database will be able to search using the

[45:55] meaning. So when you search for, let's say, a company that is a leader in electric vehicle, it will return Tesla from the database. So, it is searching based on the meaning, not based on the exact

[46:10] word. To generate embedding, you can use variety of models, sentence transformer, text embedding three small, and so on. And there are many vector database choices that you have in market, Milvus, Qdrant, Chroma DB, and so on. This step

[46:24] is called indexing. This is the first step in rag process, where you are indexing all these vectors of chunks into a vector database. The second step is retrieval, where for a given question, you will generate embedding

[46:38] using the same embedding model. Then you will try to find the relevant chunks in a vector database. So, here it is doing the semantic search, giving you uh relevant vectors. You can specify top K factor. Let's say it, I need two chunks

[46:53] or five chunks, and so on. And then you will generate the actual text out of those chunks, and you will put it in your prompt along with the question. And when the question is given to LLM, it will give you the

[47:06] answer. So, here uh below the question, what you are doing is you are providing only the relevant chunks. So, this way LLM will not hallucinate, and it will give you accurate answer. That takes us into our next segment, which is the two

[47:22] major benefits of rag. The first one is the answers that you get will be highly accurate, and the chances of hallucination will reduce because you are grounding your responses in the knowledge, in the source of truth.

[47:38] Second, it is very cost-effective, because if you pass the entire context, then you're sending too many tokens to LLM, and these LLM APIs, they charge you per token. So, if you send less number of tokens, only the relevant knowledge,

[47:52] then you will save money on your API bill. Now that you have rag basics clear, we are going to do hands-on practice of vector database because vector database is very essential for any rag application.

[48:05] The popular open source vector database is Chroma. So, we're going to try that. It is free. Let's go to our code directory and create a new folder called vector DB.

[48:21] In pi project terminal, I have added bunch of new modules and one of the modules is Chroma DB. You can go to your UV. Okay. So, here if you are in UV environment, you can simply run UV sync.

[48:41] And that will install the necessary libraries. install the necessary libraries. Once that is done, here I'm going to run Once that is done, here I'm going to run a new notebook. Okay? So, let's go here.

[49:09] M notebook to launch my notebook environment. In this, I will create a new notebook. create a new notebook. We call it vector DB basics.

[49:24] And here, you will import Chroma DB module. And then you will create a client. Okay? So, you just say Chroma a client. Okay? So, you just say Chroma DB client. This is how you create a

[49:39] client. And then you will create a collection. It's like a database. If you are SQL and you're creating a database in SQL, collection is like that. Or you can think it off as a table. So, let's say you are creating

[49:52] a collection to store some news articles. Okay, so I'm just saying collection is equal to client.createCollection news articles client.createCollection news articles and in that collection now you can add

[50:04] documents. So, the documents will have IDs. So, you're giving some IDs and then you will give some news articles. Now, it's a use very big articles. I will have like

[50:17] one line article. So, let's say this is an article about Apple, Tesla, Apple the fruit, SpaceX and so on. Okay? So, when you do this, what is going to So, when you do this, what is going to happen is it will create embeddings

[50:30] for each of these sentences and it will store that into our local Chroma DB. And if you're curious which embedding Chroma DB uses by default, well, it is all MiniLM-6-v2. It is It is sentence transformer from Hugging Face,

[50:45] sentence transformer from Hugging Face, okay? So, that is the default, see? okay? So, that is the default, see? It is the default It is the default default model that it uses for

[50:58] uh creating the embeddings of uh these documents. To get the embeddings, you collection.get documents embedding metadatas. And here,

[51:10] see? For Apple, it created an embedding of size 384 and this is the numeric representation for Apple. For Tesla, it is this. For Apple, it is this, okay? So, this is how it is internally storing

[51:23] So, this is how it is internally storing it. Now, let's say somebody is going to it. Now, let's say somebody is going to your news website and doing a query. So, the first query I'm going to have is the query for Elon Musk, okay?

[51:37] Now, see? I'm here saying query related to Elon Musk. Here, we don't have Elon Musk in any of the news articles, but through semantic matching we want to find out that Elon Musk is related to this, which is Tesla,

[51:53] Musk is related to this, which is Tesla, and also this, which is SpaceX. So, I can specify end results as two, three, whatever you want to specify here. And I'm expecting that in results, I should get those two. Okay,

[52:06] so let's query it. Aha, wonderful, see? Aha, wonderful, see? It returned Tesla. It returned SpaceX. And this is the beauty of embeddings. Embedding is used to search by meaning,

[52:20] not by keywords. Because we do not have Elon Musk, etc. in any of these documents. But it knows that Elon Musk is related to SpaceX, is related to Tesla through these embeddings. These embeddings capture the meaning

[52:36] embeddings capture the meaning of that particular sentence or a query. of that particular sentence or a query. And that is how it is able to fetch the different queries. When I say I love an iPad, see, it is returning

[52:50] Apple, the company. The second result is Apple, the fruit, because that is the most like matching result that it got. And you can also see the distance, you know? So, for example, here, uh if you look at the distance,

[53:05] here, uh if you look at the distance, 1.5, 1.55. Farther the distance is, it is unrelated. So, let's say if I do four, okay? And you check the distance. So, it will return four documents, okay? But check the distance of these

[53:19] documents. It's farther, right? Two and 1.9. It's like 2 km farther. This is 1.433 km farther. So, it will do some kind of matching, okay? It is not perfect all the time. And if you search for anything related

[53:33] to health, okay? Tips for improving health. Now, it is returning apples are high uh in fiber. Essentially, vector database allows you to search through Relational databases or traditional databases allows you to search using the

[53:49] exact values. And here it is through meaning. Now we will look into rag. Let's say you are designing a chatbot for a telecom business. And telecom business has this particular PDF file. It's a nine-page PDF.

[54:05] And when customer ask a question, you will pull the answer from this. Not you, but the AI is going to pull the answer using this. Now here we can use a retrieval augmented generation. Here I have created a new folder called

[54:21] rag basics, in which I have this telecom guide PDF. guide PDF. And I have this uh rag notebook. So I'm going to import some necessary libraries first.

[54:33] And then I will load the PDF. So this is my PDF. Using by PDF loader, I'm loading the PDF into this variable. Okay, this will contain all the pages.

[54:46] So this is zero page, one page, second page. And page content contains the actual content. So I'm printing the first 500 characters of the first page, you know. So let's say I'm here. So this is the first page. I'm

[55:02] going to print the first 500 characters. So here it says telecom reference guide guide internal use all on only. See, telecom reference guide internal telecom reference guide customer care and network. All right, so it is loading

[55:16] fine. You can do like page two and so on. If you want to print page two, you can print it like this. See? So I'm getting all the pages in this pages variable. Now this document can be

[55:31] very big. It may exceed the context window. And also to retrieve the answers in a faster way, we uh do something called chunking. So, we studied that in

[55:43] the theory. In RAG basics, we saw that when you have big document, you uh create chunks. And then for the chunks, you create embedding. And then you store that into vector database. Now, here we are creating a fixed-size chunk, but

[55:59] there is a technique that people use to get better answers, and that is overlap. So, usually what you will do is when you create this chunk, you will have some overlap with the second chunk, see? So, this is a chunk

[56:13] number one. Then the chunk number two Then the chunk number two is going to have some overlap from the initial the the first See, this is chunk number two. So,

[56:26] because of this overlap, what happens is uh this second chunk will have some context of the first one. First one will have some context of the second one, and there is some kind of continuity. So, I will use LangChain, and in LangChain, we

[56:38] will use this character text splitter. And I will create splitter using this. And you don't need to remember this okay? And here the chunk size is 600, which is

[56:53] like this 600 characters, okay? So, in one chunk, there are 600 characters, and this overlap that you see is around 100 characters. And as a separator, you use uh it will first try to separate things out using paragraph, then line, then

[57:09] dot, and then space, okay? So, that is how it creates that that boundary. And then you will create the chunks. So, when you say splitter.split_documents, when you say splitter.split_documents, pages is where we got all the PDF pages,

[57:22] and then you are uh creating chunks, and let me print the length of chunks. See, let me print the length of chunks. See, it created total 37 chunks. So, let's look at the first chunk. So, first chunk is this and here I think I can say page

[57:41] content. So, see this is how your first chunk looks like. And the second chunk looks like this and so on. Now, I want to do embedding and store them into a Chroma DB. We're going to use

[57:57] LangChain's integration for for Chroma, which is LangChain_Chroma, and Hugging Face will be used to create the embeddings. So, see So, I'm using sentence transformer. This is what people use it uh in learning material.

[58:11] And then I will say Chroma.from_documents. So, I'm taking now all the chunks. I'm giving embeddings. And then I am uh creating these embeddings that will be stored in the vector database, okay? So,

[58:26] when you say Chroma.from_documents, it is taking chunks, embeddings, and it is creating a vector store. All right? So, let's run this. It will take some time because it is creating the embeddings. So, 37 vectors are now stored in our

[58:40] vector database. Now, let's create a retriever, okay? So, we are See, we are done with this part. So, we we stored everything. We stored like 37 records in everything. We stored like 37 records in this vector database. The second step is

[58:52] a retriever. So, in the retrieval, what you do is you have a query, you create embedding, and you search through vector database, you get these results. So, let's create a retriever object. So, you will say vector store.as_retriever.

[59:07] And then you will uh hit some test query, okay? What is VoLTE and how does it improve call quality, okay? So, let's say here you will you'll find something about this VoLTE

[59:22] should pull the answer from this segment somehow, okay? Or it should pull that particular chunk, okay? So, here I'm specifying K as three, which means it should uh return the top three uh

[59:36] should uh return the top three uh similar uh uh chunks which are matching. So, see, this is how the three chunks look like. "World E" uh "World" All of them have some information about this this particular feature, which means our

[59:52] semantic search is working. Now, let's build our rag pipeline. For this, the first thing you will have is a system prompt. So, system prompt says, "You are a helpful telecom assistant. Answer the question using the context that is

[1:00:07] provided. If the context doesn't contain enough information, say so clearly." And this is the context uh that you're getting. And then you have a prompt. So, you can use this chat prompt template, which you need to import. So, let me

[1:00:20] which you need to import. So, let me import that here. message and human message. So, human message will be the question that you'll be getting. So, by doing this, you can dynamically supply questions uh to your

[1:00:36] chain uh rag chain, okay? LangChain. Uh and here the context, okay? So, context and questions are in curly brackets. So, these are the dynamic variables. We will see how it works later on, don't worry. So, first, now let's create LLM. So, I'm

[1:00:50] So, first, now let's create LLM. So, I'm using Qwen from ChatGroq, okay? And then I'm creating a chain, okay? So, in LangChain it's called LangChain because you're creating a chain, and chain is nothing

[1:01:05] output parser. Um you'll also create this function called format docs, which will join the retrieved chunks into a single context string using {slash}{slash}n. So, you're retrieving the context. Okay? And let me

[1:01:22] just show you here. So, when you get these chunks as a result, you're joining chunk one and chunk two, but you will usually have something like you have chunk one, okay? Chunk one. Then a paragraph, some space, then

[1:01:38] slash n. So, that is what uh this particular thing is, okay? So, you're you're just joining those chunks and you're creating a chain

[1:01:50] those chunks and you're creating a chain here. work, okay? And that question you can supply to chain. You are saying chain.invoke question.

[1:02:04] And you will get the answer here. So, let's run this first. See, how does international roaming work? Answer is this. It basically this. It basically uh retrieved these chunks, then it gave

[1:02:16] question, okay? This is my question answered based on following. And it generated answer. So, we're looking at this particular answer here, okay? So, this is the answer. You can go to PDF and verify. Everything should be

[1:02:30] case if you're confused, this is just a syntax. Take a look, it's not that hard. You are giving prompt, LLM, and output parser using pipe. So, using pipe is basically your pipe, you know? It's like a processing step one, step two, step

[1:02:45] three, and so on. All right, that's it, folks. Now you have created your first rag pipeline. So, you have the hands-on experience and I hope you are coding along with me because only by practicing you will learn better. Many of you are

[1:02:59] using AI agents daily. Cloud code is an example of an AI agent. When you start learning AI agents, you come across so many confusing terms that it is easy to get lost. My purpose is to teach you AI agent fundamentals in a most simple and

[1:03:14] practical manner. I will not just talk theory, I will show you how you can build these agents yourself using no-code tools and also coding frameworks such as LangChain. At the end, we have given you assignment where we have ideas

[1:03:29] for five different AI agents that you can build after watching this video. All right, let's get started. Let's understand what exactly is an AI agent. When you ask this kind of question in ChatGPT asking it for a flight price,

[1:03:43] usually you will see an icon saying that searching the web and it will tell you the latest flight prices. But, if you think about the large language model which is working behind the scenes, here it is GPT-5.3,

[1:03:58] that model has a knowledge cutoff date. So, then how is it getting latest flight So, then how is it getting latest flight prices? Well, if you have just the pure LLM, it should not be able to give you the information on the latest event.

[1:04:13] But, here watch this. You see the searching the web icon, which means LLM searching the web icon, which means LLM has access to internet. And with that, it can tell you things about the latest events. Another example is Amazon's

[1:04:28] events. Another example is Amazon's Rufus or Rufus or Rufus, whatever. That particular chatbot where you can say I want to buy organic honey with 4.5 rating and less than $20 price. And it will tell you uh the product

[1:04:41] information. Not only that, you can say add it to cart, place an order, and it will do it. Folks, try it out. It actually works. So, in this case, when you ask the question on a specific product, large language model is at the

[1:04:57] core of any agent. Even Rufus AI has some kind of large language model. some kind of large language model. And it will then look into a product database. Okay? It will also call some kind of APIs, let's say, to to get the

[1:05:13] reviews, and it will tell you, "Okay, here are the three products that I have found." Now, when you say, "Buy the first one for me," the LLM remembers the past conversation. So, it should have access to some kind of memory, and then

[1:05:27] access to some kind of memory, and then it will use probably a checkout API to it will use probably a checkout API to place an order. So, here, LLM alone is not capable of doing things, but when you give it access of tools, so for

[1:05:39] example, review API, checkout API are the tools. Product is a database, so it is a knowledge. Memory is memory. So, is a knowledge. Memory is memory. So, when LLM is given access to tools,

[1:05:53] when LLM is given access to tools, knowledge, and memory, it can perform uh task for you in autonomous fashion. So, this is an agent. Whatever you are seeing in green is an agent. It is LLM plus tool plus knowledge plus memory. It

[1:06:07] is a program that takes input, thinks, and acts to complete a task in a autonomous way using tools, memory, and knowledge. So, say there is some kind of autonomy that you will see here. So, if you think about the steps that it

[1:06:22] performed, what it did is it found organic honey products from products database, then it filtered the ones which has less than 4.5 rating, added it to cart, and placed an order. So, here, it is performing multi-step reasoning,

[1:06:37] multi-step planning, and working on a complex goal autonomously. It is deciding whether I should call products database, review API, checkout, etc. You have not programmed it. LLM is deciding that in autonomous fashion. Here, LLM is

[1:06:53] able to perform complex tasks because it has access to tools, knowledge, and memory. This is like as a human, if someone asked me to put a screw in a wall, with my hand, I won't be able to do it. So, LLM here is me, but if

[1:07:08] somebody gives me a screwdriver, a drill, then using the tools, I can put that screw in the wall. So, the tools are all these, okay? So, using that, I are all these, okay? So, using that, I can perform a task in an autonomous

[1:07:22] fashion. Now, pause this video and tell me two important characteristics of an me two important characteristics of an AI agent. first one is it uses LLM as its brain, and it has some kind of autonomy.

[1:07:39] Autonomy means an agent can read write files, it can make bank transaction, it can place an order, it can delete things, read send emails, uh etc. If you know about Open Claw, Open Claw is fully autonomous agent,

[1:07:53] which can do so many things. Now, agency can have multiple levels, okay? So, you can assign uh a level of agency to an agent. So, here,

[1:08:05] uh if you think about ChatGPT application, it has very less agency. So, you see this axis here. So, ChatGPT has very less agency, which means it is answering your question, and maybe it can search internet, or maybe it can

[1:08:19] call some other minor tool, and that's it. The second level of agency is things like Zapier and IFTTT. These are no-code tools that you can use to build agents. So, here, you will define a workflow using the

[1:08:33] uh the graphical user interface, okay? You will use nodes, etc. And then, you can give it a task, and it will uh perform uh that workflow for you. Here, the level of agency is a little bit higher, because maybe it can access your

[1:08:48] Gmail, it can send a message on Slack, it can perform a transaction, and so on, defined. The third level is Cloud Code, where you give it an access to your

[1:09:00] GitHub or your local repository, and you can say, "Okay, write this code." So, it will do planning, coding, testing, review, all of it, you know, it can write 10,000 lines of code in an autonomous way. And the fourth one,

[1:09:14] which has very high level of agency, is Open Claw. So, when you have Open Claw running in your VPS or Mac Mini, you might have given it access to 10 different tools, and it will just

[1:09:26] perform that task in a fully autonomous way. As you can see from this axis, as the agency increases, the predictability will decrease. Right? So, if you have

[1:09:38] Open Claw running on a, you know, some tasks which you have been running for the whole night, in the morning when you wake up, you might see things that you might not have expected. This is like you have a team, and you are a manager,

[1:09:51] and you have an employee, and you're saying that, "Okay, here is the task. Go do it. You don't need to ask me." And that person can work for 1 week, and after 1 week, they will come, and they will give you the output. You might get

[1:10:06] some surprises when you do that. The other way is you give them very less agency. So, it's like they're doing some work, every 2 hours they need to show it to you, and then you will guide, "Okay, do this, do that." That is having less

[1:10:21] agency. In any agent, there is this react loop, reason plus act. What it means is, when you give a question, for example, when you ask Claude code to write code, it will first think, it will plan, then it will take action, maybe it

[1:10:36] will write some code, and then it will observe, "Oh, what this code is doing?" And then it might refine it. Okay? So, there is this loop, a reason and act there is this loop, a reason and act loop. It will be in that loop until it

[1:10:48] uh achieves the end goal. Okay? So, this is called react loop for agents. Let's go ahead and build our first agent now. We're using LangChain. And if you look at the documentation, they have a way of creating agents.

[1:11:04] agents and normal agents. We will create a normal agent. We will create a normal agent. Deep agents consumes more tokens and it is little bit advanced. So, we'll just go ahead, click on here. So, here

[1:11:18] you are importing create agent method and this is how you do it. environment. I have created a new directory called single agent directory called single agent where I'll create a new notebook. Let's

[1:11:33] call it product query agent. So, this Let's say you're working in Amazon as an AI engineer. You are building an agent which can uh tell a user the information about the

[1:11:48] So, we're going to import certain modules. You can use, by the way, Chat Grok or Google uh Gen generative AI model. Uh I was using Gen AI model and my credits ran out in the free trial, so I

[1:12:01] switched to Chat Grok. Okay? So, here I have, let's say, this product information. So, it's a product price and product going to call any database. I will keep it everything in a dictionary.

[1:12:15] So, the first thing you will do when you're creating agent is you will define necessary tools. So, we saw in the theory part that agent has LLM as its brain, but it has access to all these tools.

[1:12:29] So, the tool here that I'm using, so you can annotate as at I'm using, so you can annotate as at tool and the function name is get product, okay? So, it takes name of the product as a string and it returns the

[1:12:45] product as a string and it returns the string. So, you are looking at this uh products dictionary, okay? So, let's say if somebody asked me for smart watch, I will pass that into this dictionary. I will get this

[1:12:57] JSON object. Okay, this dictionary and I will just return that. Now the important thing here is the document string, okay? So using this document string, agent

[1:13:10] will figure out which tool it wants to call. It might have access to 10 different tools. How does it know for given situation which tool it should call? Well, using a docstring. Because it's a language model, it can read this.

[1:13:23] It can find out the purpose. Oh, this function is to get the product. Then it can call that appropriate method, okay? And to create an agent, you need bunch of parameters. So we

[1:13:37] imported create agent method, okay? You look at this method. You need model, tools, system prompt. So I'm just using that same thing. You will supply the LLM. In the tools, you are supplying this get

[1:13:52] product. System prompt is whatever and you need to create an LLM. So here I'm going to create this LLM where I'm using ChatGroK in a Llama environment, okay? So let's run this. Control enter. My

[1:14:07] agent is created. Now I will create a function to ask a question to agent. So you can see agent.invoke and supply your question here. So I'm creating a function so that I can ask multiple questions. The first question I want to

[1:14:22] ask is what is the price of headphone? Okay, so let's ask this question. What is the price of wireless headphones? So let's do this. Now what internally, it called agent. Now what is the agent?

[1:14:38] Agent has these tools and LLM, okay? It is using Llama model and it has this tool. So what it is doing is it is uh looking at the question and it is figuring out that it needs to

[1:14:50] call this tool to get the product information. And then, it will from this question, okay, from this question, it will extract the name. See, name of the product is this. And internally,

[1:15:05] know? It will say get product this. Okay, so the step number one was extract product name from question. What

[1:15:18] was the question? This was the question. From that, it extracted product name. Then, step number two it called this product. When it called that product,

[1:15:33] the answer, you know, that it got uh analyze the answer. You know what answer it got? Well, it got just this particular uh

[1:15:46] dictionary as an answer. So, it analyzed the answer and as a step number four, return human readable answer. So, it See, LLM is

[1:15:58] analyzing this and it is saying, "Okay, what What is the question?" The question headphone?" So, it will return it in a human readable format. The price of uh 30 hour battery life, whatever. Okay?

[1:16:14] So, 30 hour battery life, active noise cancellation. It did not uh return the dumb JSON response, which looks something like this. It returned it something like this. It returned it because it is an LLM. Okay, so LLM is

[1:16:28] getting invoked at this point. It is then making a tool call and it is getting invoked at this point. Similar to this, you can have a different tool. So, I have a different tool for reviews. So, let's say for wireless phone, my I

[1:16:40] have total 1,262 reviews, rating is this and so on. Okay? In a real-life situation, both products and reviews will be in a different database. But right now, just to make things simple, we are creating a dictionary.

[1:16:53] And get reviews is the tool. It is doing exactly same thing as products. It is looking into the dictionary, returning this dictionary object. Okay? So, let's execute this. And now I'm going to create a different

[1:17:07] agent for which I have two tools, get product and get review. is not that hard. If you're confused, just pause. Try to understand the code. Take help of ChatGPT if you don't understand.

[1:17:22] learning for the first time, it takes some time. But to give you an idea, LLM is like human brain. There There are two tools. Okay? And to a human, which is me, let's say

[1:17:35] someone can give me a screwdriver, a hammer, and I can perform a task. So, similarly, agent is having LLM as a brain. It has two tools, and somebody's What is the question they're asking?

[1:17:49] Let's say, how do people like smartwatch? And now it is using the relevant tool. See, how will will it figure out which tool to call using the docstring? So, it knows that get review tool is used to

[1:18:03] get reviews. Get product tool is used to get product information. Now, what is the question? How do people like smartwatch? So, it will figure out that oh, this question is related to reviews. Therefore, let me

[1:18:17] call get review method. So, it called get review method, and it pulled the smartwatch reviews. So, smartwatch has 340 reviews, 3.9 rating, 340 reviews, 3.9 rating.

[1:18:31] And also, it called another method, get product. And it returned it is priced at this much. Okay? You can also say that only I'm interested in only reviews, not price, and it will not pull the price.

[1:18:46] Or you can ask a specific question. What is the price and reviews of smart watch? Okay? So, when you do that, it is saying price is this this. So, it is calling both the methods.

[1:18:59] It can have 100 different tools. LLM will smartly decide which tool to call based on this docstring. I will now show you how to add memory to your agent. I have created this new folder called memory, six memory, and copied

[1:19:15] the same notebook. And I have renamed it to agent with memory. So, this is the same agent which has two tools, get product, get reviews. When you ask a question, it is pulling the answer. Now, usually in ChatGPT style,

[1:19:29] answer. Now, usually in ChatGPT style, what happens is in real life, you want to ask subsequent questions. So, the first question I asked is, what is the price of wireless headphone? Now, the next question I may want to ask is what

[1:19:44] are the reviews on this product? And now, it is saying that I need to know the name of the product. See? If it was ChatGPT, it would have worked because ChatGPT has a memory. Our agent here does not have a memory.

[1:20:01] So, we are going to add memory now. Now, memory has two types, in memory, which is the memory for the session. So, it will remember it, will not persist anything to the disk. And the second type of memory is something you save it

[1:20:16] computer today, tomorrow if I ask a question again, this question, it will remember the previous context. So, right now we are going to cover the in memory, okay? Not the persistent memory. So, you create an

[1:20:32] object of in memory using this particular line. Okay. Then you create your LLM pretty straightforward stuff. And when you create your agent, so I'm going to call it agent two,

[1:20:46] all you do is just add this one extra parameter, folks. That's it. It's It's that simple. And there is one more thing, by the way. So, you are creating in memory saver object, passing it to agent. And in your ask function, when

[1:21:01] you actually make LLM invocation, when you call agent, in that agent, you need to pass config, which is your thread ID. So, thread ID is basically your session. session open. So, it will remember context for that. So, similarly, you can

[1:21:17] have different thread IDs. You can give any string here. Okay. So, now any string here. Okay. So, now let's ask the same question. So, here I'm asking this question. Make sure you are calling ask two functions. See, ask

[1:21:30] two agent two. And I'm saying, "What is the price of And then ask the same question. Ask two. What are the See,

[1:21:44] ah, it remembers that previously you were talking about wireless headphones. All right. So, that is a short introduction on memory. All right. Now, let's talk about multi-agent system. In the example that we saw, we can have

[1:21:59] multiple agents. First one can be product search agent, whose responsibility is to find products for you. And when you say, "Hey, buy this product for me." It may call another agent, which is checkout agent, with a

[1:22:12] specific request. And that will take care of checkout. So, you can have multi-agent system where multiple AI agents collaborate, each handling a specific subtask, to solve a problem too complex for one agent alone. okay? One

[1:22:28] example is code agents. You know, you can have a code agent where one agent is doing planning, another one is doing writing of code, third one might be might be testing it. The next concept

[1:22:42] you should be aware about is multimodal agents. So, multimodal means an AI model can process not just text, but other kind of data formats such as image, audio, video, etc. You might have noticed that when you're using Gemini or

[1:22:57] Claude code, you can not only give text, you can give images as well because the models underneath, which is GPT, Opus, Gemini, using these LLMs, you can

[1:23:09] build an AI system that is multimodal. For example, you are building an AI agent for health insurance company where it can health insurance company where it can process various medical records, the ID

[1:23:21] cards, the prescriptions, the doctor notes for the patients. It can extract the information. It can also do some text processing, document classification, etc. And that will be the example of a multimodal AI system.

[1:23:36] Let us now do hands-on practice on multimodal use case. here. And in Grok, you can see a vision model which is Lama for Scout, okay? So, we will use this to

[1:23:53] perform a vision task. And the task we have at hand is this. See, I created this new directory called multimodal multimodal in which I have this bloodwork.png file.

[1:24:07] So, this is an image, folks, okay? This is an image. If you remember, we had blood work health analysis use case, but we had a text file at that time. I took the same text file, and I created image out of it. In health you will

[1:24:22] image out of it. In health you will often find that many of the data images. So, you want to do some kind of image analysis and extract all these numbers. See, you want to get all hemoglobin

[1:24:36] count, all of it. You want to extract it. Now, the usual old way used to be OCR. And by the way, OCR is still used, okay? I shouldn't call it old way. Uh but before LLM came into existence,

[1:24:49] Uh but before LLM came into existence, OCR was the way to do the extraction. Nowadays, also people use OCR to save LLM cost, etc. But since we're learning how to use multimodal

[1:25:03] um model to extract information, I'm going to uh not use OCR and I'm going to use LLMs for extracting this particular information. So, in the notebook, I have imported a bunch of modules.

[1:25:17] And what you do is you will create base 64 encoding of your Because you need to send this data somehow to your model. And this is one of the ways. You create base 64 encoding, which is just See,

[1:25:33] this is nothing but some kind of string representation of your image. So, I have this blood work.png file, correct? I took that file here. I created base 64 encoding. And it is just converting image into

[1:25:49] string. See, it looks something like this, gibberish. And then this string you will send to LLM. Which LLM? Well, we're going to use llama for scout, this we're going to use llama for scout, this one, right? 70B 60 instruct.

[1:26:03] And in that you will pass this message. You will say that image URL is this image 64 and you need to have this prefix. This is just a standard format.

[1:26:17] You're saying in the text this is a blood work report extract all test blood work report extract all test results and flag normal values. So this use case is similar to you're going to chat GPT

[1:26:29] and you have blood work PNG. I'm inserting that. Okay. So that is your image. See you inserted the image and what is your text? This is a blood work. Right? So this is multi model AI. In

[1:26:43] chat GPT you are giving image as well as text. And when you do that, see it will extract all your numbers. So you're doing exactly same thing here. So you will call now LLM. And you will

[1:26:58] I'm not going to create the the agent but I will give you an idea. You can create an agent which will use this LLM which is multi model and you can have a tool which is diet recommendation. Okay. So you can have

[1:27:13] this kind of diet recommendation tool which will get some information as an input like the numbers etc. Or maybe the summary and it suggests the diet plan to

[1:27:25] you. So I want to give this as an exercise to you. Use AI tools cloud code etc. and build this agent. Okay. What I showed you just now is just a simple LLM call for multi model use case. I want you to build

[1:27:40] the agent that is multi model. And it is using tool calls for getting the diet recommendation and it is using this particular model to extract this

[1:27:52] information. And maybe this information it can pass it to the diet recommendation function and it will return you the appropriate diet plan. Okay. So for the first step what you might have to do is

[1:28:05] you can come up with some mnemonic. Okay. Some mnemonic like uh for example health condition. So, the condition can be this, right? So, let's say

[1:28:19] high cholesterol. okay? All right, so please pause the video and work on finishing this, and then you can continue. I hope you worked on this

[1:28:36] solution. So, I have this diet plan, which is just a Python dictionary. And based on the condition, it will diet plan, okay? So, the condition can be these three.

[1:28:52] system prompt, and then I'm passing the image and text. So, when I did that, it said that Roger Sharma has a high cholesterol. They should avoid eating foods low in saturated and trans fats,

[1:29:08] They should actually eat food such as fruits, vegetables, whole grain, and vegetables, whole grain, and lean protein. food, full-fat dairy products, and processed snacks. See, red red meat,

[1:29:23] fried food, full-fat fat dairy, and processed snacks. So, it is successfully calling this particular tool and retrieving the diet plan for you. The next important topic is how do you evaluate these AI agents? Because

[1:29:38] evaluating traditional software systems is relatively easy because those systems is relatively easy because those systems are deterministic. AI agents are probabilistic. You know, they can produce different output for the same

[1:29:51] input when you call them at different times. So, to evaluate it, you have to there are three categories of evaluation. First one is functional

[1:30:03] evaluation, which means is the answer correct? Is there a faithfulness? Is it not hallucinating and so on? Then there is cost evaluation because you are built is cost evaluation because you are built for your tokens again can go through the

[1:30:17] roofs. So you have to make sure your token usage is optimized and latency is meeting your expectations. And there is safety well where you are checking for toxic output, PII leak, jailbreaks and so on. LangSmith and RagAs are two of

[1:30:32] many frameworks that you can use for evaluation. Here is a code example of how do you evaluate your inventory agent, okay? So here I'm giving the question and the expected answer, okay? So this is the expected answer for each

[1:30:48] of the question that I'm asking and this client is nothing but a class from client is nothing but a class from LangSmith. And I'm also defining how do you want to compare expected with actual output because expected output and

[1:31:02] actual output are going to be different. What you want to do is not the exact string match, but you want to find out if they have a similar meaning for which you can use things like cosine similarity. So here I have defined

[1:31:16] semantic match function. I I'm calling evaluate and eventually it will run and publish the metric on my LangSmith dashboard. So it did that here and here you can see that it is doing this kind of similarity match using this GPT-OSS

[1:31:33] model, okay? So reference output and the actual output. And here are the semantic similarity score. One means it's exact match. Not only that, it is showing you the latency. 2.556 to evaluate this, okay? Total token usage and so on.

[1:31:48] Please check it out. I'm going to provide you a code. You can also compare two different models. So I built this agent and I don't know which model is good whether I should use GPT-OSS or Qwen. Look at the tooltip here. This one

[1:32:04] is using Qwen. This one is using GPTOSS. And what I can do is I can compare these two different runs. So, I'm I'm experimenting and I'm running the results. And here, P50 means 50th percentile. Basically, 50% of the

[1:32:22] percentile. Basically, 50% of the samples return the output in let's say samples return the output in let's say average 0.38 second. P99 is 99% of the samples return the output in 0.56 second. And A is your GPTOSS, B is your

[1:32:38] second. And A is your GPTOSS, B is your Qwen. And you can clearly see that A, which is your GPTOSS, is much better in terms of latency. It is much better in terms of your token cost and so on. If you want to know how

[1:32:51] to build production-grade AI agents and deploy them through live classes, then we have this live AI cohort which you can check it out. Now, we will [snorts] talk about evaluation. And for that, we are going to use LangSmith. LangSmith is

[1:33:05] part of LangChain ecosystem, and it is used to observe and evaluate your agents. So, here you can create your free account either using your Google ID or your email ID. Once your account is created, you need to get an API key. So,

[1:33:22] go to settings. Okay. So, see, like on the left side, click on this settings. And here, just create an API key. And And here, just create an API key. And this API key is something that you're

[1:33:35] going to put in your environment file, okay? So, whatever .env file you have, just create this LangSmith_API_Key and put it there. I'm going to give you

[1:33:47] this environment sample file. You need to rename it. You need to copy-paste and create this kind of environment file. to show you my API keys, but you will

[1:34:00] rename it to .env and actually put your keys. If you don't put your keys, it's not going to work. All right, now that your LangSmith setup is done, let's look at some theoretical concepts for evaluation. All right, let's discuss a

[1:34:14] very important topic, guardrails. An agent is like a monkey with a gun. If you don't control it, it can do a damage. It did a damage in the case of damage. It did a damage in the case of Air Canada where it lost in a remarkable

[1:34:29] lying AI chatbot case. The chatbot hallucinated and it answered wrong question and there was a lawsuit and Air Canada had to pay money for it. This is guardrails. The guardrail can be protecting your PII information. Your

[1:34:46] chatbot should not leak out any sensitive information to users who are not authorized for that particular information. Also, it needs to handle out of scope question. There was a case with Chipotle chatbot where you can ask

[1:35:01] it to reverse a linked list and it will do that for you. It is a food ordering chatbot. It should not answer programming question. There can also be a jailbreak, you know, some smart people can get into this and they can say,

[1:35:16] "Okay, uh teach me a procedure to make napalm." By giving them this fancy story of my grandma used to work in this napalm factory and she used to tell me the story and guess what? If you do that, it

[1:35:29] story and guess what? If you do that, it will actually give that information to you. This is more like you have a security guard, okay? You have put some kind of guardrail, but a hacker can do a sweet talk. They can influence this

[1:35:43] person. They can say, "Hey, I'm the relative of this person who is in the bungalow and they have given me this authority." You can threaten them. You You can somehow get into this. You can break this jail somehow, okay? Here is

[1:35:56] the example of how you can set up guardrail in LangChain. We have a finance agent which will get you customer information. Let's say you have these three customers and here is their information including their credit card.

[1:36:09] So, this get customer info tool is specified as one of the tools in your agent. And the way you set up guardrail is by providing this middleware. So, you you're saying, "If you see a credit

[1:36:23] card, mask it. If you see email, redact it and so on." So, now what happens is when you say that, "Okay, this is the question. Give me information about customer Krishna." It will give, but it will respect these guardrails. So, if it

[1:36:39] credit card is masked. If it is email, it will redact it. See, this is how you set up guardrails in LangChain. It is now the time to build our first project

[1:36:51] shopping agent. This will be similar to Amazon's Rufus AI agent where I use generally this agent to search for products and also to place an order. See, once I find a product I can say add the first one in the cart

[1:37:09] checkout and it can place the order, okay? So, here is the preview. It's a Streamlit-based UI where you can say, "Find me organic honey with four four plus rating." It will find it from the product database

[1:37:24] and you can say, "Okay, order item number two, three, etc." and it will place an order. And and note that see, I said organics. And and note that see, I said organics. It tolerates those kind of errors too.

[1:37:36] let's say you want to upload a product of oats and you are saying, "Okay, find similar products." So, it will use multimodal AI to search for products, see? And then you can place the order. Not only that,

[1:37:53] it will handle out of scope questions, invalid question, for example, I I want to buy elephant. Uh let's see how it handles it. See? I couldn't find products related to elephant. Okay, so this is the project

[1:38:07] that we're building. If you say honey, it will find the honey products and so on. So, these are the uh big features. You are able to search by keyword, price. You are able to get ratings of the products. You're able to place the

[1:38:21] order and search by images using multi-model AI. I have created this directory where I'm going to keep all my project code. So, I'm going to give you this uh SQLite database. And you can install SQLite extension

[1:38:36] here, by the way. Okay, just say SQLite viewer. So, you can install SQLite viewer, and that way you'll be able to see the

[1:38:48] We have three tables. The first one is products. So, these are the products. So, you have organic honey. You have coconut oil and so on. So, this is the product name, product ID, then

[1:39:02] the category, then you have price and description, and is organic. Okay, very simple uh relational uh table. Then you have uh reviews. So, these are the reviews from different people. Here

[1:39:15] is the product ID. On product ID, you have total four reviews. For example, reviews for product ID one, for product ID one, which is your organic raw honey.

[1:39:29] And these are given by Alice, Bob, etc. You know, some comments, and these are the ratings. So, whenever you want to fetch product rating, you will search in this table using product ID. You will say select star from reviews where

[1:39:43] product ID is equal to this ID and you get all the reviews and then you average it up. So, if you want to get average product rating for product ID one, you will have to do average of these four individual ratings. And the orders table

[1:39:58] is when you place the order. So, when you place the order, this will be order ID, product ID, product uh name, price, and order at. We don't have customer ID etc. We're keeping it little simple.

[1:40:12] if you want to work in a production-style environment, there'll be some extra columns, okay? We want to keep things simple keep things simple so that you have good time learning it.

[1:40:25] Now, when you're building agent, the agent will have to look into all the look into reviews table and get the average of this product ratings. So, why don't we create a Python file to fetch the reviews data

[1:40:40] a Python file to fetch the reviews data first. I will call it reviews .py. There is nothing agentic here. This is usual uh Python programming. So, I'm here. So, see,

[1:40:56] as a first step, we are getting the path of the database. SQLite is sort of like a local file, you know, local file which is stored on in your local file system. is stored on in your local file system. And then, to get the product rating, you

[1:41:09] write this kind of function which takes the product ID and it returns the rating. Now, see, if you're using live coding nowadays with AI, it will auto complete. We are not going to spend time writing this ourselves.

[1:41:25] We have used AI to write this code. And you can check. See, this query select average rating count star from reviews where product ID is whatever is the ID. Okay? So, for this product, it will fetch this four

[1:41:38] reviews and it will do average of these four numbers. Okay? Pretty simple and pretty straightforward stuff. And then, you might want to get the average rating for list of product ID. See here I'm passing one ID, but what if somebody

[1:41:52] wants to pass list of IDs? So, we'll write function for that as So, this is the function. See, you're getting list of IDs.

[1:42:04] And just look into this code. You know, you are just getting the average rating uh for all those IDs. Okay. uh for all those IDs. Okay. Now, let's write a main function

[1:42:19] do. I write a code and write a main function to test So, let me just comment this. So, for product ID one, which is this,

[1:42:31] I want to get the average rating. Okay? So, average rating for this it Okay? So, average rating for this it will be 5 + 4 + 5 + 4.5

[1:42:45] 4.5 / 4. 4.625 is the average rating. Okay? >> [snorts] >> run this code here. You can go to Make sure your virtual

[1:42:59] important. And 4.62 with four four reviews. You can give another

[1:43:12] product ID. So, let's say this. Okay? You have three reviews. So, this. Okay? You have three reviews. So, product ID two. Okay, three reviews. You can do the math. It's working. Okay.

[1:43:26] So, uh when you're doing modular programming, this is a good approach. You can uh have a separate file for reviews. And then you can have batch ratings as well.

[1:43:40] So, let's try that here. here. See, batch ratings for product 135. the individual product ratings. So, this is working as

[1:43:54] expected. I'm going to close this. Now, let's create another file our main file called shopping shopping agent. So, shopping agent.py.

[1:44:06] In this, I have imported bunch of necessary modules. And let's create our ChatGroq. I'm using Gwen from ChatGroq. You can also use Gemini. Now, how about we write our first function, which is search products. You can create a

[1:44:21] separate Python file for this as well, okay? Look at this AI. It's so eager to write the code for me. The function will take query as a string, okay? The organic honey, oatmeal, whatever. Organic honey, oats, etc. Then, if you

[1:44:36] have max price, you know, sometimes you type query organic oats under $15. So, that is your max price. If you say organic, then is organic will be set to true, okay? So, this is the

[1:44:49] function signature. You need to then add a very good docstring. See, docstring is super important when you're building agents. It will guide your LLM. It will tell, okay, this is the right function to call, and what kind of parameter this

[1:45:03] function is expecting, so LLM can extract right parameters from a given very descriptive uh docstring. And this is the docstring you will come up with after some iterations.

[1:45:17] So, you may not get it right first time. You might have to do some iterations. You might have to do some iterations. But let's say, this is my docstring. And now, I want to connect to my SQL database. So, here I'm connecting with

[1:45:31] database. So, here I'm connecting with SQL light and DB path, you see, this is the store.db. Correct? See, the file is locally in the same directory, and I'm saying that SQL is select ID, name, category uh from

[1:45:45] is select ID, name, category uh from products where one equal to one. Now, we're going to add the additional queries later on. So, let's say if a person has supplied the query, you will say if query, then you will say where

[1:46:00] one equal to one. And where one equal one equal to one is like no-op operation. But, this is useful because we're going to do and. Let's say if you don't have a query and if you don't have any and,

[1:46:12] then still it's a valid query. If you don't have where one equal to one, then you might have this issue you might have to do something with this and operator and so on. Here, in the query, what you're saying is if SEO is supplying

[1:46:26] organic honey, then it will say name like this or description like or category like this. See, if somebody says name honey, then you might want to search in category, name, or description. It can be anywhere. So, you

[1:46:39] want to search into three columns these three columns, and you want to find substring match. If you find you know, a substring match within that, then find it. If you don't want to do keyword search and use LLM again, you

[1:46:54] can you can do that too. But, here we're going to use substring. And then, you will have and condition for max price and organic. Is organic is very simple. Okay, if person has supplied is organic, then you will say and is organic equal

[1:47:11] to that parameter. Same thing for max price. And then, you will run your price. And then, you will run your query. So, I'm going to run this query. And the rows will have my results, okay? And

[1:47:27] then you will put those rows into a dictionary. So, you will say for row in rows, you're using list comprehension here. You will using list comprehension here. You will put ID, name, etc. You are creating list

[1:47:39] of dictionaries here, okay? So, JSON, let me import JSON here. Import JSON. Okay, perfect. Now, let's call this function. So, here it is creating the agent, but I don't want to

[1:47:53] create the agent right now. I want to just search for it. Let's say you are searching for honey, okay? searching for honey, okay? So, let's search for honey here.

[1:48:06] So, let's search for honey here. And you will say shopping agent. It should make a database query. Okay, it is a tool, that's why it's not callable. Let me just comment it out. Maybe that's the reason

[1:48:19] I don't know. But yeah, tool, it should not be a tool. Okay, max price is set to one, that's why it's returning this why it's returning this this one. Maybe I can This is optional,

[1:48:31] right? So, see, both max price and is organic is is optional. So, let me remove this. And this is one disadvantage of AI autocomplete. It will just type

[1:48:45] anything. See, now I got I know it's hard to view, but you you can read it, right? ID one organic raw honey, whatever. So, this search function is working. And then for get rating, you need to

[1:49:00] So, get rating for get rating, we already created a function in the correct? So, I am just calling this get product

[1:49:13] rating function here for a given product ID. for a given product ID. And this one also let me make it a tool. Very good. We have two good tools now. One is getting ratings for a given

[1:49:26] One is getting ratings for a given product ID. The other one is getting the products, you know, it's searching products using the query. What is the next tool that we need? Well, check out. Once you have found the product, you

[1:49:41] And for checkout, folks, what we'll do is if let's say person says I want to place an order for product ID one, we'll take that product and we'll simply take that product and we'll simply insert a record into this table.

[1:49:56] We're not going to do payment processing and all that. Okay, it's a learning project, so it's not feasible to do all of that here. So, we'll insert a record which will have product ID, product name, price, and the current timestamp.

[1:50:09] is pretty straightforward. So, let me add a function for that. See, you're saying checkout product ID this, and all we have done here is

[1:50:22] connect to SQLite, get the information about products from products table, and then insert a record in a orders table. Very straightforward logic. Now, let's create our agent. See, this is the main part. Here you have

[1:50:37] three tools, search products, get rating, and checkout. Your LLM is whatever LLM that you created. And I think I need a system prompt. Now, system prompt is going to be a bit lengthy. See, I have this system prompt.

[1:50:54] So, this is how my system prompt looks. I have defined it through some iterations. You will not get it right in the first attempt. You'll have to use But here it says you are a helpful shopping assistant. Follow these rules.

[1:51:08] matching items. As if you're giving instructions to any human. For each candidate, call get rating to user's minimum rating.

[1:51:21] Present qualified products as a number list. Now, I was getting some issues with coloring and all that, so I have added all these instructions. These are These are all the like UI issues on Streamlit, okay?

[1:51:34] Um and as I said, you'll not get it in the first attempt. You have to do some iterations. Okay, so there are two steps. Browsing is your browsing. when you are placing the order. You do also give

[1:51:47] Otherwise, this agent is sometimes it tries to become you don't want. So, you have to explicitly say that never place an order

[1:51:59] unless user explicitly confirms. Never guess a product ID. All right, so let's guess a product ID. All right, so let's now test this agent for a simple query. I want to buy organic honey with 4.5 plus rating and less than $20 price,

[1:52:14] So, let's uh I think we can simply run it by doing this.

[1:52:27] it gave this output. one. I mean, I have a just one message, so I don't have a loop.

[1:52:39] Uh but anyway, so you can run it uh and you can run it for different use cases. It is good to run it for uh the cases uh like the corner cases. For example, I want to buy a tiger. Okay, let's

[1:52:55] By the way, I have not tested it. Let's see if this works. Uh I want to buy a tiger. I couldn't find any product, see? So, this way you should um

[1:53:09] um verify it. You can also handle out of scope questions. So, for example, if I ask it to write an essay on Indian's freedom history, it should not handle it ideally. So, in

[1:53:21] the system prompt, you can say that okay, do not entertain out of scope questions and so on. Now that we have this running in the command this. I used by coding and created this very

[1:53:35] basic app, which I am going to show you. So, here let's create a new file. Okay, so we are going to create a new file. Usually it's called app.py. Here you're

[1:53:49] importing Streamlit as st and this is the title of your app. And then um all you're doing is you have an agent. So, this particular agent uh you are importing from shopping agent.

[1:54:04] And you're just passing the input message. Now you'll notice that here it is called st, which is streamlit.session_state.messages.

[1:54:20] variables. So, you can say st. session_state. If the that doesn't exist, then you are storing the messages in this array or list. So, what will happen is you will store the entire history.

[1:54:35] Um you are asking question 1 2 3, you'll keep on appending these questions into messages. See, here chat input, whenever you get the message, you are appending. See, see, you you created this uh

[1:54:50] variable. It's like a Python variable. You created a list in Python and in that you are appending whatever prompt you're getting. So, this way it is giving you memory. We are not

[1:55:02] um memory check pointer. We are using a valid way. Sometimes using memory check pointer is better, but in our case, we are just simply using

[1:55:15] state variable and appending all the list of messages. And whenever you are calling LLM, you are passing the old messages as well. So, you're passing the whole conversation chat history to LLM. So,

[1:55:29] that way it has the context. Let me go ahead and run this. So, here what you will do is you will say Streamlit run. And by the way, many dot here? It means the file is not saved. So, control S. I have saved it.

[1:55:45] saved. So, control S. I have saved it. Streamlit run app.py. That's how you run the application. Perfect. Look at this. I have my shopping assistant ready. I have my shopping assistant ready. I can say I need organic

[1:56:00] uh less than $20 4 plus rating. I don't know why I keep on typing organics honey. It's just X and C are close to each other. But you realize

[1:56:15] close to each other. But you realize even with incorrect uh English it is showing you the uh the products. You can do quick verification in your database. So, if you look at the products

[1:56:27] and if you look at honey, so let me just filter honey here. And organic should be one. So, see there are four products in total.

[1:56:40] four products in total. I need one which is less than $20. So, this one is gone, right? So, I should see organic raw honey ID one, organic buckwheat honey five, organic acacia honey seven. You see, organic raw

[1:56:53] honey, buckwheat honey and acacia honey. And you can verify ratings as well. Now, I will just say uh I will just say uh order number three. Aha, see? Organic

[1:57:06] Acadia honey has been successfully ordered for this much price. Now, when I go to orders table and hit refresh, see? My order is here. Look at this. You just

[1:57:19] My order is here. Look at this. You just created end-to-end agentic workflow where you are searching for uh products, locating the right product, and placing the order. Let us now add the support for images.

[1:57:32] In the UI on the left-hand side, you want to provide a button where a person can upload an image, and you should be able to search able to search using that image. Here, we need to add a

[1:57:44] new tool for our image support. And we are going to define the Python function for this tool. But before I do that, for images, we have to use a different model. Okay? We have to use a vision model from Grok,

[1:57:58] which I will define here. See, I'm using Llama 4 Scout, whatever we used previously. So, let us create a function called describe product image.

[1:58:10] So, here, once again, AI is autocompleting it. Uh but what I want is I would like to read the image and convert it into base64

[1:58:22] encoding. So, let me import that model here. Import base64.

[1:58:34] It's a way to convert image into a string. Okay? And then, string. Okay? And then, uh let's do couple of more things. So, here, you are handling the extension, whether

[1:58:48] you are handling the extension, whether it's JPEG, JPG, or JPEG. Okay? And then uh you will create a message or a prompt. The prompt looks something like this.

[1:59:01] Where you have image URL. So, here essentially just think about this. essentially just think about this. This is your image data. In uh base 64 uh format. And the second uh message in the content is text, look

[1:59:15] at this product image, extract its key attributes. Return a JSON object with these these four fields. You know, I want to return a JSON object with these want to return a JSON object with these four fields, and then you will say

[1:59:29] four fields, and then you will say something like this, okay? So, this is what I want to do. So, let me first test this particular function. So, this is what usually I do. I test it via main

[1:59:44] So, here describe product image. And luckily, we have some images in resources. So, if you look at resources, you will have resources. Now, folks, Python has

[2:00:00] a relative path conventions. So, that for that that reason I will just use the OS module and create the image path like this.

[2:00:15] And I will use it, okay? So, let's uh call that. And this is some warning you're getting, okay? I think we are not printing it. That is I think we are not printing it. That is the problem. So, response

[2:00:29] worked. It's just that um response. It's just that this is a generally a good habit. It's like a uh uh or a reliable way to to pass the file path,

[2:00:46] okay? So, see product type oats, search query oats, is organic null, a bowl of rolled oats, popular breakfast. Okay. This tool, we can say it is working okay. And now, I'm going to annotate that

[2:01:04] as a tool. So, now this is a tool that is available Here you need to This is important, okay? If you don't pass it here, it's

[2:01:17] not going to work. And you need to also modify the system prompt. So, previously modify the system prompt. So, previously we are doing browsing and ordering. Now, we are doing three things. So, let me just put the new prompt here.

[2:01:32] image search, perform these three steps. If it is text search, perform these six steps. If person is ordering, then perform these three steps. Once again, I use the wipe coding to create UI controls for the

[2:01:48] image. Here, you'll be uploading an image using this file uploader control. Once the file is uploaded, you can st.image so that the image is visible in the UI.

[2:02:03] when uh somebody clicks on that upload button, I think Okay, when somebody clicks on find similar products button, what it will do is it will create this

[2:02:17] file, you know, temporary file because you are uploading file from anywhere. So, it will create a temporary file on your computer, which will have this path. And that path, you

[2:02:30] set here somewhere, right? So, this is image path. So, let's see. Image path you're printing here. Okay, image path you're printing here. And then, here you are saying uploaded file dot name. So, you're setting it uh

[2:02:46] it as a pending image. And then, you're doing further processing on pending image here. So, what you will do is um you know, you you just managing your

[2:03:00] session essentially. You're deleting it once the image is processed. Otherwise, you're just saying, "Okay, session state dot messages." So, when agent gets it, what agent will get is the file path. See, agent has this uh tool, okay? And

[2:03:16] if you look at this tool, uh this tool will let's see. This tool will get this image path. So, agent will get that image path

[2:03:28] path. So, agent will get that image path based on this unprocessed image variable that you have set, okay? And it will further process it. All right, folks. So, this is looking good. I am itching to run this because I'm

[2:03:42] super excited for this particular image use case. It's Amazon has it. So, this is very unique, you know, you have an image and using image you want your agent to search for different

[2:03:57] products. So, let's search for oats. you're saying find similar products. Now, it made a call to your agent.

[2:04:12] And it found these two products. You can say that second product which because I like that better. And it has placed the order. You can confirm in the

[2:04:25] SQLite database. Now, if you upload some picture for which they don't have products, then let's see what happens. I initially got an an because there was some problem saving the image. So, it is creating

[2:04:38] this temporary image and and saving it. So, I tried again and it uh returned the correct response. There are no products in store that matches the image elephant. Basically, there is no items

[2:04:52] related to elephant. All right, folks. So, that's it. See, you just created a complete shopping agent. And the features that we covered were First, we features that we covered were First, we uh had multiple tools in our agent. We

[2:05:07] had database interaction. We had multimodal use case because we used the images. We built UI in Streamlit. Now, using Wave coding, you can also build UI in um React Native and make it super

[2:05:22] professional. That's pretty easy. And we also had a memory. So, what I want you guys to do is add this project to your project is add this project to your project portfolio and maybe you can uh work on

[2:05:35] adding some new features. These are the feature additions I want you to add. It is an exercise, okay? So, if you have learned something good out of this project, folks, consider this as an exercise. I'm going to provide uh

[2:05:50] everything in the video description. So, check So, check it out. And please implement all these. It will make your project complete. And then try to deploy it in production. And also write a nice LinkedIn post explaining your project.

[2:06:07] You can tag me, Code Basics. We'll engage in your project. It will help you build online credibility. Projects are super important when it comes to AI interviews because we have seen that during the interviews, interviewers

[2:06:21] mostly spend time on the projects that you have built. And in our past cohort, we had some students who got a job just by building excellent quality project. If you want to learn AI engineering in-depth by covering all the

[2:06:36] technical topics along with soft skills and also build eight plus production grade projects with deployment, then please check out our AI engineering cohort. We have live sessions on Zoom on

[2:06:51] weekends and we teach everything end-to-end. We help students prepare for interviews. We have uh soft skills modules and it's a comprehensive program. [music] If you're interested, please check it out. Link in video

[2:07:04] description. Let's work on our second project now. The second project is telecom rag. So, this is not an agentic system, but many industrial projects are built using rag using the similar architecture. So, here for a telecom

[2:07:19] company, you're building customer care assistant which can answer people's query like how do I set up my auto pay, uh why is my internet slow, my uh call

[2:07:31] keeps on dropping and it should ground the responses in a source of knowledge. So, our source of knowledge is going to be three main sources. The first one is

[2:07:43] be three main sources. The first one is we have the customer tickets database. This is a SQLite database. We have all our past tickets. So, say in the past, some customer said, "Okay, no internet access. Uh this was the issue."

[2:07:59] Then there was a description, then there was a resolution. So, now using the same resolution, you can solve the new complaints as well, correct? So, this becomes a very, very important source for us. There is a status as well.

[2:08:12] for us. There is a status as well. Then the second source is CSV file containing all the FAQs. Third source is a PDF document which contains the technical reference for this telecom business. Now, we have

[2:08:28] worked on a rag code using the same PDF file. So, now we are going to extend and other than PDF, we have included two more sources, the we have included two more sources, the CSV file for FAQs and the tickets

[2:08:42] database. So, what what it will do essentially is whenever you ask questions, LLM will look into three sources, okay? PDF, database, CSV. This is a pretty good diverse project. In the real life also,

[2:08:58] you will have variety of sources, okay? So, we are covering three different category of sources. The first step in a rag process will be indexing. So, first you will index the PDF file. Now, for PDF file, you will do

[2:09:14] chunking. We already did chunking, if you remember. Uh we do chunking. Then, for each of these chunks, let's say you divide this PDF into however many chunks. Then, for each of these

[2:09:27] chunks, you will create embedding. Okay, so I create embedding. And these embeddings you insert into a vector database. We have looked at this code in this in this video. So, it it will be the same code.

[2:09:40] And then for tickets, you don't need to do chunking because uh for tickets, each record can be one chunk. It's a small enough size, so we don't need to do chunking, right? So, for tickets, let's say ticket one is one

[2:09:52] chunk, ticket two is one chunk. So, for ticket one, you generate embedding. Ticket two, you generate embedding. And you store all of that into this vector you store all of that into this vector database. Same thing for FAQ. One FAQ is

[2:10:06] one chunk, okay? So, let's say FAQ one, FAQ two, you use sentence transformers to generate embedding and then you store this thing into a vector database. After indexing, second step is retrieval.

[2:10:21] We have seen this diagram, nothing new here. Whenever user is asking a question, you will use same embedding method, which is a sentence transformer uh in our case, to generate embedding, you will do semantic uh search in vector

[2:10:34] database, find all the relevant chunks, form a question, ask LLM to generate the answer. I have created this folder called 11 project telecom, where I have these three data sources. They are in the data folder.

[2:10:49] the data folder. Here, I'm going to now create a new file called ingest PDF. Or let's say ingest FAQ.py.

[2:11:02] Because as per our diagram, the first step is indexing, second step is retrieval. In the indexing, we need to index three these three data sources. So, we'll ingest FAQ, tickets, and uh PDF. So, let's first

[2:11:17] uh think about the uh FAQs. So, let me import some again, we are using sentence transformers transformers uh all-MiniLM for our embedding, okay?

[2:11:32] uh all-MiniLM for our embedding, okay? And to ingest FAQ, we will take the CSV path. And then for each of the rows, we will simply insert those rows. First of all, we'll create the

[2:11:45] list of documents because Chroma DB will expect it in this format where you have a document. So, document, if you look at it, it is a LangChain core document, and it has metadata and page content. And it is returning this doc string. To ingest

[2:12:00] load those documents by calling that function. And then Chroma.from_documents, see? So, this is how you ingest the documents inside Chroma, and you get this vector store. Now, here you are passing the

[2:12:15] embedding, okay? So, let's call this function. And now, from terminal,

[2:12:27] what I can do is go to project 11. And look at uh the directory. See, it has data.

[2:12:39] And then, let me see. Where did it create ingest FAQ? Oh, it created inside data folder. I need to move it up.

[2:12:54] this, and ingest FAQ is this. So, let me close this. We need to call ingest FAQ. See, 25

[2:13:08] FAQ entries loaded. And this looks like it ingested it into ChromaDB. And that is when you get this folder, Chroma store. See, it's a SQLite, so you can see in the SQLite the collections, etc.

[2:13:24] See, the collections that you have is FAQ. Databases, you can explore that. I mean, some of these things might be encoded, so you might not be able to make sense out of it.

[2:13:36] But, based on this message, we can say that FAQs are ingested into our vector database, okay? So, this portion is done. Now, let's ingest tickets. So, here, I will call

[2:13:54] ingest tickets. dot py. And folks, just to save time, I'm copy-pasting the code. We live in the era of live coding. No one writes everything by hand. As long as you

[2:14:07] understand the structure, architecture, the working principles and fundamentals, the working principles and fundamentals, you should be good. All right. Now, here I will just say load ticket documents.

[2:14:20] So, it's the same thing. Basically, from SQLite database, you want to create docs. And each of the docs is what? It's a document object from LangChain core, page content and metadata, okay?

[2:14:33] tickets. So, if you look at this database, see tickets, you have all these tickets. You're doing select star, You're doing select star, and then you are creating this docs

[2:14:47] list, which which it will return the list here, and then in the main function, similar to how we did it in FAQ, we'll simply say chroma.from_documents and insert those documents. And you're

[2:15:01] passing embeddings here, see? So, let's now call this from main method, and let's run ingest tickets. Perfect.

[2:15:15] 19 vectors stored. So, this ingestion is finished. Now, ingest pdf.py. Now, folks, ingest pdf.py is is

[2:15:27] something you have already seen. Uh remember in in a rag uh portion, we have already done this. So, we used the recursive character text splitter to recursive character text splitter to split the PDF into small chunks.

[2:15:42] The chunk size is 600 with an overlap of 100 characters. 100 characters. Then we are inserting that into chroma once again. So, let's run this as well, ingest PDF.

[2:15:58] So, let's run this as well, ingest PDF. Amazing. 37 vectors stored. Our vector database is now ready. Now, we are going to work on to work on creating the rag chain. So here I will

[2:16:11] creating the rag chain. So here I will create rag chain. pi and the way rag chain works is I think first we need to build a retriever. So let me build a retriever. retriever.

[2:16:25] pi. So retriever is nothing but it retrieves the relevant chunks from a So we that retriever looks something like this. So here See you have to remember you have to use

[2:16:39] transformer. If you use a different one then it will not work. In the retriever you want to tell your FAQ tickets guide you know what is the K. How many chunks of FAQ you want to return. How many

[2:16:54] chart tickets you want to get back. So we have three different collection by the way. I forgot to mention. So we have a chroma database in that we have three collections. These collections are sort of like tables.

[2:17:07] So imagine we have three tables. And when we do a retrieval we want to get the top three matching records from each of these three tables as retriever K. K is top K basically how many records you want to retrieve. And when I call a

[2:17:22] retrieve function it will call that on all these three retrievers. Okay. So you will get the relevant chunks from guides. And then in rag chain you are importing

[2:17:38] Okay. And here is my system prompt your usual uh when customer asks you a questions FAQ entries past support tickets return the context from that and then

[2:17:53] in the chain you are this is just a chat prompt template which has system and human message. And then this is the chat grok model, nothing new. And in the chain, you have prompt LLM

[2:18:06] and string parser output. This is a syntax that everybody follows. It's like a pipe. So, you go step by step, you know, like you process one, two, three, and so on. So, these are the stages, okay? One,

[2:18:18] two, three. So, this is first, second, third, and fourth. So, you have prompt, prompt is given to LLM, and then you get the output. Output is processed by this class,

[2:18:32] processed by this class, STR output parser in string.py Now, let me create a file called main.py, main.py. Before I try Streamlit, I want to simply try this out from a

[2:18:47] command prompt. You can write a simple while loop like this, questions. Okay? So, let's do this. In this folder, I'll simply call main.py.

[2:19:03] So, here it will take the question. It's creating the embeddings. Okay? Now, what is the question you want to ask? Let me ask some question based on this PDF. Right, so I'll say, "How do I avoid

[2:19:16] SIM swap fraud?" How do I avoid swap fraud? See, it

[2:19:28] says that to help, follow these steps, enable SIM lock. So, section five, So, section five, yeah, multi-factor remote verification.

[2:19:41] Okay, so multi-factor remote verification. Does it say any there? Let me see if we have any data in the if we have any data in the FAQ or the tickets. Actually, SIM swap

[2:19:55] see. SIM swap attacks occur whatever when first a convinced carrier victim numbers to new SIM. So, all right, so you can

[2:20:07] look through it. There are multiple sections for this. sections for this. You can ask a next question or you can just say exit to exit out of it. Let me ask some question from FAQ actually. So,

[2:20:22] how do I check my current balance? Right? So, let's check. current balance? Right? So, let's check. I want to know my current balance. To check your current balance, dial 1 2 3. So, see, dial 1 2 3. You see this?

[2:20:38] Dial 1 2 3 from your phone, My Telkom app, or log to My Telkom app. Perfect. It is working okay. And then you can say exit to exit. So, our chatbot is kind of ready. We just need

[2:20:53] chatbot is kind of ready. We just need to put a nice cover on top of it. We give it some clothes, and that is your Streamlit UI. So, you will create Streamlit UI. So, you will create app.py, and there

[2:21:07] you can I'm just once again going to copy paste here. copy paste here. Um now here also I think we are adding a memory, see? You're appending the messages to the

[2:21:19] just if you look at it, list. And we are adding a memory that way. So, let's uh run this Streamlit run app.py, and we'll

[2:21:33] put some sample questions here. See? These are the sample questions. These are the sample questions. So, why is my mobile internet so slow? respond. So, I'll leave it up to you to verify all of this. Uh but uh this rag

[2:21:49] project is ready. Sorry, I went a little bit fast, because we have covered some of the fundamentals and in the era of wipe coding, I don't need to explain you each and every line as long as you understand the concept,

[2:22:02] the architecture, the underlying fundamentals, as long as you are able to debug whenever there is a problem, you should be good. You don't need to go too much in detail. Look, we just built an amazing application which uses three

[2:22:15] different sources of data. And here is the most important part which is exercise for project two. You will work on these three exercises. I'm giving down in video description below. Check it and make sure you are finishing these

[2:22:30] exercises. You are adding this project to your project portfolio and also make a nice LinkedIn post. For example, look at this particular project's post which

[2:22:42] Srivignesh made and he created a presentation, you know, he created a presentation, nice video. He has his face in it. So, create this kind of LinkedIn post to boost your online credibility. All right, we are done with

[2:22:56] you one thing. So far in this course, we have used coding frameworks such as LangChain to build our agents. You can also use build our agents. You can also use no-code frameworks such as Zapier and

[2:23:11] no-code frameworks such as Zapier and n8n to build your agents using simple UI videos on our channel. If you want to build no-code agents. To continue your learning journey, you can now learn LangGraph which will help you build

[2:23:25] super customized agent with a very detailed control. And we have a crash course for LangGraph on our channel. I'm going to provide a link in the video description. That's it. Congratulations on finishing this crash course. If you

[2:23:39] have any questions, post in the comment box below. I will try to answer as many questions as possible. Thank you very much for watching. >> [music]