RAG Full Course: Build a RAG App from Scratch — Step-by-Step Guide & Transcript

[00:07] using your company documents, internal knowledge base, or the latest information without relying only on what it learned during training? This is exactly what retrieval augmented generation or rag makes possible. Hello

[00:20] everyone and welcome to this course on retrieval augmented generation. Large language models are incredibly powerful, but they have important limitations. They may not know your private data, they may not have access to the latest

[00:32] information, and sometimes they can generate confident answers that are simply incorrect. So rag helps solve these problems by connecting large language models with external knowledge sources, allowing the system to retrieve

[00:45] the relevant information first and then use that context to generate more accurate, grounded, and reliable responses. So in this course, you will from end to end. We will begin with the fundamentals of large language models,

[00:58] hallucinations, embeddings, and the core architecture behind rag. Then you will learn how documents are prepared, chunked, converted into embeddings, and stored inside vector databases. We will explore retrieval techniques such as

[01:11] semantic search, metadata filtering, hybrid search, and reranking and of retrieved context. You will learn how prompt engineering is retrieved information and reduce hallucinations. Finally, we will bring

[01:26] everything together by building complete rag application using frameworks such as LangChain, followed by a practical demo where you will test, evaluate, and performance. So by the end of this course, you will

[01:38] understand how rag works and how to build knowledge-based AI assistant that can work with real-world data and deliver more trustworthy answers. Also, if you want to take your generative AI skills beyond LLM fundamentals, you can

[01:51] Generative AI and Generative AI specialization delivered in collaboration with Microsoft. This program covers generative AI, drag, multimodal AI, agentic AI, autonomous agents, multi-agent systems, MCP,

[02:04] deployment, monitoring, and responsible AI. You will also work with tools and platforms such as OpenAI, Azure AI, GitHub Copilot, LangGraph, and N8N capstone. The program is designed to help you move from simply understanding

[02:18] AI concepts to building and deploying production-ready enterprise AI solutions. You also earn a completion certificate from Microsoft and Simplilearn. So, if you want to build practical GenAI and agentic AI systems

[02:30] to check out this program through the link in the description box below. link in the description box below. So, now let's get started. >> Hello everyone and welcome to this course where we will explore the

[02:43] fundamentals and practical applications of retrieval augmented generation. So, mind, to give you a complete practical understanding of how RAG systems work from the fundamentals of large language models and embeddings to building of

[02:57] fully functional knowledge-based AI assistant. So, large language models with AI, but they still have limitations. They may not have access to the latest updates and sometimes they can generate answers that sound correct

[03:10] but are not accurate. So, RAG helps solve this challenge by connecting AI allowing them to retrieve relevant information and generate more reliable context-aware responses. So, understanding RAG is not just about

[03:23] about understanding how modern AI applications are built, systems that can work with real-world data, reduce hallucinations, and deliver accurate answers based on trusted information. So, by the end of this course, you will

[03:36] have the knowledge and hands-on experience to build your own RAG experience to build your own RAG application from scratch. cover the foundations of RAG, that is how large language models work, why

[03:51] hallucinations happen, the limitations of standalone LLMs, and how retrieval augmented generation solves these challenges by connecting models with external knowledge sources. Lesson two covers RAG pipeline and data

[04:03] collected, cleaned, split into chunks, converted into embeddings, and stored in a vector database to make information searchable. Lesson three covers search works, how vector databases retrieve relevant context, and how

[04:17] techniques like metadata filtering, hybrid search, and reranking improve And in the next lesson, we will cover prompt engineering for RAG, how to design effective prompts that ground the LLM in retrieved context, reduce

[04:29] information, and generate reliable source-aware responses. Next lesson application, how to connect document retrieval, reranking, prompts, and large language models into end-to-end RAG system using frameworks like LangChain.

[04:44] And next, we will have practical demo, how to build and test a real-world RAG optimize retrieval performance, and production-ready AI applications. So, now let's get started.

[05:03] fixed data set up to a certain date. That training process is expensive and time-consuming. So, companies don't retrain their models every day or even every month, which means the moment you ask about something outside the training

[05:16] data, your company's internal hiring policy, a process document from last changed after the model's cut-off, it simply doesn't know. doesn't tell you it doesn't know. It gives you an answer anyway. That's the

[05:30] hallucination problem, and it's what makes plain LLMs risky for business use cases. So, how do you fix this? You have three broad options. First is fine-tuning. This means taking the base model and retraining it further on your

[05:42] own company data, so the knowledge gets baked directly into the model's It's powerful and it's very specific stable use cases it works well, but it's expensive. It requires machine learning expertise and infrastructure. It takes

[05:56] changes and honestly for most business use cases it's overkill. So let's say if a company updates its compensation branch every quarter, are you really going to fine-tune a whole new model every 3 months? That is not practical.

[06:09] And option two we have plain prompting, also called as stuffing. You could just prompt every time someone ask you a question. This actually works fine for small static use cases. Say a single one-page policy document, but it breaks

[06:22] down fast. LLMs have limited context window, a cabin how much text you can hundreds of page across dozens of document, you simply can't stuff all of that into every single prompt. It's also wasteful. You'd be sending irrelevant

[06:36] information along with relevant information, which costs more and can actually confuse the model. And option three is where RAG comes in. So instead of trying to cram everything into the model's memory or its prompt, you let

[06:48] the model look things in real time, on demand. The same way you'd search your company wiki before answering a question, instead of trying to memorize the entire Wikipedia. This is the spot for most real-world use cases and it's

[07:01] exactly what we are building today. So that's really the whole idea behind together. First is the retriever. This pulls out the most relevant pieces based on what the user actually asked.

[07:14] The generator. This is your LLM, which takes these relevant pieces plus the user's original question and generates a grounded, accurate answer instead of during the training process. If you put these together, you get a

[07:27] system that's always up-to-date with your latest documents and it doesn't accurate answers instead of just example for this course, that is Talentloop. It is a HR tech startup of

[07:40] Bangalore building AI tools for hiring. So, like most fast-growing startups, this internal knowledge is scattered. Some of it lives in Notion, some in personal notes. So, recruiters constantly ask questions like, "What's

[07:52] our notice period buyout policy?" Or what salary band applies to a senior SD in Pune? Or how many interview rounds do we have answering these questions is like asking HR lead on Slack and waiting. That

[08:07] multiple cities and dozens of roles a month. So, by the end of this course, we will have built an internal RAG assistant that answers these instantly, grounded in Talent Loop's actual HR documents, which is not made up of

[08:20] answers and without needing anyone to baby sit it. So, basically, RAG shows up everywhere once you start looking for it, not just in HR tools. A few common categories is internal knowledge assistants, like the

[08:32] one we're building today, where employees or recruiters query internal policy documents. Customer support bots. Companies use RAG to ground chatbot answers in their actual product documentation and support articles,

[08:44] instead of letting the bot improve. Document Q&A tools. Particularly common assistant that can answer questions about a specific contract, filing, or report and said exactly where it got the

[08:56] answer from. Research assistant. Tools that need access to current source questions involve information that's simply too recent for the model to know The common thread across all of this is you have a large body of that changes

[09:11] over time and you need answers that are accurate, current, and traceable back to accurate, current, and traceable back to a source. understand four building blocks: embeddings, vector databases, chunking,

[09:26] and similarity search. So, none of this needs a mathematical background. We'll First, let's look into embeddings. Computers don't understand words the way we do. They understand numbers. An embedding is a way of converting a piece

[09:39] of text into a list of numbers called a vector, which captures its meaning and not just its exact wording. Text with similar meaning ends up with similar numbers positioned close together in what's called a vector space. So, as you

[09:51] can see in this example, notice period and resignation timeline will land close together even though they don't share a single common word. Meanwhile, notice period and office parking policy will land far apart because they mean

[10:03] completely different things. Think of it like a map, but instead of representing physical distance, it represents semantic distance. How close This is what lets our retriever find relevant chunks even when the user's

[10:16] exact words don't match the document's exact words. A recruiter searching how much do we pay for a mid-level back-end role in Bangalore should still match a document that says compensation band software developer engineer in

[10:28] Bangalore, even though barely any words overlap. These embeddings are generated by an embedding model, a separate smaller model trained specifically for this conversation task. OpenAI, Cohere, and open-source options like sentence

[10:41] transformers all offer embedding models. For this course, we will actually use a free local sentence transformers model, so nobody following along needs to pay for anything. We will just use a free trial API model. So, once you've

[10:53] converted your documents into embeddings, you need somewhere to store searching through thousands of vectors one by one would be painfully slow. That's what a vector database is for. It's built specifically these number

[11:06] vectors and search through them fast using specialized indexing techniques. options. First one is Chroma. It is lightweight, open-source, and runs locally on your machine with zero setup cost. It's great

[11:20] projects. FAISS, it's built by Meta, extremely fast and efficient, good for large-scale similarity search. It doesn't come with a built-in persistence storage yourself. And next we have Pinecone. It's fully managed cloud

[11:34] hosted vector database. You don't manage any infrastructure. They scale automatically and they're generally what companies move to once they're running rag in production with real user traffic. For a beginner project like the

[11:46] to build it now, Chroma is perfect. Free, simple, it runs right on your machine and you can graduate to something like Pinecone later without changing much of your code. So next is chunking. You can't just dump an entire

[11:59] 40-page HR policy handbook into your vector database as one giant block. If you did, every single query would retrieve that entire 40-page document, specific question. You need to break documents into smaller pieces or chunks,

[12:14] each one small enough to be specific but large enough to still make sense on its own. So why does chunk size matter so much? If chunks are too large, you retrieve a lot of irrelevant text along with the useful bit, wasting context and

[12:26] chunks are too small, you lose a context. A sentence pulled in isolation might not make sense without the paragraph around it. Imagine chunking that splits "Employees are entitled to a notice period buyout" from the very next

[12:40] sentence which explains the calculation. Now you've got the half answer. So a common starting point is around 300 to 500 tokens per chunk, roughly 200 to 400 words with some overlap, say 50 tokens, between consecutive chunks. That

[12:54] overlap acts as a safety net so you don't accidentally cut an important sentence in half right at a chunk boundary and lose meaning on either strategies too. Fixed size chunking just splits every end character regardless of

[13:07] content while semantic chunking tries to split in natural boundaries like paragraph or section headers. For a beginner project, fixed size with a bit of overlap works well and is what most frameworks default to. Finally, we have

[13:19] similarity search. This is the step that ties embeddings and chunking together. So when a recruiter types a question, we convert their query into an embedding tool. Using that exact same embedding model, then we compare that query vector

[13:32] against every chunk vector already sitting in our database. The comparison method most commonly used is called cosine similarity. It measures the angle between two vectors, which tells us how close they are in meaning regardless of

[13:44] the length of the text. A cosine similarity close to one means very similar meaning. Close to zero is unrelated. We don't just want the single the top few closest matches, commonly called as top K, where K might be three,

[13:58] five, or more depending on your use case. This gives a generator a bit of than betting everything on one single have a recap. Your documents get embedded and chunks stored in a vector

[14:11] database. When a question comes in, it gets embedded too, compared against everything in the database using cosine similarity, and the top K most relevant chunks get pulled out and handed to the LLM. That's the entire theoretical

[14:23] foundation behind RAG. It sounds like a lot of moving parts, but as you'll see in the next lesson, implementing it takes surprisingly very little code. So, now let's go ahead and build it. >> [music]

[14:39] fundamentals and core concepts of how RAG works, let's just go ahead and quickly build one piece of RAG pipeline. So, we don't need any API building or any API keys here. We are just using the hugging face sentence transformers for

[14:52] embeddings, which runs locally on the machine. So, and we also use Groq for actual answer generation as well. And it is very fast and it is a free API tier, so we can use that. So, first before starting anything, we just have to

[15:05] install few packages. Uh we have to install the LangChain community text splitters and hugging face sentence transformers. So, let's just go ahead and build it, and this is the syntax for it. LangChain

[16:03] The syntax, you should have a hyphen here. system, I'm showing requirement already satisfied. If you don't have, uh you can enter this uh syntax, and it'll download the uh

[16:17] required packages for you. Now, since I already have it installed, let's quickly So, now it is uninstalled. This Yeah. So, now it's done. We'll just So, for you know, we're building this Talent Loop uh in our demo. So, I have three

[16:33] documents for this. That is, first one is the interview process guide, and next is the compensation bands by role and city as well. So, city I'm considering it as Bangalore. So, now I have three documents for the Talent Loop. That is,

[16:45] the interview process guide, compensation bands by the specific role, the hiring process for any startup company. whenever we ask a question, it should go through this particular document, and it

[16:59] should give the give us the answer accordingly. So, first I'm importing accordingly. So, first I'm importing OS.

[17:14] Loop docs.

[17:50] I'm opening the file and I'm writing the document. Talent loop

[18:06] three uh three documents that is So I've written documents for all the three of them. This is for the general hiring policy, notice period buyout, referral bonus, and background This is like the general policy for all the startups. And

[18:19] particular Talent Loop. That is Softenger has four rounds of interviews. Uh one is on four rounds for this particular role. That is online assessments, technical screening, system design, coding, and then hiring HR

[18:34] Uh as well as for the product manager role as well. And then I've written one for the compensation as well. Uh this particular role SD earns this much amount of uh salary. And you can also mention the city here if it is Pune or

[18:49] Mumbai, Bangalore, anything is fine. You can go with the role and city of the You can go with the role and city of the compensation.

[19:07] So if this is correct, it should give us this particular statement. Let's see. Yeah. So now the sample documents are created. Now, first step is loading this documents, we need to install a few

[19:19] packages from LangChain. I mean, we have to import a few packages. That is directory loader and text loader. So, this basically this will read the text this basically this will read the text file in our folder into the memory.

[19:57] code, which basically reads every text file in our folder into memory. And we are using text loader because we only have the information that I've written in plain text. So, this basically keeps things very lightweight, and it avoids

[20:11] taking in heavier dependencies, which are which is only meant for PDFs and Word documents. So, it's just the plain text. I'm loading the text loader here. So, the three documents that we have given has been loaded successfully. Now,

[20:25] know, we've talked about this in the previous lesson as well. We spoke about the 500 character chunks and 50 character overlap. So, this will keep the information very specific, and it's still meaningful as well. So, for

[20:40] this, we'll be using something called as recursive character text splitter. It is from LangChain, and it is used to divide larger documents into very small pieces of chunks, so that they can be efficiently processed by a particular

[20:54] LLM. And it can also be stored in a vector database for any type application. So, we'll be using that recursive character text splitter.

[21:24] >> And chunk size will be 500.

[22:18] next chunk to preserve the context between the sections. And the split documents here which I've written, it is it means that the function this function takes the original documents and breaks them down

[22:32] into smaller documents. And the length of chunks here, it returns the total length function or the responsibility of the length function is just giving us the length of the particular chunks created. So, this

[22:46] helps us improve the retrieval accuracy and it avoids sending large documents directly into the LLM. So, that is why we are doing this. So, it has created embedding and storing. So, first we have

[23:00] was chunking. And the third step is embedding and storing. We will uh take the LangChain hugging face, and then import the hugging face embeddings only.

[23:50] it converts every chunk into embedding using a small open-source model, which runs right into the machine, and it stores in a local Chroma database. And and also hugging face embeddings which I've used here, it loads the

[24:04] pre-trained model that is sentence transformers and all-MiniLM-L6-v2, this version, which converts each text chunk into a vector that captures its meaning as well. And this syntax, particular syntax here, that is Chroma from

[24:18] directory, talent loop, is basically takes the previous created chunks, and it generates the embeddings for them, and it stores them in a uh Chroma vector database, which is named as talent loop database. That is this directory, talent

[24:33] loop database. And so, this allows the RAG system to quickly find the uh relevant document section whenever a user asks a particular question. And this print statement, as you know, it confirms that

[24:47] the vector is stored vector has been stored successfully. So, let's just quickly run it. So, yeah, it has successfully uh loaded. Vector store is created. Now, the next step is retrieval. Third step was

[25:01] embeddings. Now, the retrieval step. So, the next step is retrieval. We'll just the next step is retrieval. We'll just go and retrieve the answers.

[25:27] which is already there in the document that I've provided.

[25:42] senior SD and then and then we will write the for loop here.

[25:59] this is working. So, I asked it what is the compensation band for this particular role and it has pulled us the information that is talent group compensation band. As you can see it has pulled two of them, hiring policy

[26:13] and compensation band. So, it has given us the I've only asked it for senior SD in Bangalore. But, it has given all the information which is not supposed to happen. So, basically I've written the same

[26:30] uh city here for the same role which is senior SD. I've written Bangalore and then senior SD again Bangalore. This has to be Pune or any other city, but it should not be same. So, whenever I retrieve whenever I asked it to retrieve

[26:44] the compensation band for this particular role, it has given us I went again and changed this particular role particular city for that compensation band. And now if I load again, it is giving us both the bands which is

[26:57] is already stored where the first when I loaded the Bangalore one is already stored and now the Pune one is also stored. It has given us two of the information two of the vectors separately and the hiring

[27:11] the vectors separately and the hiring policy for the particular role as well. So if you just from the starting if you just because it is stored already, it is showing us twice. If you from starting if you written the document correctly,

[27:25] don't write like me. Don't misplace or don't change the city or don't keep the don't change the city or don't keep the same city again. So just try to read the same city again. So just try to read the document again and make sure you

[27:39] put separate city like different city for each role. So now it So basically the result for this it should just load this one like it should show us the particular role. But since it is already stored, it is

[27:54] showing that showing us all of this information. So once you've written it correctly and loaded again from the starting, again it will show you the answer for this particular question that you've given. So now we'll move into

[28:08] generating the answer. We'll load the like we'll import the Grok Grok API key here and we'll import it from chat Grok and yeah, we'll import few other modules as well.

[28:43] So now you can go to Grok API and get the API key here. You can give some name. And

[29:01] here. So, I have imported the chart prompt template, enable pass through, uh string out parser, etc. And

[29:13] this is the chart chart group model that I'm using. And the API key I've given the particular API key. The format is docs and format docs is docs. And I've

[29:25] that you are an internal HR assistant for Talent Loop. Answer the question answer is not correct, say you don't have that information [clears throat] as well. Uh so, I've written the context and question here. Context is retriever

[29:39] and format docs. Question will be runnable pass through. Uh prompt LLM string out parser, and then it should give us the answer for this. Let's uh just do it in two steps. Copy the key and then

[29:55] executed. So, I have given it the key and again telling it to take the Grok API API key from this particular Grok API key function. Now, I can read on this again.

[30:22] So, now the compensation band for senior SD in Pune 20 28 to 38 LPA. Let's just see if it is correct. Yeah. Senior SD in Pune 28 to 38 LPA. It has given us the exact correct answer.

[30:45] from the particular this you don't have to take any API keys outside also you just have to take it from the hugging sentence transformers and just basic prompt you can give it and this is the syntax you can just get

[31:00] the answer here easily. That is how that works. the basics of how rag works like how to how the embeddings work chunking storing

[31:15] and retrieving and loading the document how does everything works let's just go ahead and try to improve the retrieval quality and execute it as well. So, what we have built is just a basic version but in real world it does not

[31:28] work like that retrieval does not always get it right in the first time it always fails in the first or second times if it is a huge information that we are So, let's just talk about how to make it better and we'll actually build as well.

[31:43] So, let's go ahead. So, the most common failure mode is irrelevant retrieval. So, the system will pull back the chunks that are like related but don't actually answer the question correctly.

[31:57] So, for example if a recruiter ask about interview rounds for a project manager role and it retrieves a chunk about engineering interviews and all the other interviews as well. So, let's just fix it with three techniques that is

[32:11] re-ranking hybrid search and metadata filtering. So, re-ranking is nothing but instead of trusting your first retrieval pass completely we will retrieve a slightly larger set of candidates and then we will run a second model over

[32:25] just these candidates to re-score that particular candidate and we will reorder them by the actual relevance. We will use cross encoder for this it's a small model basically it's specifically designed to score how relevant a

[32:37] document is to a query and it's more accurate than the embeddings alone as accurate than the embeddings alone as well. So, let's just quickly install it.

[32:56] showing it requirement already satisfied. Now, from sentence satisfied. Now, from sentence transformers

[33:10] This is the version that I'm using that I want to use. we're creating a document re-ranker for this particular rack system and we will

[33:24] use cross-encoder here because this cross-encoder model that is this macro Marco mini LM version, it it is used to compare a user query with each retrieve document and it it will assign a relevant score as well. So, this re-rank

[33:40] function which I'm using, it is to create the query document pairs. It will use the model to predict the scores and it will sort the documents from the most to least relevant as well. And it will also retrieve the top end documents.

[33:56] So, this also improve the search quality because most like instead of relying only on fast and approximate vector similarity, it also uses a deeper language model to choose the documents that best matches a user

[34:09] question. So, this is done. So, as I told you, it will return the document that is like it returns the top end documents as well. That this is a syntax for that. It will show us the scores. It will score it it will show us the score

[34:24] documents. And it like sorts the documents from most to least relevant and returns the end top end documents as well. So, let's just quickly update the retrieval to pull more

[34:36] candidates initially and then we will narrow them down with the reranker. So, narrow them down with the reranker. So, let's just take this query.

[34:50] candidates here. And we're telling it to retrieve the And we're telling it to retrieve the uh candidates which are top three. So, now this is done. We just have to test it.

[35:03] I'm just asking some random question. How many rounds are there in the project How many rounds are there in the project manager role?

[35:44] retrieval pipeline. It first retrieves a wider set of uh relevant documents using the vector database. Then it will re-rank them using the cross encoder to keep only the most relevant ones. So, this wide retriever which I've used

[35:57] here, it will basically uh it means that the vector store will initially fetch the top six uh similar documents for a query because the case K equal to six. You can uh rename it to 10 or five or any documents. I've given sample six

[36:13] documents. It will fetch the top six documents for a query, similar documents, and then this retrieve and uh re-rank function, it will take a query and then it'll get these six candidates documents using the invoke function. And

[36:26] then it will pass to the rerank function as well. Since you can see here, invoke this and then it will pass it to the rerank option as well. And then which scores and returns the

[36:40] top three most relevant documents. That is how this is all this all works. And in the query that I have given, I have asked how many rounds are there for And it will retrieve the related documents. It reranks them based on the

[36:55] relevance and prints the answers. So basically this will improve the accuracy by combining the fast vector search with more good reranking model. So as you can see here it has given us the interview process and the hiring

[37:10] policy as well. So since I have entered the other other role as well like Pune in software SD. So basically this is how the answer

[37:22] It has given us the exact answer but since I went and in gave a gave a duplicate entry like so senior SD in Pune and Bangalore I've changed it again. So it has stored twice duplicate entries were there. So it has given us

[37:38] the answer twice. So once you execute everything properly from step one then it will give you only the related answer. So now let's move ahead. The next step is the hybrid search one. So the embeddings are sometimes it will

[37:53] miss the exact matches like a specific role title or a policy term or any number. So with a simple keyword search we can actually catch it instantly. So we can search it. So now we can combine both hybrid hybrid search and

[38:08] the embedding as well.

[38:21] BM2. It's already installed. So, now I've written a code for the hybrid search. This is basically uh we are here, which will combine both the keyword-based search, that is BM25,

[38:36] tried to install here, but it's already installed. This BM25 and the semantic search, that is vector uh search as well, it will improve the document retrieval. Uh so, this BM25 retriever sends that

[38:50] a keyword search retriever that finds the exact documents based on exact word matches. And this BM25 retriever.k, that is keyword equal to three. It will limit

[39:02] to returning the top three results. And this vector retriever, it will use the embeddings to find the similar documents and also returns the top three results as well, since I've written k3. And the ensemble retriever, which I've

[39:15] mentioned here, it combines both the retrievers, giving the BM25 0.4 weight uh search weight and 0.6, that is the similarity has slightly more influence.

[39:27] will improve the rag performance because BM25 captures exact terms, while the vector search understands the meaning and context. So, let's just quickly run it. Now it's done, we just have to test it.

[39:41] results equal to hybrid retriever

[39:55] to hybrid. So, it will basically invoke this and then it will fetch the salary of the SD2 Pune. Uh based on the role and city. And I've used a basic for loop here. And this is

[40:08] from the hybrid retriever. Now, let's check with the vector retriever. So, vector basically hybrid retriever will send it will uh test I mean, the code test the retriever by sending the query this SDE Pune and the info function here

[40:24] it runs the query through the BM25 keyword search and vector based semantic search and it will combine the result as well using the predefined weights that we have given that is 0.4 and 0.6 here. And it will return the most relevant

[40:37] documents. That is these documents. And the for loop here it will print the content of each retrieved document with this as a separator. If it's not there then it doesn't visually look good so that is why I've uh

[40:51] And this will basically allow you to verify that the retriever is finding documents related to both the exact keywords that is this SDE and Pune and the overall meaning of the query. Now let's just quickly check it with the

[41:04] vector database. So for the vector also it's similar. It has pulled us the information correctly. So this has executed. Now let's just move into metadata filtering. So basically when we chunk the documents

[41:20] LangChain automatically attaches some metadata to it like uh which file each chunk came from and we can use that to filter the retrieval when we already know which document type is relevant. So for chunk in chunks

[41:35] print chunk.metadata. So it has given us the source talent loop. So we'll see each chunk carries its own source file name. So it it is showing us where that particular uh metadata is coming from. We can

[41:51] filter retrieval to only search within a specific document. So basically it's useful for our uh AI assistant if it classifies what type of question it is uh it's being asked and if it is compensation interview process or policy

[42:05] and then it will restrict the retrieval to just that document. Like it will not care about the other two documents entirely. So, that's about three of the techniques that is re-ranking, hybrid search, and metadata. So, basically

[42:19] the pipeline of this would be the wide retrieval and then re-rank and then generate. So, basically the these three techniques that is re-ranking, hybrid search, and metadata these are the difference between a demo that works on

[42:33] three sample files and a system which works with uh hundreds of documents at once. So, we will see that in the next mod- coming module. We will combine everything and we'll see the final production style assistant of Rag how it

[42:48] works. >> [music] this uh lesson. So, let's just go ahead and understand about prompt engineering in Rag. So, I'm just giving it a basic uh prompt.

[43:03] So, basically uh this is the prompt that we are using in the whole course. let's just understand why we are using only this particular prompt in this way in this manner. So, this line, your internal HR assistant for Talent Group,

[43:17] is basically it will set the role and the persona as well. So, it tells the model what voice to answer in and implicitly what kind of questions to Then we have this line, answer the question using only context below. Why

[43:30] single This is the most important line in the entire prompt because without it the model will blend what's in your retrieved content with whatever it already knows from the training data. So, it sounds helpful but also it

[43:43] defeats the purpose of Rag basically. You want the model constrained to your documents, then do not use its general knowledge. So, this is very important to mention that use only the context below and not the previous context.

[43:57] And if the answer isn't context, say you don't have that information. Because doesn't find the answer in the context, its default behavior is to guess anyway, like to guess any answer because it tells us what is helpful for that

[44:11] particular question. So, it retrieves the answers from the other documents as well. So, when we asked in the previous question if we don't have the information it was showing us that

[44:25] it doesn't know the answer. Where is it? So, yeah, here when we asked the question, what's the compensation band for a senior team Bangalore and how many rounds and what was the maternity leave policy? This I did not mention this in

[44:38] the document that I provided. So, to check if it is given this question and it has shown us that yeah, I don't have that information. So, this is where we understand that the model correctly

[44:50] declined instead of inventing a particular answer. It is showing that the it doesn't know the information. So, writing a perfect prompt prompt is very important. Let's just understand

[45:03] a good prompt because it's very important when you're working with rag applications. It will hallucinate very easily and it will give you wrong clear when you're giving the instructions.

[45:17] So, now what I've written is I've used retriever here to retriever here to search the context. So, I've given a code in such a way that I've asked a question, what's the standard notice

[45:30] period for Talent Loop and does it apply to interns? Whatever it is given us, it has assumed that this is the 30 days is the notice period for Talent Loop and because this is what the weak prompt does. I have just given a basic context

[45:47] and question without any proper guidance and proper instructions. Just weak chain through runnable pass through and then you just have to print this. Now, if we go through the like good prompt good

[46:00] instructions So, now I've given it the template, like the proper structure and proper prompt, basically. This is the good prompt that assistant. Answer the question using only This is I haven't asked these I

[46:14] haven't used this prompt in the above command. I'm just using it here because between a weak and a good prompt. I'm telling it the instruct giving the assistant. Answer the question. If the answer is not correct, say you don't

[46:28] answer is not correct, say you don't have that information. So, now I've written a good prompt. You are internal HR assistant for Talent Loop questions using only the context below. If the answer is not correct, then you

[46:41] have to say I mean if the answer isn't in the context, you say you don't have that information accordingly. And I've given the same question. Like I've asked standard notice period for Talent Loop

[46:54] us that it doesn't have the provided information in the provided document. like if the prompt is weak, you can see the answer that it is randomly generated the answer from the document that is given.

[47:09] So, but if you use the good prompt, then you can definitely get the answer that you want. That is it doesn't have the information that is provided.

[47:25] with source. This function will it takes the retrieve chunks and prefixes each with its file name so that model can see where each piece of context came from. And the cited prompt here, it is a template that adds one more instruction

[47:39] that is cite the source documents at the end of the answer. And the cited chain together the hybrid retrieval press reranking that is hybrid retrieval and rerank with the new source of aware formatter and the citing prompt and then

[47:55] sends it to the LLM. The runnable lambda here it lets you A runnable lambda lets you plug a regular Python function and the format docs with source here which I've already told you it is a

[48:11] It helps you plug the LLM into the LCL that is this chain as a processing setup. And this side of chain in work it runs a full pipeline end to end and brings the answer that is either correct or incorrect. That's how it works.

[48:30] Now let's build a real world project that is building the HR assistant. The complete project from end to end. So we will first aid documents then chunks and then embed embeddings and then store it then hybrid retrieve re-rank and then

[48:44] answer generation. So basically I've written this prompt here. This basically change together what we built separately in the previous module. It will first pull a broader more accurate candidate and then using

[49:00] narrow it down using the cross encoder and decoder. Then I've written a prompt know this is a good prompt how it should be and then if I run it it's executed and now we have to test

[49:15] it's executed and now we have to test it. it's a rag chain for the talent loop internal HR assistant which we are building now. So the prompt instructs the LLM to answer questions only using

[49:28] I've mentioned here. It does not have the information if the answer is not available and which helps it will reduce the hallucinations basically. So chain first takes the user question that is the hybrid retrieve and re-rank here to

[49:43] retrieve and rank the most relevant documents and then this format docs prepares these documents as context and then as you know this runnable pass question and both the context and question are inserted into the prompt

[49:56] basically. The prompt is then sent to the LLM and the string out parser, this function, it converts the response into clean and text output basically. So, this creates a proper structured workflow with retrieval, re-ranking,

[50:10] context, and controlled like answer generation. Now, we just have to ask it It is showing that it doesn't have information. Let's just test it with information. Let's just test it with information that it has.

[50:36] policy which I have mentioned in the document. So, we can also test it with document. So, we can also test it with asking how many interviews.

[50:57] interview rounds for this particular role in Pune. So, everything is working perfectly. So, let's just ask again which is not in the all document policy, that is the maternity leave.

[51:17] is already done. You can just ask for what is the earned what is the earned How many earned leaves do we have?

[51:30] information. So, it is showing us that. So, asking something that's not in any of the documents because we instructed the model to only answer from the given information. So, this is the hallucination

[51:42] that I have explained to you before. So, it's working perfectly for us And this is a fully function document and hybrid search assistant for you. It is built entirely from whatever we have learned earlier, like from the basics.

[51:59] So, this is how we have created the Talent Loop Hiring Assistant created the Talent Loop Hiring Assistant and made sure that the like the rag assistant doesn't show any hallucinations and it like provides

[52:13] perfect answers for us whenever we ask a question. I hope you got a clear understanding of how rag works in behind the scenes. So, yeah. This is in behind the scenes. So, yeah. This is the This is the end of the demo.