[0:00] In almost all Gen AI engineer job
[0:02] postings, you will find one common
[0:04] skill, retrieval augmented generation,
[0:06] also known as rag. In my company at
[0:09] Lake, when we build AI projects for our
[0:11] clients, more than 40% of these projects
[0:14] have rag component in it. So, what
[0:16] exactly is rag? What are different
[0:18] types? Is naive rag dead due to
[0:20] vectorless rag? We are going to cover
[0:22] all these rag basics in a very simple
[0:25] and intuitive language. We will not just
[0:27] talk theory. I will show you a rag
[0:30] project which is a customer care chatbot
[0:32] in telecom domain. In the end, I will
[0:35] share some useful resources including
[0:38] rag interview questions. All right,
[0:40] let's get started. Let's understand rag
[0:42] using a simple example. When you ask
[0:44] ChatGPT a policy question for some
[0:47] private company, it won't be able to
[0:49] answer it because ChatGPT is trained on
[0:53] general internet knowledge. It doesn't
[0:55] know the HR policy details of any
[0:58] private company. But, if you give HR
[1:02] policy document to this LLM, which acts
[1:04] like a brain, it should be able to read
[1:07] a relevant section and provide you with
[1:09] the answer. This is similar to having a
[1:13] very smart student Mira, who is a
[1:15] computer science student, and you are
[1:17] asking her to appear in a microbiology
[1:20] exam, which is an open book exam. Now,
[1:22] Mira is generally good in terms of
[1:24] reading writing comprehension
[1:26] understanding, etc., but she doesn't
[1:28] know anything about microbiology. But,
[1:30] in this exam, she has been given a book
[1:33] on microbiology from which they are
[1:35] going to ask the questions. Now, Mira
[1:37] can use her reading, writing,
[1:40] comprehension skills to uh look at the
[1:42] book and she can write answers in the
[1:45] exam. So, here Mira's brain is like LLM,
[1:48] which has a good understanding of
[1:49] language, it has reasoning capabilities,
[1:52] and the book is like an HR policy
[1:54] document. It is an external knowledge
[1:56] where LLM can look into it and pull the
[1:59] answers. Let's now understand the
[2:01] two-step process of how rag works
[2:04] underneath. So, here I have HR policy
[2:06] document from Atliq, which will have a
[2:08] section on retirement benefits, okay?
[2:12] So, now if I go to ChatGPT, and if I
[2:14] copy this particular section, okay, and
[2:17] I ask my question related to
[2:20] contribution to employees retirement
[2:22] fund, in that case, ChatGPT will be able
[2:25] to answer that question. Because here
[2:27] you are asking question and providing uh
[2:31] knowledge as a reference in the context,
[2:33] and it can pull the answer. But, what if
[2:36] your HR policy document is 3,000 page
[2:38] PDF, okay? What if that knowledge is
[2:41] very big? What's going to happen in that
[2:44] case is you will run out of your context
[2:46] window limit. And even if you have a
[2:48] huge context window,
[2:50] uh you should still not feed the entire
[2:53] knowledge because it will be too many
[2:56] tokens, it will be costly. So, what
[2:58] people do is they will chunk this
[3:01] document. So, they will create, let's
[3:02] say, basic strategy is fixed-size
[3:06] chunks. And then, for a given question,
[3:09] you can pull the relevant chunks. So,
[3:11] for this particular question, let's say
[3:13] my first chunk is 70% uh probability
[3:17] that it will it will contain the answer.
[3:19] Second chunk is 60% match. And you can
[3:21] have, by the way, I'm showing just
[3:22] three, but you can have 1,000 chunks,
[3:24] and some of the chunks might have 5% or
[3:27] even 10% possibility that it may contain
[3:30] the answer. Let's say the chunk contains
[3:33] uh information on uh when Atliq was
[3:36] founded culture founders etc. then
[3:39] that doesn't has anything to do with the
[3:41] retirement question that you are asking,
[3:43] okay? So, the the relevance of that
[3:46] chunk will be very, very low. Now, how
[3:48] do you exactly find this kind of
[3:50] similarity? So, there is this concept of
[3:53] embeddings, okay? So, embedding is a
[3:55] process of converting text into a vector
[3:59] such that it can represent its meaning,
[4:02] okay? So, all the chunks, you will
[4:04] convert them into vector embeddings, and
[4:08] then you will store them into a vector
[4:10] database. This is different than your
[4:12] regular database. Your regular database
[4:15] can search using exact values, whereas
[4:17] vector database will be able to search
[4:20] using the meaning. So, when you search
[4:22] for, let's say, uh a company that is a
[4:25] leader in electric vehicle, it will
[4:29] return Tesla uh from the database. So,
[4:31] it is searching based on the meaning,
[4:33] not based on the exact word. To generate
[4:36] embedding, you can use variety of
[4:38] models, sentence transformer, text
[4:40] embedding three small, and so on. And
[4:42] there are many vector database choices
[4:44] that you have in market, Milvus,
[4:46] Quadrant, Chroma DB, and so on. This
[4:48] step is called indexing. This is the
[4:51] first step in rag process, where you are
[4:54] indexing all these vectors of chunks
[4:57] into a vector database. The second step
[4:59] is retrieval, where for a given
[5:01] question, you will generate embedding
[5:03] using the same embedding model. Then,
[5:05] you will try to find the relevant chunks
[5:08] in a vector database. So, here it is
[5:10] doing the semantic search, giving you a
[5:13] relevant vectors. You can specify top K
[5:16] factor, let's say I need two chunks or
[5:18] five chunks, and so on. And then, you
[5:20] will generate the
[5:22] actual text out of those chunks, and you
[5:25] will put it in your prompt along with
[5:28] the question. And when the question is
[5:30] given to LLM, it will give you the
[5:32] answer. So, here uh below the question,
[5:35] what you are doing is you are providing
[5:37] only the relevant chunks. So, this way
[5:39] LLM will not hallucinate, and it will
[5:42] give you accurate answer. That takes us
[5:45] into our next segment, which is the two
[5:47] major benefits of rag. The first one is
[5:50] the answers that you get will be highly
[5:53] accurate, and the chances of
[5:56] hallucination will reduce because you
[5:59] are grounding your responses in the
[6:01] knowledge, in the source of truth.
[6:03] Second, it is very cost-effective
[6:05] because if you pass the entire context,
[6:07] then you are sending too many tokens to
[6:10] LLM, and these LLM APIs, they charge you
[6:13] per token. So, if you send less number
[6:15] of tokens, only the relevant knowledge,
[6:17] then you will save money on your API
[6:19] bill. Here is a hands-on customer care
[6:22] assistant rag project. I have given the
[6:24] code in the video description below. You
[6:26] can ask different questions, for
[6:27] example, why is my mobile internet slow?
[6:30] And it will find the answer based on the
[6:33] knowledge that it has. So, the knowledge
[6:35] is stored in terms of the
[6:37] troubleshooting PDF file. So, here is
[6:39] the PDF file, and let's say you have
[6:42] this question on how do you want to
[6:44] enable the LTE, then it is pulling that
[6:47] answer from this particular PDF file.
[6:50] The other source is the CSV file
[6:53] containing all the FAQs. And the third
[6:56] source is a SQLite database containing
[6:59] all the past ticket. Here we are using
[7:01] Chroma DB as vector database. So, we are
[7:03] ingesting FAQs, then PDF, and tickets
[7:07] into Chroma DB, okay? So, these are the
[7:10] three files which is ingesting into a
[7:12] vector database. If you look at ingest
[7:14] PDF, here we are using the chunk size of
[7:17] 600, overlap of 100. We are using the
[7:19] recursive character text splitter
[7:22] strategy. And for embedding, we are
[7:24] using this particular embedding model
[7:26] from Hugging Face. As a framework, we
[7:28] have used LangChain. Now, the retriever
[7:31] will try to find relevant chunks from
[7:34] FAQ, tickets, or guides. In terms of
[7:37] LLM, we are using Quen from Chat Grok.
[7:40] Please download the project on your
[7:41] computer, try to run it to enhance your
[7:44] understanding on rag concepts. Telecom
[7:47] support chatbot that we just saw is an
[7:49] example of enterprise QA chatbot. There
[7:52] are many other industry use cases for
[7:55] rag. For example, you can build medical
[7:57] knowledge assistant, which can look at
[8:00] the vast amount of medical knowledge and
[8:02] pulls a relevant answer for your query.
[8:05] The other one is legal and compliance
[8:07] tools. Once again, here the knowledge
[8:09] will be your legal documents, and you
[8:12] want to pull the most relevant and
[8:14] accurate answer. HR chatbot is another
[8:16] example. Let's now look at rag
[8:18] categories. The first one is a vector
[8:20] rag, and naive rag is the example that
[8:23] we just saw, where you pull the top K
[8:27] relevant chunks from a vector database
[8:29] and answer user's question. The second
[8:32] category is vectorless rag, in which you
[8:35] can perform a keyword rag. So, here you
[8:38] are not generating any vector
[8:40] embeddings. You don't have a vector
[8:42] database, but you are using keyword
[8:45] match, uh techniques like BM25, TF-IDF,
[8:49] etc., to uh query into the document
[8:52] using the exact keywords. This method
[8:55] will work when you have a lot of codes,
[8:58] jargons, IDs, citations. Let's say you
[9:01] are doing research, and you are always
[9:03] searching using some particular ID or a
[9:07] particular keyword, then this will work.
[9:10] This is weak for semantic understanding.
[9:12] When you are not doing exact keyword
[9:14] matching and searching using meaning,
[9:16] this is not a good choice. And the key
[9:19] tools that uses keyword rag concepts
[9:21] like BM25 is Elasticsearch and Apache
[9:25] Solr. The next category in vector rag is
[9:28] hybrid rag, where you're combining
[9:31] vector search and keyword search, okay?
[9:34] You do both of these in parallel and
[9:37] merge the results. This is best for most
[9:39] of the production systems. The key tools
[9:42] here are Elasticsearch plus any vector
[9:45] DB. Now, in Atliq, we worked on one rag
[9:49] project for our client, where we have
[9:51] developed our own custom hybrid method
[9:55] for doing rag, and we have given details
[9:57] of this approach in a different video.
[10:00] You can check it out if you are
[10:01] interested. Also, if you want to learn
[10:03] AI engineering by building production
[10:05] grade AI systems similar to the projects
[10:07] that I just mentioned, then check our AI
[10:10] engineering cohort where we have live
[10:13] sessions on weekends and we will teach
[10:16] you all the concepts plus you will build
[10:18] eight plus production grade projects.
[10:20] The next category in vector less rag is
[10:22] a graph rag. It is also known as KG rag.
[10:27] So here you will generate a knowledge
[10:29] graph. So let's say your knowledge is
[10:31] Elon Musk and all the companies he has
[10:33] founded. So in that case you will build
[10:35] this kind of knowledge graph where you
[10:37] will say Elon Musk founded Tesla,
[10:39] SpaceX, Neuralink, OpenAI and so on.
[10:42] And then these companies will be
[10:43] operating in these different domains. So
[10:45] these are all the entities and they are
[10:47] connected through some kind of
[10:49] relationship. Now when you ask a
[10:51] question, which companies are founded by
[10:53] Elon Musk which are working in AI, you
[10:56] will traverse this particular path,
[10:58] okay? So you will look at all the
[11:00] companies and then you will do breadth
[11:02] first traversal and you will find that
[11:05] OpenAI is working in AI. The next one is
[11:08] SQL rag. This is also known as text to
[11:11] SQL. This method is very simple. Let's
[11:13] say you have sales database which
[11:15] contains the sales of
[11:18] products. Now you are asking this
[11:19] question, which product sold the most
[11:21] last month? Using LLM you can first
[11:24] generate a query for that database. You
[11:26] will execute the query, get the results
[11:29] and then give it back to LLM to generate
[11:31] a comprehensive answer. Very simple
[11:33] technique. You are taking a sentence in
[11:36] a natural language, converting it to SQL
[11:38] using LLM and putting a query in your
[11:41] database to get the results. And the
[11:43] last method, which is relatively new, is
[11:47] called page index. It is reasoning based
[11:50] rag. So here let's say you have 3,000
[11:53] page PDF document. First you will
[11:56] generate the table of content, okay? The
[11:59] table of content or your information
[12:02] structure. This is like you are having a
[12:04] book and you are having all the chapter
[12:06] and topic layout. Now when somebody asks
[12:08] this question, what does the contract
[12:09] say about compensation for breach of
[12:11] contract, the LLM will use its reasoning
[12:15] capability and this particular table of
[12:18] content to traverse this particular
[12:22] graph and locate the thing that it is
[12:25] looking for. So for example, in this
[12:28] case it will first find out that this is
[12:30] related to performance of contracts
[12:32] because the contract is already
[12:33] executed. So it has to be related to
[12:35] this and then it finds compensation of
[12:38] breach. So it goes from here to here and
[12:40] then
[12:41] you are discussing loss. So due to that
[12:44] it will out of all these nodes, it will
[12:46] go to this particular node and it will
[12:49] pull the relevant document. Now this
[12:53] might give you an index and using index
[12:55] you might have to refer back to the
[12:57] original knowledge. So here is the
[12:59] GitHub for page index. It is known as
[13:02] vector less rag but it is one of the
[13:04] categories of vector less rag, okay? The
[13:07] right term is reasoning based rag. So
[13:10] here you can see from document you are
[13:11] generating a tree, which is your
[13:13] knowledge tree structure index of
[13:16] documents and then LLM will do its
[13:18] reasoning to find the relevant chunk.
[13:21] Here you are not using any vectors. You
[13:22] are not using any
[13:24] embeddings. No vector DB. Just by
[13:26] looking at the the structure, you know,
[13:30] the table layout, which looks something
[13:32] like this,
[13:33] you will try to find a given node, okay?
[13:36] And see here there is a summary. So
[13:38] using the summary, LLM can reason and it
[13:41] can say, "Okay, maybe the answer is in
[13:44] this particular node." And then it will
[13:46] go to that node, refer to the original
[13:48] document and pull the answer. I have
[13:50] attached this PDF in the video
[13:51] description below where you have
[13:53] categories of rag. You also have a table
[13:56] comparing when to use what. It is not
[13:59] that
[14:00] reasoning rag is here so you should not
[14:02] use naive rag. You should use it when
[14:05] you have general text Q&A bots, etc. And
[14:08] the complexity here is low. The
[14:10] complexity in case of page index is
[14:12] high. You should use it when you have
[14:14] hierarchical tree index LLM traversal.
[14:18] You know, these are the use cases. So
[14:19] you can use this table to determine when
[14:21] to use what kind of rag. And at the end
[14:24] we have rag interview questions. All
[14:26] right, folks. So please check it out. If
[14:28] you have any question, post in the
[14:29] comment box below.
[14:33] >> [music]