Retrieval-augmented generation (RAG), Clearly Explained (Why it Matters)

[00:00] Have you ever tried asking who won the

[00:02] IPL in 2025

[00:04] [Music]

[00:06] or explain the code I wrote last week

[00:11] and what happens? Nine times out of 10

[00:14] it just starts hallucinated just making

[00:16] stuff up going completely off the rails.

[00:18] Well, if you used any LLM in the past

[00:20] years, whether it's chat GPT, Claude,

[00:22] Gemini Grock Mistral whatever you've

[00:24] probably run into this one big annoying

[00:27] problem. You ask something super

[00:29] specific like a detailed question,

[00:30] something about yourself, some code you

[00:32] wrote last week or a spreadsheet that

[00:34] you uploaded and the model answers super

[00:37] confidently, like it knows everything,

[00:38] but it completely misses the point.

[00:40] Sometimes it just straight up

[00:42] hallucinates and gives answers that

[00:44] don't even exist. And look, the reason

[00:46] is dead simple. Large language models

[00:48] are pattern matching machines. They're

[00:50] incredible at regurgitating what they've

[00:52] already been trained on. But here's the

[00:54] kicker. They don't know your data, your

[00:56] context, or your secret source. And this

[00:59] is exactly why AI is still struggling to

[01:01] make a massive dent in fields like law,

[01:03] medicine, and compliance. You know, the

[01:05] places where hallucination isn't just an

[01:07] oops, my bad kind of situation. It's

[01:09] downright dangerous. Because let's be

[01:11] real, when you yank out your context,

[01:13] that fancy AI model just becomes

[01:15] generic. It becomes mid. But there's got

[01:17] to be a fix for this, right? We can't be

[01:19] pushing AI this hard and just leave this

[01:22] massive huge problem hanging. So, what

[01:25] are we going to do? We've actually got

[01:26] two solid ways to tackle this. And

[01:28] today, we're going to break them down

[01:29] for you.

[01:34] Ways of fixing AI. All right, so we've

[01:36] got this context problem. How do we

[01:38] actually solve it? It turns out that

[01:40] we've got two main players in the book.

[01:42] So, let's dissect them. The first option

[01:43] that we have is fine-tuning. Think of

[01:45] this as sending your AI model back to

[01:47] school. But this time, the curriculum is

[01:49] all about you. You literally take that

[01:50] base model and retrain it from the

[01:52] ground up with your own data. Like your

[01:54] emails, your entire code base, your

[01:56] chats, your pictures, everything gets

[01:58] thrown into the mix and it literally

[02:00] learns your specific domain and becomes

[02:02] a native. The upside is massive. Once

[02:04] the model is trained up, it's like it

[02:06] was literally born for your use cases.

[02:08] You don't need to keep spoon feeding it

[02:09] and giving it extra context every single

[02:11] time. It just gets it. but and it's a

[02:13] big butt. It can be extremely painful.

[02:16] Seriously. So, GPU time is going to cost

[02:18] you an arm and a leg. And what happens

[02:20] when your data changes? New data, new

[02:22] code, you guessed it, back to square

[02:24] one. Repeat the entire process. And

[02:26] plus, managing versions of these huge

[02:28] model checkpoints is a messy logistical

[02:31] nightmare. Trust me. So, that brings us

[02:34] to option number two, which is RAG. And

[02:36] RAG stands for retrieval augmented

[02:38] generation. And folks, this is where

[02:40] things get really, really interesting.

[02:42] This is the street smart agile cousin.

[02:45] Way way simpler. You don't even need to

[02:47] touch the underlying base model. No

[02:49] expensive retraining. Instead, you just

[02:51] build the clever context engine. And you

[02:54] can just think of this as a

[02:55] superefficient research assistant that

[02:57] sits around the LLM. And then at

[02:59] runtime, when a query comes in, the

[03:01] engine zips in and feeds the model just

[03:03] the right pieces of information it needs

[03:05] right when it needs them. Let's imagine

[03:07] that you're a world-class chef. You know

[03:08] how to cook anything, but you don't know

[03:10] what the next order from the dining room

[03:12] is going to be. With Rag, the moment

[03:14] that order hits the kitchen, bam,

[03:16] someone magically hands you the perfect

[03:18] detailed recipe for the exact dish. You

[03:20] didn't even have to deal on cooking. You

[03:22] just got the precise instructions that

[03:24] you needed. That is Rag right there.

[03:27] That's the power. No retraining, live

[03:29] updates, and way cheaper. So, now you

[03:31] guys understand the beauty of Rag. But

[03:33] why does this setup work so incredibly

[03:35] well? Why is it becoming the go-to for

[03:37] so many people trying to make LLMs

[03:39] actually useful with their own data? Why

[03:41] does Rag work so well? And here are the

[03:43] reasons. Number one is fast iterations,

[03:45] new docs, no sweat. Add them. Re-mbed

[03:48] them and your rag will instantly get

[03:50] smarter. No waiting for weeks for a

[03:52] retrain. Next is cheap infrastructure.

[03:54] Forget burning cash on endless GPU

[03:56] cycles. Rag is lean, minimal compute and

[03:58] your wallet will always stay happy. Next

[04:00] is it's always fresh. Your info never

[04:03] gets stale. upload a doc, your rag

[04:05] adapts in seconds and always with the

[04:07] latest intel. So you get speed, you can

[04:09] save cash, and your AI always stays

[04:12] current. That's a pretty powerful combo.

[04:14] Okay, so now you're probably thinking,

[04:15] okay, this sounds cool, but how does

[04:17] this rag magic actually work under the

[04:20] hood? Don't worry, we've got you. Rag

[04:22] pipeline. Okay, so how does this rag

[04:24] wizardry pull off giving your LM the

[04:26] brains it needs without the pain of

[04:28] retraining? We're going to break down

[04:30] the entire pipeline. And to make sure

[04:33] it's super easy to lock into your

[04:35] memory, we're going to use an analogy.

[04:37] Imagine you're setting up the most

[04:39] insanely organized high- techch library

[04:41] ever built. And for all you visual

[04:43] thinkers out there, we've created this

[04:45] crazy crazy massive diagram. So, we're

[04:47] going to drop a link so you can explore

[04:48] it on your own later, but for now, let's

[04:49] walk through it together. All right,

[04:51] let's dive in. All right, so step number

[04:53] one is your data intake. Imagine this

[04:55] being the part where the books arrive at

[04:57] the library. The first things first,

[04:59] your data. This is where all your books

[05:01] start showing up at the library doors.

[05:04] Think of your company's PDFs, your email

[05:06] archives, critical CSVs, even your

[05:08] entire codebase, all your content. So

[05:10] consider this to be your raw materials,

[05:12] the books that need to be cataloged in

[05:14] our super library. Now we move on to

[05:15] step two, which is chunking. Now imagine

[05:18] this is where you're breaking down the

[05:19] books into index cards. Now you're not

[05:21] just going to cram the entire

[05:22] encyclopedia onto one shelf, right? So,

[05:24] you take each book, each document, and

[05:27] you chunk it. You break it down into

[05:28] smaller bite-sized pieces. And you can

[05:30] think of them as individual index cards,

[05:32] maybe one paragraph per card or logical

[05:35] section. And the key is digestible

[05:37] pieces. Why? So, instead of your AI

[05:40] librarian having to flip through 300

[05:43] pages to find a single answer, it can

[05:45] search these cards way faster and way

[05:47] more effectively. Precision, people,

[05:49] it's all about precision. So the tools

[05:51] that you can use for this is lang text

[05:52] split or you can also use llama index.

[05:55] Now we're moving on to step three which

[05:56] is embedding. Now imagine this to be the

[05:58] part where you're giving each card GPS

[06:00] coordinates. Now this is where the real

[06:03] AI magic starts to kick in. We take

[06:05] those text chunks, those index cards and

[06:07] we run them into coordinates. Now think

[06:09] of it as assigning a super precise GPS

[06:12] location to every single piece of

[06:13] information in your library but for

[06:15] language. The trick is that the cards

[06:18] with similar meaning get plotted in

[06:20] nearby locations in this massive

[06:22] multi-dimensional space. So words like

[06:24] similar, same, identical, they're all

[06:27] hanging out in the same neighborhood.

[06:28] Popular models that you can use for this

[06:29] is Google's text embedding API or you

[06:31] can also use OpenAI's text embedding 3.

[06:34] So you have a lot of horsepower to

[06:35] choose from. Now we're moving on to step

[06:36] number four, which is vector storage.

[06:38] Now this you can imagine as organizing

[06:40] the high-tech shelves. All right, so our

[06:42] Index cars now have their GPS

[06:44] coordinates. So, next up, we're going to

[06:45] need some serious shelving to store

[06:47] them. And this isn't your grandma's

[06:49] dusty bookshelf. This is a high

[06:51] performance vector database. So, you've

[06:53] got names like Pine Corn, Chroma,

[06:55] Qentrint in the Ring. Pick the one whose

[06:56] landing page you vibe with the most or

[06:58] the one that fits your scale and budget.

[07:00] Seriously, they're all pretty good. And

[07:02] it doesn't matter if you got a,000 cards

[07:04] or 10 million. These databases are built

[07:06] for speed. They can use semantic

[07:09] searches, finding those relevant meaning

[07:11] coordinates in milliseconds. Blink and

[07:13] you'll probably miss it. Okay, now we're

[07:15] moving on to step number five which is

[07:16] retrieval. Imagine this to be the part

[07:17] where the librarian finds the exact

[07:19] cards. Okay, so now your library is set

[07:21] up. Now user walks in with a question.

[07:24] So after the user asks that questions,

[07:25] what do you think is going to happen? So

[07:26] first the rag system takes that user's

[07:29] query, embeds it and turns it into a

[07:31] vector just like it did with all your

[07:33] documents. Then it performs a similarity

[07:35] search against your entire vector

[07:37] database and does something like show me

[07:38] the top five or six cards whose content

[07:41] is semantically closest to this

[07:42] question. So those are going to be your

[07:44] golden index cards with each one of them

[07:46] holding a crucial part of the answer and

[07:49] also a relevant snippet of information.

[07:51] Now we're moving on to step number six

[07:52] which is synthesis. This is the part

[07:54] where the librarian writes the perfect

[07:56] answer. This is where our super smart

[07:58] LLM, our AI librarian steps up to the

[08:01] plate. We feed it those top rank

[08:03] relevant chunks plus the original user

[08:05] query and we usually give it a little

[08:07] nudge, a guardrail prompt so to speak,

[08:10] something like use only the context

[08:12] provided. If the answer isn't there,

[08:14] just say so. The LLM then reads these

[08:16] carefully selected cards, understands

[08:18] the question in that specific context,

[08:20] and spits out focused, accurate, and

[08:22] contextual answer. No hallucination, no

[08:24] wild guessing, and no making stuff up.

[08:27] It's answering like it knows your data

[08:29] because in that moment, for that query,

[08:31] thanks to Rag, it actually does. So now,

[08:34] theory is great. Analogies are fun, but

[08:36] at Builder Central, we're all about

[08:38] building and shipping. So, now that

[08:40] we've walked you through how Ragg

[08:41] actually works, how about we show you

[08:42] what we actually built using the same

[08:44] exact approach.

[08:46] Ragbot, this isn't a full-blown line by

[08:49] line coding tutorial on how we built

[08:50] this specific chatbot. So, we actually

[08:53] dove deep into any which was our main

[08:55] tool for this in a previous video. If

[08:57] you've missed that video, make sure you

[08:58] check it out. The link is going to be

[08:59] either in the description or somewhere

[09:00] over here, depends on where the editor

[09:02] puts it. So, we showed how you can

[09:03] visually build these kind of powerful

[09:05] workflows with minimal to no code. So

[09:07] here's our flow for the data source.

[09:09] What we did is we used Google Drive and

[09:11] connected it via GCP. Now the reason we

[09:13] did this is because it enables us to

[09:15] upload the documents in real time

[09:16] effectively turning it into a live

[09:18] database. For embeddings we used OpenAI

[09:20] to generate them which was really really

[09:22] easy and inexpensive. For storage we use

[09:24] Pine Cone as our main vector storage

[09:26] database because they offer a fairly

[09:28] generous free storage tier. For

[09:30] retrieval and synthesis, we use Google's

[09:32] API, which handled the LLM part by

[09:33] synthesizing answers based on the

[09:35] embedded chunks that were received. So,

[09:37] what does this actually look like in

[09:39] action? Well, with this setup, you can

[09:41] throw pretty much any file at it. PDFs,

[09:43] Word docs, you name it. The bot chews it

[09:45] up, processes it, and then boom, you're

[09:47] chatting with your own data. So, you

[09:49] need to ask it something specific like,

[09:50] "What's the difference between function

[09:51] A and function B in this massive

[09:53] codebase I just uploaded?" And it should

[09:55] spit back the answer according to the

[09:57] document. It also works for CVs if

[09:59] you're hiring. So those complicated

[10:01] recipes you can never follow, dense

[10:04] legal documents, your chaotic lecture

[10:06] notes, whatever you've got, just, you

[10:07] know, be smart about it. Don't upload

[10:09] some deepest darkest secrets. Okay,

[10:11] that's kind of stupid. Don't do that. So

[10:13] what would be the end result? You can

[10:15] literally just drag and drop any

[10:17] document into the Google Drive folder

[10:19] and the chatbot it updates either in

[10:21] real time or on a schedule that you set

[10:24] and then it's ready to answer your

[10:26] questions using that fresh updated

[10:28] context which is pretty cool, right?

[10:29] JSON file for NA10 is in the

[10:31] description. So, make sure you use that

[10:32] and create your own ragbot. Basically,

[10:34] in a nutshell, rag is making your AI

[10:37] actually know your world. All right,

[10:39] ladies and gentlemen, that's our session

[10:40] for today. Until next time, keep

[10:42] building, keep experimenting, and stay

[10:44] tuned to Builder Central for more such

[10:45] content.