[00:00]
Have you ever tried asking who won the
[00:02]
IPL in 2025
[00:04]
[Music]
[00:06]
or explain the code I wrote last week
[00:11]
and what happens? Nine times out of 10
[00:14]
it just starts hallucinated just making
[00:16]
stuff up going completely off the rails.
[00:18]
Well, if you used any LLM in the past
[00:20]
years, whether it's chat GPT, Claude,
[00:22]
Gemini Grock Mistral whatever you've
[00:24]
probably run into this one big annoying
[00:27]
problem. You ask something super
[00:29]
specific like a detailed question,
[00:30]
something about yourself, some code you
[00:32]
wrote last week or a spreadsheet that
[00:34]
you uploaded and the model answers super
[00:37]
confidently, like it knows everything,
[00:38]
but it completely misses the point.
[00:40]
Sometimes it just straight up
[00:42]
hallucinates and gives answers that
[00:44]
don't even exist. And look, the reason
[00:46]
is dead simple. Large language models
[00:48]
are pattern matching machines. They're
[00:50]
incredible at regurgitating what they've
[00:52]
already been trained on. But here's the
[00:54]
kicker. They don't know your data, your
[00:56]
context, or your secret source. And this
[00:59]
is exactly why AI is still struggling to
[01:01]
make a massive dent in fields like law,
[01:03]
medicine, and compliance. You know, the
[01:05]
places where hallucination isn't just an
[01:07]
oops, my bad kind of situation. It's
[01:09]
downright dangerous. Because let's be
[01:11]
real, when you yank out your context,
[01:13]
that fancy AI model just becomes
[01:15]
generic. It becomes mid. But there's got
[01:17]
to be a fix for this, right? We can't be
[01:19]
pushing AI this hard and just leave this
[01:22]
massive huge problem hanging. So, what
[01:25]
are we going to do? We've actually got
[01:26]
two solid ways to tackle this. And
[01:28]
today, we're going to break them down
[01:29]
for you.
[01:34]
Ways of fixing AI. All right, so we've
[01:36]
got this context problem. How do we
[01:38]
actually solve it? It turns out that
[01:40]
we've got two main players in the book.
[01:42]
So, let's dissect them. The first option
[01:43]
that we have is fine-tuning. Think of
[01:45]
this as sending your AI model back to
[01:47]
school. But this time, the curriculum is
[01:49]
all about you. You literally take that
[01:50]
base model and retrain it from the
[01:52]
ground up with your own data. Like your
[01:54]
emails, your entire code base, your
[01:56]
chats, your pictures, everything gets
[01:58]
thrown into the mix and it literally
[02:00]
learns your specific domain and becomes
[02:02]
a native. The upside is massive. Once
[02:04]
the model is trained up, it's like it
[02:06]
was literally born for your use cases.
[02:08]
You don't need to keep spoon feeding it
[02:09]
and giving it extra context every single
[02:11]
time. It just gets it. but and it's a
[02:13]
big butt. It can be extremely painful.
[02:16]
Seriously. So, GPU time is going to cost
[02:18]
you an arm and a leg. And what happens
[02:20]
when your data changes? New data, new
[02:22]
code, you guessed it, back to square
[02:24]
one. Repeat the entire process. And
[02:26]
plus, managing versions of these huge
[02:28]
model checkpoints is a messy logistical
[02:31]
nightmare. Trust me. So, that brings us
[02:34]
to option number two, which is RAG. And
[02:36]
RAG stands for retrieval augmented
[02:38]
generation. And folks, this is where
[02:40]
things get really, really interesting.
[02:42]
This is the street smart agile cousin.
[02:45]
Way way simpler. You don't even need to
[02:47]
touch the underlying base model. No
[02:49]
expensive retraining. Instead, you just
[02:51]
build the clever context engine. And you
[02:54]
can just think of this as a
[02:55]
superefficient research assistant that
[02:57]
sits around the LLM. And then at
[02:59]
runtime, when a query comes in, the
[03:01]
engine zips in and feeds the model just
[03:03]
the right pieces of information it needs
[03:05]
right when it needs them. Let's imagine
[03:07]
that you're a world-class chef. You know
[03:08]
how to cook anything, but you don't know
[03:10]
what the next order from the dining room
[03:12]
is going to be. With Rag, the moment
[03:14]
that order hits the kitchen, bam,
[03:16]
someone magically hands you the perfect
[03:18]
detailed recipe for the exact dish. You
[03:20]
didn't even have to deal on cooking. You
[03:22]
just got the precise instructions that
[03:24]
you needed. That is Rag right there.
[03:27]
That's the power. No retraining, live
[03:29]
updates, and way cheaper. So, now you
[03:31]
guys understand the beauty of Rag. But
[03:33]
why does this setup work so incredibly
[03:35]
well? Why is it becoming the go-to for
[03:37]
so many people trying to make LLMs
[03:39]
actually useful with their own data? Why
[03:41]
does Rag work so well? And here are the
[03:43]
reasons. Number one is fast iterations,
[03:45]
new docs, no sweat. Add them. Re-mbed
[03:48]
them and your rag will instantly get
[03:50]
smarter. No waiting for weeks for a
[03:52]
retrain. Next is cheap infrastructure.
[03:54]
Forget burning cash on endless GPU
[03:56]
cycles. Rag is lean, minimal compute and
[03:58]
your wallet will always stay happy. Next
[04:00]
is it's always fresh. Your info never
[04:03]
gets stale. upload a doc, your rag
[04:05]
adapts in seconds and always with the
[04:07]
latest intel. So you get speed, you can
[04:09]
save cash, and your AI always stays
[04:12]
current. That's a pretty powerful combo.
[04:14]
Okay, so now you're probably thinking,
[04:15]
okay, this sounds cool, but how does
[04:17]
this rag magic actually work under the
[04:20]
hood? Don't worry, we've got you. Rag
[04:22]
pipeline. Okay, so how does this rag
[04:24]
wizardry pull off giving your LM the
[04:26]
brains it needs without the pain of
[04:28]
retraining? We're going to break down
[04:30]
the entire pipeline. And to make sure
[04:33]
it's super easy to lock into your
[04:35]
memory, we're going to use an analogy.
[04:37]
Imagine you're setting up the most
[04:39]
insanely organized high- techch library
[04:41]
ever built. And for all you visual
[04:43]
thinkers out there, we've created this
[04:45]
crazy crazy massive diagram. So, we're
[04:47]
going to drop a link so you can explore
[04:48]
it on your own later, but for now, let's
[04:49]
walk through it together. All right,
[04:51]
let's dive in. All right, so step number
[04:53]
one is your data intake. Imagine this
[04:55]
being the part where the books arrive at
[04:57]
the library. The first things first,
[04:59]
your data. This is where all your books
[05:01]
start showing up at the library doors.
[05:04]
Think of your company's PDFs, your email
[05:06]
archives, critical CSVs, even your
[05:08]
entire codebase, all your content. So
[05:10]
consider this to be your raw materials,
[05:12]
the books that need to be cataloged in
[05:14]
our super library. Now we move on to
[05:15]
step two, which is chunking. Now imagine
[05:18]
this is where you're breaking down the
[05:19]
books into index cards. Now you're not
[05:21]
just going to cram the entire
[05:22]
encyclopedia onto one shelf, right? So,
[05:24]
you take each book, each document, and
[05:27]
you chunk it. You break it down into
[05:28]
smaller bite-sized pieces. And you can
[05:30]
think of them as individual index cards,
[05:32]
maybe one paragraph per card or logical
[05:35]
section. And the key is digestible
[05:37]
pieces. Why? So, instead of your AI
[05:40]
librarian having to flip through 300
[05:43]
pages to find a single answer, it can
[05:45]
search these cards way faster and way
[05:47]
more effectively. Precision, people,
[05:49]
it's all about precision. So the tools
[05:51]
that you can use for this is lang text
[05:52]
split or you can also use llama index.
[05:55]
Now we're moving on to step three which
[05:56]
is embedding. Now imagine this to be the
[05:58]
part where you're giving each card GPS
[06:00]
coordinates. Now this is where the real
[06:03]
AI magic starts to kick in. We take
[06:05]
those text chunks, those index cards and
[06:07]
we run them into coordinates. Now think
[06:09]
of it as assigning a super precise GPS
[06:12]
location to every single piece of
[06:13]
information in your library but for
[06:15]
language. The trick is that the cards
[06:18]
with similar meaning get plotted in
[06:20]
nearby locations in this massive
[06:22]
multi-dimensional space. So words like
[06:24]
similar, same, identical, they're all
[06:27]
hanging out in the same neighborhood.
[06:28]
Popular models that you can use for this
[06:29]
is Google's text embedding API or you
[06:31]
can also use OpenAI's text embedding 3.
[06:34]
So you have a lot of horsepower to
[06:35]
choose from. Now we're moving on to step
[06:36]
number four, which is vector storage.
[06:38]
Now this you can imagine as organizing
[06:40]
the high-tech shelves. All right, so our
[06:42]
Index cars now have their GPS
[06:44]
coordinates. So, next up, we're going to
[06:45]
need some serious shelving to store
[06:47]
them. And this isn't your grandma's
[06:49]
dusty bookshelf. This is a high
[06:51]
performance vector database. So, you've
[06:53]
got names like Pine Corn, Chroma,
[06:55]
Qentrint in the Ring. Pick the one whose
[06:56]
landing page you vibe with the most or
[06:58]
the one that fits your scale and budget.
[07:00]
Seriously, they're all pretty good. And
[07:02]
it doesn't matter if you got a,000 cards
[07:04]
or 10 million. These databases are built
[07:06]
for speed. They can use semantic
[07:09]
searches, finding those relevant meaning
[07:11]
coordinates in milliseconds. Blink and
[07:13]
you'll probably miss it. Okay, now we're
[07:15]
moving on to step number five which is
[07:16]
retrieval. Imagine this to be the part
[07:17]
where the librarian finds the exact
[07:19]
cards. Okay, so now your library is set
[07:21]
up. Now user walks in with a question.
[07:24]
So after the user asks that questions,
[07:25]
what do you think is going to happen? So
[07:26]
first the rag system takes that user's
[07:29]
query, embeds it and turns it into a
[07:31]
vector just like it did with all your
[07:33]
documents. Then it performs a similarity
[07:35]
search against your entire vector
[07:37]
database and does something like show me
[07:38]
the top five or six cards whose content
[07:41]
is semantically closest to this
[07:42]
question. So those are going to be your
[07:44]
golden index cards with each one of them
[07:46]
holding a crucial part of the answer and
[07:49]
also a relevant snippet of information.
[07:51]
Now we're moving on to step number six
[07:52]
which is synthesis. This is the part
[07:54]
where the librarian writes the perfect
[07:56]
answer. This is where our super smart
[07:58]
LLM, our AI librarian steps up to the
[08:01]
plate. We feed it those top rank
[08:03]
relevant chunks plus the original user
[08:05]
query and we usually give it a little
[08:07]
nudge, a guardrail prompt so to speak,
[08:10]
something like use only the context
[08:12]
provided. If the answer isn't there,
[08:14]
just say so. The LLM then reads these
[08:16]
carefully selected cards, understands
[08:18]
the question in that specific context,
[08:20]
and spits out focused, accurate, and
[08:22]
contextual answer. No hallucination, no
[08:24]
wild guessing, and no making stuff up.
[08:27]
It's answering like it knows your data
[08:29]
because in that moment, for that query,
[08:31]
thanks to Rag, it actually does. So now,
[08:34]
theory is great. Analogies are fun, but
[08:36]
at Builder Central, we're all about
[08:38]
building and shipping. So, now that
[08:40]
we've walked you through how Ragg
[08:41]
actually works, how about we show you
[08:42]
what we actually built using the same
[08:44]
exact approach.
[08:46]
Ragbot, this isn't a full-blown line by
[08:49]
line coding tutorial on how we built
[08:50]
this specific chatbot. So, we actually
[08:53]
dove deep into any which was our main
[08:55]
tool for this in a previous video. If
[08:57]
you've missed that video, make sure you
[08:58]
check it out. The link is going to be
[08:59]
either in the description or somewhere
[09:00]
over here, depends on where the editor
[09:02]
puts it. So, we showed how you can
[09:03]
visually build these kind of powerful
[09:05]
workflows with minimal to no code. So
[09:07]
here's our flow for the data source.
[09:09]
What we did is we used Google Drive and
[09:11]
connected it via GCP. Now the reason we
[09:13]
did this is because it enables us to
[09:15]
upload the documents in real time
[09:16]
effectively turning it into a live
[09:18]
database. For embeddings we used OpenAI
[09:20]
to generate them which was really really
[09:22]
easy and inexpensive. For storage we use
[09:24]
Pine Cone as our main vector storage
[09:26]
database because they offer a fairly
[09:28]
generous free storage tier. For
[09:30]
retrieval and synthesis, we use Google's
[09:32]
API, which handled the LLM part by
[09:33]
synthesizing answers based on the
[09:35]
embedded chunks that were received. So,
[09:37]
what does this actually look like in
[09:39]
action? Well, with this setup, you can
[09:41]
throw pretty much any file at it. PDFs,
[09:43]
Word docs, you name it. The bot chews it
[09:45]
up, processes it, and then boom, you're
[09:47]
chatting with your own data. So, you
[09:49]
need to ask it something specific like,
[09:50]
"What's the difference between function
[09:51]
A and function B in this massive
[09:53]
codebase I just uploaded?" And it should
[09:55]
spit back the answer according to the
[09:57]
document. It also works for CVs if
[09:59]
you're hiring. So those complicated
[10:01]
recipes you can never follow, dense
[10:04]
legal documents, your chaotic lecture
[10:06]
notes, whatever you've got, just, you
[10:07]
know, be smart about it. Don't upload
[10:09]
some deepest darkest secrets. Okay,
[10:11]
that's kind of stupid. Don't do that. So
[10:13]
what would be the end result? You can
[10:15]
literally just drag and drop any
[10:17]
document into the Google Drive folder
[10:19]
and the chatbot it updates either in
[10:21]
real time or on a schedule that you set
[10:24]
and then it's ready to answer your
[10:26]
questions using that fresh updated
[10:28]
context which is pretty cool, right?
[10:29]
JSON file for NA10 is in the
[10:31]
description. So, make sure you use that
[10:32]
and create your own ragbot. Basically,
[10:34]
in a nutshell, rag is making your AI
[10:37]
actually know your world. All right,
[10:39]
ladies and gentlemen, that's our session
[10:40]
for today. Until next time, keep
[10:42]
building, keep experimenting, and stay
[10:44]
tuned to Builder Central for more such
[10:45]
content.