What is RAG? (Retrieval Augmented Generation)

[00:00] hello, everyone, uh, welcome, to, my, code

[00:02] deare, uh, video, series, um, what, I'm, doing

[00:06] is, I'm, rotating, through, three, different

[00:08] types, of, topics, educational, topics, uh

[00:11] use, case, topics, and, then, kind, of, bias

[00:13] ethics, safety, uh, topic, so, now, on, the

[00:17] education, rotation, and, today, what, I

[00:19] wanted, to, talk, about, is, uh, what, is

[00:21] retrieval, augmented, generation, or, rag, uh

[00:26] and, you, may, think, that, I'm, going, into

[00:28] some, kind, of, nook, and, cranny, of, the, AI

[00:31] uh, field, but, this, is, a, very, important

[00:33] and, popular, kind, of, solution, pattern

[00:36] that, I, see, um, being, used, over, and, over

[00:39] and, over, again, for, uh, how, to, leverage

[00:42] large, language, models, so, I, thought, I

[00:44] would, explain, it, uh, to, you, uh, and, the

[00:47] the, the, thing, that, this, is, used, for, is

[00:49] basically, systems, that, leverage, large

[00:51] language, models, but, on, your, own, content

[00:55] so, let, me, describe, that, if, you, think, of

[00:57] like, the, chat, GPT, experience, and, if, you

[01:00] think, about, that, um, relative, to, like, the

[01:02] search, engine, experience, that, we, had

[01:05] before, if, you, ask, a, question, like, um, I

[01:08] don't, know, what, color, is, the, sky, or, how

[01:10] do, I, fix, this, plumbing, issue, or

[01:12] something, like, that, a, search, engine

[01:14] would, go, out, uh, or, appear, to, go, out

[01:17] search, the, internet, find, relevant

[01:19] content, and, then, just, list, that, content

[01:21] for, you, list, those, links, and, then, you, as

[01:24] a, user, would, need, to, click, on, the, links

[01:26] that, seem, seem, right, read, it, digest, it

[01:29] and, figure, out, the, answer, to, your

[01:31] question, what, a, large, language, model

[01:33] does, is, it, seems, to, do, that, first, part

[01:35] meaning, leverage, the, content, on, the

[01:37] whole, internet, but, instead, of, just

[01:39] listing, that, content, it, sort, of, digests

[01:41] it, digests, it, combines, it, assembles, it

[01:44] together, and, answers, your, question, sort

[01:46] of, generates, an, answer, um, so, it's, a

[01:49] whole, lot, better, I, mean, search, engines

[01:51] have, been, great, but, this, is, taking, the

[01:52] whole, experience, to, another, level, and, in

[01:55] addition, the, question, and, answering, uh

[01:57] you, can, also, give, it, instructions, like

[01:59] write, me, this, document, or, write, me, a

[02:01] lesson, plan, to, teach, geometry, to, seventh

[02:03] graders, uh, and, it, will, do, something

[02:05] similar, it, will, kind, of, assemble, content

[02:08] that, it, SE, that, it, has, seen, uh, that

[02:10] talks, about, geometry, or, seventh, graders

[02:12] or, how, to, do, lesson, plans, or, whatever, uh

[02:15] pulls, that, together, assembles, it, and

[02:17] then, writes, out, a, lesson, plan, okay, so

[02:21] it's, a, much, better, experience, than, just

[02:23] taking, the, raw, content, from, the, internet

[02:25] but, it, really, uh, creates, something, new

[02:28] from, that, now, let's, say, you, want, that

[02:30] same, experience, but, on, your, own, content

[02:33] so, it, might, be, a, chatbot, on, your, website

[02:36] or, you, might, have, a, library, of, PDF

[02:37] documents, that, this, documentation, for

[02:40] one, of, your, products, uh, and, instead, of

[02:42] just, linking, the, user, to, parag, sections

[02:46] of, the, documentation, you, want, to

[02:47] actually, answer, their, question, uh, it

[02:50] might, be, your, service, ticketing, uh

[02:52] system, so, when, a, new, issue, comes, in, you

[02:54] could, say, how, would, I, resolve, this, issue

[02:56] and, it, can, assemble, past, similar, issues

[02:59] uh, and, then, come, up, with, a, new, uh, new

[03:01] solution, based, on, that, so, this, is, an

[03:04] incredible, experience, that, these, large

[03:07] language, models, offer, but, how, can, you

[03:09] create, that, experience, on, your, own

[03:11] content, uh, that, might, not, be, available

[03:14] to, the, internet, or, available, to, these

[03:16] large, language, models, well, the, solution

[03:18] to, this, is, this, rag, um, architecture, this

[03:22] retrieval, augmented, uh, generation

[03:24] architecture, so, now, I'm, going, to, do, my

[03:25] best, to, explain, that, uh, to

[03:28] you, so, let's, say, you, have, a, um

[03:32] user, and, I'm, going, to, use, the, example, of

[03:35] a, uh, patient, chatbot, and, the, content

[03:39] source, is, going, to, be, that, content, from

[03:41] your, website, let's, say, or, could, be

[03:43] content, from, PDF, documents, or, or

[03:46] whatever, but, you, want, this, to, be, the

[03:47] content, to, answer, the, patient's

[03:49] questions, so, if, the, patient, has, a

[03:50] question, like, how, do, I, prepare, for, my

[03:52] knee, surgery, instead, of, just, going, to

[03:55] chat, sheet, PT, and, getting, a, generic

[03:56] answer, you'd, like, to, provide, an, answer

[03:59] that's, from, your, health, system, or, a

[04:02] question, like, do, you, have, parking, you'd

[04:04] like, to, provide, an, answer, for, your

[04:06] health, system, for, your, the, office, where

[04:07] the, patient, is, seen, okay, so, that's, a

[04:10] scenario, that, I'd, like, to, do, so, the

[04:12] patient, has, a

[04:13] question, uh, and, I'm, going, to, do, do, you

[04:16] have

[04:17] parking, have

[04:23] parking, um, you, can, uh, imagine, that

[04:26] question, being, bundled, up, into, a, prompt

[04:30] what's, called, a, prompt, and, I'll, describe

[04:32] this, more

[04:33] later, so, there, is, the, question, that

[04:37] prompt, is, sent, to, a, large, language

[04:40] model, and, that, large, language, model, will

[04:44] come, up, with, a, response, to, that, question

[04:48] okay, now, um, if, you, just, wanted, to, use, uh

[04:51] chat, GPT, let's, say, or, some, other, llm, uh

[04:54] without, any, extra, content, you, could, just

[04:57] use, this, flow, how, do, I, prepare, for, my

[04:59] knee, surgery, or, do, you, have, parking, put

[05:02] that, into, a, prompt, send, that, to, the, uh

[05:04] large, language, model, and, get, a, response

[05:06] back, okay, but, uh, but, what, we, want, to, do

[05:09] is, enhance, this, experience, with, our, own

[05:11] content, so, let's, say, here, is, your

[05:13] content

[05:14] source, and, again, this, might, be, all, the

[05:17] content, of, your, website, or, PDF, documents

[05:21] or, internal, ticketing, system, or

[05:23] databases, or, that, uh, that, sort, of, thing

[05:27] and, what, you'd, like, to, do, is, something

[05:29] called, called, the, prop, before, the, propt

[05:32] so, in, these, systems, you, don't, just, send

[05:34] the, user, question, to, the, large, language

[05:36] model, you, usually, have, some, level, of

[05:38] instructions, So, the, instructions, might

[05:41] be, you, are, a, contact, center, specialist

[05:44] working, for, a, hospital, answering, patient

[05:47] questions, that, come, in, over, the, Internet

[05:50] uh, please, be, uh, nice, to, the, patients, and

[05:53] responsive, and, folksy, because, that, fits

[05:55] with, our, brand, or, some, instructions, like

[05:58] that, are, sometimes, sent, with, the, prompt

[06:01] um, and, then, uh

[06:03] Additionally, you, want, to, provide, the

[06:06] information, that, the, L, llm, needs, to

[06:08] answer, the, question, so, what, you'd

[06:11] ideally, like, is, information, from, your

[06:14] website, to, be, included, here, um, and, uh

[06:18] and, that, to, be, sent, to, the, llm, as, well

[06:20] so, the, full, prompt, might, be, your

[06:23] instructions, it, might, be, something, like

[06:25] please, use, this, content, um, in, order, to

[06:28] answer, the, patient, question, at, the, end

[06:30] and, then, you, put, in, a, bunch, of

[06:32] information, about, parking, or, about, knee

[06:34] surgery, or, whatever, the, patient, asked

[06:36] you, put, that, in, the, prompt, before, the

[06:38] prompt, then, you, have, the, question, then

[06:40] you, send, that, whole, package, to, the, llm

[06:43] and, the, llm, will, give, a, great, response

[06:45] based, on, your

[06:47] content, okay, with, me, so, far, so, um, so

[06:51] this, notion, is, the, prop, before, the

[06:53] prompt, um, and, and, that's, why, prompt

[06:56] engineering, and, these, types, of, things

[06:58] are, a, big, field, right, now, now, because

[07:00] you, can, really, hone, the, um, these, systems

[07:03] by, doing, a, better, and, better, job, with

[07:05] the, actual, prompt, before, the, prompt, um

[07:08] in, uh, in, this

[07:10] style, now, the, last, trick, here, is, your

[07:14] website, or, your, content, is, huge, and, it

[07:16] talks, about, all, kinds, of, topics, Beyond

[07:19] parking, and, Beyond, knee, surgery, so, you

[07:21] really, want, to, somehow, pull, out, only, the

[07:24] parts, of, your, content, that, are, relevant

[07:26] to, the, patient's, question, so, this, is

[07:29] another, um, a, tricky, part, of, this, whole

[07:32] rag, architecture, uh, and, the, way, that

[07:34] works, is, that, um, you, take, all, your

[07:37] content, and, you, break, it, into, chunks, or

[07:40] these, systems, will, break, it, into, chunks

[07:42] so, chunk, might, be, a, paragraph, of, content

[07:44] or, a, p, or, a, couple, paragraphs, a, page

[07:46] something, like, that, and, then, those, um

[07:50] chunks, are, sent, to, a, large, language

[07:53] model, could, be, the, same, one, or, a

[07:55] different, one, and, they, are, turned, into, a

[07:58] vector

[08:01] and, uh, so, each, each, paragraph, or, each

[08:04] chunk, will, have, a

[08:06] vector, which, is, just, is, just, a, series, of

[08:09] numbers, and, that, series, of

[08:12] numbers, you, can, think, of, it, as, the

[08:14] numeric, representation, of, the, essence, of

[08:17] that

[08:18] paragraph, and, what's, uh, different, about

[08:21] these, numbers, just, they're, not, random

[08:23] numbers, but, paragraphs, that, talk, about, a

[08:25] similar, topic, have, close, by, numbers, they

[08:28] almost, have, the, same, vectors, okay, so, in

[08:31] addition, to, the, uh, it's, a, numera, Zed

[08:33] version, of, the, paragraph, but, it's, such

[08:37] that, similar, paragraphs, on, similar

[08:39] topics, will, have, similar, vectors, will

[08:42] have, similar, numbers, so, that, means, that

[08:46] what, happens, is, when, um, uh, a, user, will

[08:49] ask, a, question, like, do, you, have, parking

[08:51] let's

[08:52] say, then, that, is, also, sent, to, the, llm, in

[08:55] real, time, right, after, the, user, asked, the

[08:58] question

[08:59] that, comes, up, with, the, vector, as, well

[09:02] you, could, think, of, that, as, the, question

[09:04] vector, and, then, what, happens, we, do, we, do

[09:06] a, mathematical, comparison, real, quick

[09:09] between, the, vector, of, the, question, and

[09:11] then, the, vectors, of, your, content, and

[09:13] pick, like, the, top, five, documents, that

[09:15] are, closest, to, this, question, so, do, you

[09:17] have, parking, will, be, a, vector, then, you

[09:21] have, all, your, content, and, it's, going, to

[09:23] try, and, find, the, five, documents, that

[09:25] taught, the, most, about, parking, basically

[09:28] um, and, so, it'll, find, those, I, don't, know

[09:30] what, that, is, it'll, find, those, documents

[09:32] let's, say, uh, from, these, it'll, grab, the

[09:34] paragraphs, associated, with, those

[09:37] documents, um, and, it'll, use, that

[09:41] here, so, those, will, be, the, subset, of, your

[09:45] content, basically, that, is, used, as, part

[09:48] of, the, prompt, before, the, prompt, okay, so

[09:51] this, whole, uh, concept, is, uh, kind, of

[09:54] vectorizing, your, content, uh, typically

[09:58] that, then, our, storage, in, something

[09:59] called, a, vector, database, which, is

[10:01] basically, a, representation, of, your

[10:03] content, in, this, numeric, form, and, then

[10:06] this, system, that, you, build, this, rag

[10:08] system, will, uh, take, the, question, find

[10:12] retrieve, the, most, relevant, content, make

[10:15] that, as, part, of, the, prompt, before, the

[10:17] prompt, send, that, to, the, llm, and, then

[10:20] you'll, get, a, good, response, back, actually

[10:23] so, it's, a, little, bit, confusing, but, um

[10:25] but, it's, actually, not, that, confusing, um

[10:28] uh, I, just, made, it, more, confusing, by, this

[10:30] horrible, uh, horrible, drawing, but, this

[10:32] whole, thing, is, um, what, is, uh, called, rag

[10:37] retrieval, so, you're, retrieving, the

[10:39] relevant, documents, from, your, content

[10:42] you're, augmenting, the, generation, process

[10:45] so, you're, augmenting, the, lm's, ability, to

[10:48] do, generative, AI, based, on, the, documents

[10:51] that, you, retrieve, so, that's, why, it's

[10:52] retrieval, augmenting

[10:55] generation, okay, so, I, hope, that, made

[10:58] sense, uh, like, I, said, this, is, a, very

[11:00] popular, um, solution, pattern, that, I'm

[11:03] seeing, over, and, over, again, in, fact, the

[11:05] majority, of, llm, projects, that, I, see, are

[11:08] this, kind, of, thing, using, my, content

[11:11] packaging, that, up, with, an, llm, system, to

[11:14] create, a, kind, of, chat, chpt, like

[11:16] experience, for, my, employees, or, for, my

[11:20] customers, for, my, users, that, kind, of

[11:22] thing, and, it, works, extremely, well, that's

[11:24] why, uh, that's, why, it's, so, popular, so, I

[11:27] hope, that, was, interesting, and

[11:29] educational, and, made, sense, if, you, have

[11:31] any, questions, please, leave, them, for, me

[11:33] uh, as, part, of, the, comments, uh, thank, you

[11:35] very, much