[0:00] olama is an open-source tool that
[0:02] simplifies running large language models
[0:05] locally on your personal computer in
[0:09] this course you'll learn how to set up
[0:11] and use olama this Hands-On course
[0:14] covers polling and customizing models
[0:17] rest apis python Integrations and real
[0:21] world projects like a grocer lless
[0:23] organizer rag system and an AI recruiter
[0:26] agency Paulo created this course hello
[0:30] and welcome to this mini course I'm very
[0:33] excited that you're here because I know
[0:35] you want to get right in and start
[0:37] learning about AMA and building AI based
[0:41] Solutions using AMA first let me go
[0:43] ahead and introduce myself again which
[0:45] is always awkward okay my name is Paulo
[0:48] deson I am a software Ai and Cloud
[0:51] engineer but most importantly I am an
[0:54] online instructor and I have taught over
[0:58] 280,000 students
[1:00] all over the world in about 175
[1:03] countries uh skills ranging from
[1:06] building mobile applications Android to
[1:08] learning Ai and Cloud Technologies so
[1:12] this is what I do and I love what I do
[1:14] what is this minicourse about I want to
[1:16] show you how to leverage olama and its
[1:19] many many models so that you can build
[1:22] AI Solutions a applications locally for
[1:26] free usually the structure that I like
[1:28] to follow and has Pro proven to work is
[1:31] in the beginning or the first part of
[1:33] the courses I start with the basics the
[1:36] fundamentals as well as Theory Concepts
[1:40] because I think it's important for you
[1:42] to also understand the theory before you
[1:44] go into Hands-On and then the second
[1:46] part is usually Hands-On but sometimes I
[1:49] like to mix those two together so that
[1:52] it's more exciting and you don't get
[1:54] bored okay so I know you're very excited
[1:56] to get started and I'm excited that
[1:58] you're here okay let's go ahead and get
[2:00] started well this course is going to be
[2:02] about ol that's the main thing so we're
[2:04] going to learn how to build local large
[2:07] language model llm applications using
[2:10] AMA so the idea is that we're going to
[2:12] be able to use AMA to customize our
[2:15] models meaning that we are able to use
[2:18] different flavors of models so we can
[2:21] test them around and all of that is
[2:23] actually going to be free which is very
[2:25] exciting and also we're going to be able
[2:27] to build rag system so retrieval
[2:30] augmented generation systems that are
[2:33] powered solely by AMA models keep in
[2:37] mind all of this is going to be for free
[2:39] you're not going to have to pay for
[2:41] anything okay and we are going to learn
[2:45] about tools and function calling when we
[2:49] are using these AMA models so we can Aid
[2:52] our large language models that are going
[2:54] to be using with more things that they
[2:57] can do okay I'm going to look at all of
[2:59] that and ultimately I'm going to show
[3:01] you how to use AMA models to build
[3:04] full-fledged large language model
[3:06] applications using these models okay
[3:10] keep in mind all of these models are
[3:12] going to be for free we don't have to
[3:14] pay a dime to use these models now let's
[3:17] talk about who is this course for now
[3:20] this course is for developers AI
[3:21] Engineers open-minded Learners machine
[3:25] learning engineers and so forth as well
[3:28] as data scientists so if you are
[3:31] somebody who is willing to put in the
[3:33] work and wants to learn about AMA and
[3:38] build local llm applications then this
[3:41] course is for you now let's talk about
[3:43] course prerequisites in this course I
[3:46] assume that you know programming this is
[3:49] a very um important skill obviously and
[3:52] I expect you to at least know the basics
[3:55] you'll be seeing a lot of python in this
[3:58] course so understand understanding how
[4:00] python Works how to write python code is
[4:03] obviously very important you also need
[4:06] to understand very basics of AI machine
[4:09] learning large language model you don't
[4:11] have to be an expert but just have some
[4:14] general
[4:15] understanding keep in mind also that
[4:18] this is not a programming course because
[4:20] I'm not going to be teaching you how to
[4:22] program and most importantly I always
[4:24] tell my Learners is that you have to be
[4:26] willing to learn and if there things
[4:29] that you need to go and brush on just go
[4:32] and do that and come back to the course
[4:34] but most likely you won't need to go
[4:37] anywhere but continue in this course and
[4:39] learn as much as you can so most of my
[4:42] courses I have this mixture of two
[4:45] things I have Theory so this is where we
[4:47] talk about the fundamental concepts the
[4:50] lingo and so forth and I have Hands-On
[4:53] because it's all about actually doing
[4:55] things that way you actually understand
[4:57] and know how to get things done that's
[5:00] the whole point and so the thing is I'd
[5:02] like to do a mixture of both and
[5:05] sometimes we have more handson and
[5:07] sometimes we we may have more Theory but
[5:10] I try to balance it all out where we
[5:12] have mixture both because I know I
[5:14] believe at least that is important for
[5:16] you to have both in your learning
[5:19] process but in this course you are
[5:21] mainly going to be focusing on Hands-On
[5:24] although you will still have some Theory
[5:27] fundamentals as well all right so let's
[5:29] look at the development environment
[5:32] setup so in this case you know that this
[5:35] is all P about python which means you'll
[5:37] have to have python installed and also
[5:40] you have to have some sort of a code
[5:42] editor for me I'm going to be using VSS
[5:45] code so you don't have to use that if
[5:48] you don't like it but that's what I'm
[5:49] going to be using you can use whatever
[5:51] you want okay now I'm not going to go
[5:53] through the process of installing python
[5:55] or vs code or anything like that because
[5:57] you should be able to have that if not
[5:59] you should be able to do that on your
[6:01] own so one place that I always recommend
[6:04] people go is to this link here kin.com
[6:08] knowledgebase install python there are
[6:10] many places online where you can learn
[6:13] or they will walk you through the
[6:15] installation of python if that's your
[6:17] case uh but this seems to be one of the
[6:20] best my favorite okay so if you don't
[6:22] have python installed please go ahead
[6:24] and get that set up so that we can
[6:28] continue in the next next videos now we
[6:30] have everything set up on our machine
[6:32] and we're ready to go so we're going to
[6:34] do a deep dive into AMA so you
[6:36] understand exactly what is how it works
[6:39] and what problem um AMA solves all right
[6:42] so let's go ahead and get started now
[6:44] it's time for us to do the Deep dive in
[6:47] ama so we understand exactly what is AMA
[6:51] what is the motivation behind and the
[6:53] advantages that ama brings for us
[6:55] developers okay so first let's look at
[6:58] AMA what is it ama is an open-source
[7:02] tool that is designed to essentially
[7:04] simplify the process of running large
[7:08] language models locally meaning on your
[7:12] own Hardware the idea here is very
[7:14] simple as you know right now if you want
[7:17] to run large language models or if you
[7:20] want to use a model large language model
[7:22] in this case just a model most likely
[7:25] you'll have to use open Ai chbt and so
[7:28] forth and many others out there that are
[7:31] paid and the thing is with a Lama you
[7:34] don't have to pay for anything it's free
[7:37] and that's the beauty so the idea is
[7:38] that ama sits at the center and allows
[7:41] us developers to pick different large
[7:44] language models depending on the
[7:46] situation depending on what we want to
[7:49] do at its core AMA uses what we call the
[7:54] CLI which is command line interface what
[7:57] this does is it manages is all the
[8:00] things in the back end so the
[8:02] installation and also the execution of
[8:05] different models but all of that again
[8:08] keep in mind it's all locally you will
[8:11] see as we go through that ama abstracts
[8:13] away the technicality so the technical
[8:16] complexity that are involved when we
[8:19] want to set up these models which makes
[8:22] Advanced language processing accessible
[8:25] to a broader audience such as developers
[8:28] researchers and hobbyists in nutshell
[8:32] AMA provides a straightforward way to
[8:35] download run and interact with various
[8:40] models or LMS without us having to rely
[8:44] on cloud-based services or even dealing
[8:47] with complex setup procedures rather
[8:49] what is the problem that ama solves now
[8:52] when we talk about large language models
[8:55] we are usually talking about this thing
[8:58] called rag system retrieval augmented
[9:00] generation the idea is very simple is
[9:02] that we have documents those documents
[9:05] are chopped into smaller chunks right
[9:08] and then we pass those through a model
[9:12] that is responsible for creating
[9:14] embeddings which is the vector
[9:17] representation of these chunks those
[9:20] embeddings are saved somewhere and then
[9:22] a query comes in goes through the same
[9:25] process of embedding right and then once
[9:27] we have this embedding that is indeed
[9:29] what's actually used to query do similar
[9:33] search internally in a factor database
[9:37] so that we can then pass all that
[9:39] information right through a large
[9:41] language model a model and of course get
[9:44] the response now again usually what
[9:46] happen is that all these models that we
[9:48] need to run a rag system for instance we
[9:51] may need to pay such as when we use
[9:54] openingi trpt and all different models
[9:56] that they provide okay now the other
[9:59] thing also behind all of that is that
[10:01] when we are using AMA models these are
[10:05] ours meaning that we download them
[10:07] internally in our local machine and we
[10:10] have a lot of control okay so that is
[10:12] the beauty of or in this case that is
[10:15] the problem that they solved but that's
[10:17] not the only problem that been solved
[10:19] the other one is that privacy so in this
[10:22] case here when we run our own models
[10:26] locally uh we are making sure that our
[10:28] data doesn't need to be sent to external
[10:32] servers because remember when we passing
[10:35] data through a large language model
[10:37] using chbt in this case or openi models
[10:40] and so forth we literally are passing
[10:42] through or sending our private data to
[10:46] different servers that we don't know
[10:48] what will happen out there so this way
[10:51] we have some security issues but with
[10:53] AMA that goes away because we have this
[10:57] enhanced privacy and security this is
[10:59] very important because imagine that you
[11:00] have an application that needs to be uh
[11:04] dealing with sensitive information then
[11:07] it's important to have your own models
[11:11] your own llm so that you have this
[11:15] contained enir environment where you're
[11:17] not sending out that sensitive
[11:19] information when you want to set up
[11:20] large language models it is cumbersome
[11:22] which means is technically challenging
[11:25] and often requiring knowledge of machine
[11:28] learning
[11:30] Frameworks and also Hardware
[11:32] configurations but as you will see with
[11:34] AMA all of that is simplified so this
[11:36] whole process is simplified by handling
[11:40] the heavy lifting for you also most
[11:43] importantly cost efficiency now we're
[11:46] going to be eliminating the need for
[11:48] cloud-based Services which means you're
[11:50] going to avoid this ongoing costs
[11:53] associated with API calls or server
[11:56] usage because everything remember is
[11:58] going to be local goal so once you have
[12:01] everything set up you can run models
[12:03] without additional expenses the other
[12:05] one is lency reduction what that means
[12:08] is that local execution obviously
[12:11] reduces the latency inherent in network
[12:14] communications so when we are
[12:16] communicating with something that is
[12:18] somewhere else through the network of
[12:21] course there is always a delay even if
[12:23] the network is really fast there's
[12:25] always this latency the delay okay but
[12:28] since every everything is local we don't
[12:30] have that problem okay because we're not
[12:32] dealing with remote servers which
[12:35] results in Faster response times for
[12:38] interactive
[12:39] applications and also most importantly
[12:42] we are able to customize our own models
[12:45] because when we running these models
[12:47] locally it allows for greater
[12:50] flexibility in customizing or even
[12:53] fine-tuning those models to better suit
[12:56] specific needs without having this
[12:59] limitations imposed by thirdparty
[13:01] services so these are some of the
[13:04] advantages we have by using AMA
[13:07] models so I know that we are eager to go
[13:10] ahead and start doing the Hands-On but
[13:12] let's continue here to talk about some
[13:14] key features of AMA okay so the first
[13:17] one is what we call the Model Management
[13:20] this is a big one which means that we
[13:21] are able to easily download and if we
[13:25] want to able to switch between different
[13:29] models or between different large
[13:31] language models and that is the main
[13:33] point about AMA is that the idea is that
[13:37] we have this Center place called Lama as
[13:39] a open source as a framework as a tool
[13:42] that essentially allows us to manage all
[13:44] of the different models that we can use
[13:47] interchangeably okay and next one of
[13:50] course is we have this unified interface
[13:53] what does that mean well that means that
[13:54] we're able to interact with various
[13:57] models using one consistent set of
[14:00] commands as you will see and of course
[14:02] we also have what we call extensibility
[14:05] what that means well it that means we
[14:07] have this support for adding custom
[14:09] models and extensions if we need to
[14:13] obviously we have what we call also
[14:15] performance optimizations which means
[14:17] you're able to utilize your own Hardware
[14:20] effectively including GPU acceleration
[14:22] if available because remember all AMA
[14:25] allows us to do is to manage all this
[14:28] large language models that we can
[14:30] download locally that is the key word
[14:33] key Point here and then we can do all
[14:35] sort of things with that we have this
[14:37] performance that is optimized because
[14:39] everything is internal it's locally as
[14:42] well extensibility because we can add
[14:45] different models and extensions as we
[14:47] see fit and of course we have this
[14:49] unified interface because we're able to
[14:51] interact with different models uh but
[14:54] still using a very consistent set of
[14:56] commands as you will see but the main
[14:58] point here of of course is that we have
[15:00] this Model Management in one place we're
[15:03] able to easily download and switch
[15:05] between different large language models
[15:08] let's look at some use cases well as you
[15:10] can see you can imagine we have a lot of
[15:12] different use cases uh
[15:14] for one of the first one is development
[15:17] and testing so you can imagine that as a
[15:20] developer you are able to look for
[15:23] different large language models and
[15:25] switch them to test them out to see
[15:28] which one is is going to perform better
[15:30] depending on what you want done so
[15:33] that's huge right so this is a deal of
[15:36] course for developers who are looking
[15:38] for ways to test applications that
[15:40] integrate large language models without
[15:43] having to set up different environments
[15:46] and all that complexity so also for
[15:48] education and resource what that means
[15:50] is that makes it really easy as a
[15:53] platform for Learning and
[15:55] experimentation without the barriers of
[15:57] Entry that are asso ated with cloud
[16:00] services so it's really easy just
[16:02] download certain kinds of large language
[16:05] models and you can go ahead and do
[16:07] testing and do some research and so
[16:09] forth and we talked about this secure
[16:12] applications the beauty of having AMA is
[16:15] that it provides this managed or this
[16:18] management platform of all of these
[16:21] different large language models which
[16:23] means it's suitable for Industries like
[16:25] healthc care or Finance where data
[16:28] privacy is very critical because
[16:31] everything is your own you download the
[16:34] large language models on your own
[16:36] machines so in conclusion AMA addresses
[16:39] these challenges of accessibility
[16:42] privacy and of course as we talked about
[16:44] cost in the realm of large language
[16:48] models so by enabling local execution
[16:51] which is exactly what we do with AMA it
[16:54] democratizes the use of advanced AI
[16:57] Technologies make making them more
[17:00] accessible and practical for a wide
[17:02] range of applications which is exactly
[17:05] what you would want okay so now it's
[17:07] time for us to look into setting up AMA
[17:10] locally um and what we'll do we're going
[17:12] to do go through the installation
[17:14] process and do the setup before we do
[17:17] setup there's some things that I need to
[17:18] let you know most of you should be okay
[17:20] there are some system requirements we
[17:22] need to be aware of first of all uh AMA
[17:26] supports Mac Linux and windows
[17:29] and I'm sure other operating systems
[17:31] there as well but these are the main
[17:33] ones and also make sure that we have at
[17:36] least close to 10 GB of free storage on
[17:39] your machine this is very important
[17:41] because as you will see some of these
[17:42] models require a lot of space and also
[17:47] processor as long as you have a modern
[17:49] CPU processor you should be fine and
[17:51] most of you should have that in place
[17:54] okay so just something to keep in mind
[17:56] all right let's go ahead and get started
[17:58] with
[17:59] setting installing and setting up AMA so
[18:03] if you go to ama.com O lam.com as such
[18:08] this is what you are going to be able to
[18:10] see says here get up and running with
[18:12] large L models now because this is a
[18:15] moving Target in technology things may
[18:18] change a little bit or dramatically by
[18:21] the time you watch this video hopefully
[18:23] not but the most important thing here to
[18:25] remember is that the IDE is still the
[18:27] same you will be able to download this
[18:29] tool and use that way independent on how
[18:33] it looks okay so should look the same
[18:36] but you never know and you can see here
[18:38] at the recording of this video says here
[18:41] Lama 3.2 this is the latest Lama 3.2
[18:44] model that they provide we need to
[18:46] download AMA how do we do this well it's
[18:49] very simple all you do click here
[18:51] download Once you click that this will
[18:53] take you to download AMA page now you
[18:56] can see we have three different flavors
[18:59] we have Mac OS so the browser was able
[19:01] to pick up that I am on Mac and so
[19:05] offered Mac if you're on Linux of course
[19:07] you'll have to click and look at this so
[19:10] you'll have to install by taking this
[19:12] command and running on your terminal if
[19:15] your windows of course is also just like
[19:17] mac you have to download as an
[19:19] application and run it okay whichever
[19:21] way you have to download the AMA zip
[19:25] file so I'm on Mac I'm going to click
[19:27] download
[19:29] and then you can see that it is
[19:30] downloading here so I'm going to wait
[19:32] for a
[19:33] second all right so it was downloaded
[19:36] and let's go ahead and click and open
[19:40] real quick so this is what you should
[19:42] see a zip file at least on Mac all right
[19:45] so I'm going to go ahead and double
[19:46] click and it's going to and zip and then
[19:49] it's going to look like this as an
[19:51] application okay so what I will do I
[19:53] just double click again and it will
[19:55] install again as an application on my
[19:58] back so I'm going to say open that's
[20:00] fine and then I'm going to move to
[20:03] applications and there we go so once
[20:05] we've done that I'm going to show you
[20:07] real quick
[20:08] here this is what I see so I have now
[20:13] this window on Mac that allows us to go
[20:15] through the process I'm going to say
[20:17] next and install the command line I'm
[20:20] going to say go ahead and say install
[20:22] and I'm going to pass here my
[20:24] credentials say okay so run your first
[20:27] model so we're going to run this AMA run
[20:30] Lama 3.2 now again this number here may
[20:33] be different from when you're watching
[20:35] this doesn't matter just follow the
[20:37] instructions so I'm going to say finish
[20:38] so what happens now is going to go ahead
[20:40] in the back and run the command and
[20:43] everything is good so it's going to
[20:45] install the Llama 3.2 which is a model
[20:49] that's a large language model that's
[20:51] going to be installed with our
[20:53] installation so we can get started and
[20:55] you can see once that happens that the
[20:57] Llama is running because now now we have
[20:59] this icon at the top that is showing
[21:02] there this is Lama icon and you can quit
[21:04] from here as well okay next what we can
[21:06] do is let's go back to models this page
[21:09] this is where all the different models
[21:12] that ama has this is where they are
[21:15] aggregated and we can look into them and
[21:17] learn about them you can see at the top
[21:20] we have Lama 3.2 which is the latest one
[21:22] which is the one that was installed
[21:25] already locally in our machine well come
[21:29] to this later and see more now let's go
[21:31] ahead and click on this one so I can
[21:32] show you something real quick here okay
[21:35] the beauty here is that when you click
[21:37] on any of these large language models
[21:39] you will see essentially the same things
[21:42] okay we have these 3B we'll talk about
[21:46] this later and um 1B latest and so forth
[21:51] okay and then most importantly we also
[21:54] have all of these things that we don't
[21:57] care about at this point we have a
[21:59] readme we have the sizes it tell us
[22:01] exactly what they mean what they good at
[22:04] this 3B default for instance and how to
[22:07] run it so this is essentially what you
[22:09] have at the top here so we're going to
[22:12] cop this and we're going to open a
[22:14] command line or a terminal I have one
[22:16] open and I'm going to paste that so what
[22:18] this does well it say Lama run Lama 3.2
[22:21] so what we saying here we're going to
[22:23] use a llama as the tool the framework
[22:25] right to run llama 3.2 which
[22:29] incidentally it's already installed on
[22:31] our machine right I'm going to just
[22:33] enter and what we'll do is going to spin
[22:36] up the Lama
[22:37] 3.2 model for us and then open this
[22:42] shell where we can start interacting
[22:44] with the model just like that now I need
[22:47] you to think to go back to what we
[22:50] talked about the beauty or the
[22:52] advantages of using ol is that ama has
[22:55] different models that we can use and
[22:58] install them internally locally on our
[23:00] machine and we can easily then through
[23:03] AMA as this manager of this Model start
[23:06] interacting with different models right
[23:09] now we are interacting we're able to
[23:11] interacting through the shell here with
[23:13] Lama 3.2 model which means I can ask
[23:16] question
[23:17] like what how
[23:20] old is the
[23:22] universe look at that and it gives me an
[23:25] answer the thing to keep in mind is that
[23:28] Lama 3.2 tends to be a little bit verose
[23:31] and as you can see uh it is quite veros
[23:34] but that's okay I can say in short tell
[23:37] me how old is the universe so you can
[23:41] actually direct it to tell you exactly
[23:43] what you need right kind of kind of like
[23:46] prompting it a little bit so the
[23:47] universe is approximately 13.8 billion
[23:50] years all right so I can also say for
[23:54] instance clear see what's going to
[23:56] happen glad you could convey that
[23:57] information clearly
[23:59] I guess I have to say something like
[24:01] this clear it cleared the session okay
[24:05] now I can continue here by saying for
[24:08] instance for help I can say forward
[24:09] slash I can put the exclamation point or
[24:14] I can say help like this so this will
[24:16] give me it's actually exactly other
[24:20] commands that I can use we have the set
[24:22] show load save and clear session as we
[24:26] saw as well as Buy to exit the shell so
[24:30] let's go ahead and use this show here
[24:33] okay I'm going to say forward slash show
[24:35] what we want what is that I want to show
[24:37] I want to show info so this will give us
[24:40] information about the model that we are
[24:42] looking at which is the Lama 3.2 says
[24:46] here architecture is llama the
[24:49] parameters is 3.2b we'll talk about that
[24:51] and the context length is that and
[24:54] embedding length and so forth and some
[24:57] other licensing information so just like
[25:00] that ladies and gentlemen we're able to
[25:02] install AMA and of course because AMA
[25:06] itself is just a manager we need to have
[25:09] the actual large language model and
[25:11] that's easy to do by installing in this
[25:13] case llama 3.2 we could have installed
[25:15] any other model of course but we start
[25:18] with that one when we're installing
[25:21] AMA all right so now I can say another
[25:25] question tell me a short
[25:29] joke okay here's one why couldn't the
[25:32] bicycle stand up by itself because it
[25:34] was too tired very funny all right and
[25:38] if I want to get out of this shell I can
[25:39] say for a slash buy like this and you
[25:43] can see we're no longer in our Lama 3
[25:46] shell so I can clear and we're done okay
[25:49] very good so I hope this is a good
[25:51] introduction there is a lot that coming
[25:54] um and I hope this excites you because
[25:56] the power that we have right now is at
[25:58] all this is locally we've imported we
[26:01] have downloaded the AMA tool in this
[26:04] case manager along with one model which
[26:08] is Lama 3.2 okay make sure that you're
[26:11] able to do this and I'll see you next
[26:13] now to keep in mind that the whole idea
[26:16] of a Lama is that we have this place
[26:18] where we have many different models we
[26:20] can use interchangeably we can just pick
[26:23] one and then drop it and pick another
[26:25] one and test until we find something
[26:27] that works
[26:28] so it is important to always look at
[26:31] these
[26:31] models page here so we can look at
[26:34] what's available and most importantly as
[26:37] you can see here you can also go ahead
[26:39] and search or actually filter by
[26:41] featured most popular and uh or newest
[26:46] and so forth okay so newest here seems
[26:49] to be neotron so keep in mind again that
[26:53] these will change depending on when
[26:55] you're watching this video okay so maybe
[26:57] you're not going to see this as the
[26:59] newest but the important thing is this
[27:01] is the place for you to come and look at
[27:04] what's going on so as far as we're
[27:06] concerned right now updated 3 weeks ago
[27:09] we have the Lama
[27:10] 3.2 it is also important for us to have
[27:13] more information about those large
[27:15] language models because the idea is for
[27:17] us to be able to choose what works for
[27:20] what we trying to accomplish okay so
[27:22] let's click on Lama 3.2 so when you
[27:25] click on Lama 3.2 or on any large
[27:28] language model that is shown here you're
[27:30] taken to this description page now
[27:33] there's a lot going on here the
[27:34] important part here is that we
[27:36] understand at least the basics of what's
[27:38] Happening first of all we see that is
[27:40] under Tools 1B 3B which we'll talk about
[27:43] but also we see that it was pull 1.1
[27:47] million times okay which means this is
[27:50] essentially being used by many many many
[27:53] people and then here we have these tags
[27:56] six to three tags but then we have here
[27:59] this drop- down list that tells us
[28:02] different things first one we have
[28:04] different flavors of this Lama 3.2 we
[28:07] have this 3B which is the latest that's
[28:10] the one we have and then if we want a
[28:12] shorter smaller one is 1B one important
[28:15] to keep in mind also is that depending
[28:18] on the flavor that we get here right you
[28:21] notice that also we have the sizes so
[28:26] depending on the flavor that we have
[28:29] here we can have different sizes so this
[28:31] one 3B is 3 2.0 gigabytes okay so it's
[28:35] not too bad but 1B is 13 GB and latest
[28:39] of course is 2 GB which is this one
[28:41] latest same thing okay and we've seen
[28:44] this before we have this command that
[28:46] we're going to use to run in this case
[28:49] this particular
[28:51] model now if you go down here we have a
[28:54] read me tells a little bit more about
[28:56] this model important for us to read and
[29:00] then we have the sizes so 3B parameters
[29:03] which is
[29:04] default this says here the 3B model
[29:07] outperforms the Jamma 2 2.6 B and the 5
[29:11] 3.5 Min models on tasks such as
[29:14] following instructions summarization
[29:17] prompt rewriting tool use but what does
[29:20] this really mean so I want you to look
[29:22] at this as a guide because again we go
[29:25] back to the whole point of Ama is that
[29:28] we have different models that we can use
[29:32] and us as developers or as people who
[29:35] are wanting to use these models is that
[29:39] we need to find something that works for
[29:40] us this is why it's important for you to
[29:43] always think about testing different
[29:45] models until you find something that
[29:47] works for you what works for me in a
[29:50] situation X Y and Z it's not obviously
[29:53] going to be what's going to work for you
[29:55] in your own situation so that is
[29:56] something to keep in mind
[29:58] always all right so it tells us here the
[30:02] comparison that for 3B parameter this
[30:05] size here it's really good at following
[30:07] instructions summarization prompt
[30:09] rewriting and Tool use so use tools or
[30:13] function calling and so forth now when
[30:15] it comes to 1B parameter this is a
[30:17] smaller version of 3.2 it says here the
[30:21] model is compatitive with other 1
[30:23] through 3B parameter models its use
[30:26] cases include personal information
[30:28] management multilingual knowledge
[30:30] retrieval rewriting tasks running
[30:33] locally on edge okay so if you want to
[30:37] pull Lama 3.2 1B then you would run this
[30:42] so you can imagine if you want something
[30:44] that will be good for this use case
[30:48] obviously you will use Lama 3.2
[30:50] 1B and let's go ahead I'm going to show
[30:52] you how to do this so we can run or Lama
[30:55] run Lama 3.2 to pull down to get this
[30:59] Lama 3.2 1B so I'm going to copy
[31:03] this go back to our terminal just paste
[31:06] that
[31:08] actually I should copy the whole thing
[31:11] and paste it so run Lama 32 1B so this
[31:16] will download this particular subset of
[31:19] model this model so hit enter you can
[31:22] see it's pulling down manifest it's
[31:24] pulling everything in what may take a
[31:26] little bit depending on on this size
[31:28] remember we're actually downloading
[31:31] locally so that means it will take up
[31:33] some
[31:35] space farewell so now you can see that
[31:38] we have installed Lama 3 1B so if I were
[31:42] to go ahead and say for slash again show
[31:46] info you can see voila it's now we're
[31:50] now using architecture Lama of course
[31:52] the same thing parameters is 1 2 B which
[31:55] is exactly different than the
[31:57] full-fledged Lama 3.2 so it's a little
[32:00] smaller okay and context length and all
[32:03] that information so we can go and ask
[32:06] question how old is the
[32:09] universe like this and it's going to go
[32:11] ahead and do the same thing but because
[32:14] we
[32:14] know from Reading here we know that this
[32:18] particular one here is really good at
[32:22] writing tasks running locally on edge
[32:25] multilingual knowledge retrievals so I
[32:28] can ask things like as an example how do
[32:32] you say hello I am
[32:37] fine in Tai let's
[32:41] see okay so it was really good this is
[32:44] just a very simplistic uh test here so
[32:48] again you can see the things you can do
[32:49] you can pull in different large language
[32:52] models to see what's going on now the
[32:54] other thing I can do here I can say I'm
[32:57] going to just say buy just to get out of
[32:59] here and still here I can say
[33:03] AMA and I can say list so this command
[33:07] here is going to list all of the models
[33:10] that have been download downloaded
[33:12] locally look at this now you can see we
[33:14] have this Lama 3.2 1B this is the latest
[33:18] one okay it says exactly when this was
[33:21] downloaded and of course I have a few
[33:23] others that I've done loaded a while ago
[33:25] so L 32 latest time ago I have this
[33:29] embedding large as well which we'll talk
[33:31] about later very good so here it shows
[33:34] everything that has been downloaded in
[33:36] terms of a list of all of the
[33:40] models let's look at parameters and do a
[33:44] quick Deep dive uh so that we understand
[33:46] a little bit more about the information
[33:49] that comes with models so we're going to
[33:51] learn what are parameters as well as
[33:53] what is that they really mean right so
[33:56] when we say show info that command we
[33:59] saw this we have this model architecture
[34:03] llama parameters of course
[34:05] 3.2b context length embedding length
[34:08] quantization and so forth so what do all
[34:11] of these mean let's start with the first
[34:14] one so the first one is in this these
[34:17] llama architecture here first of all the
[34:19] architecture side of thing that means it
[34:21] was architected it's a llama large
[34:23] language model meta AI that means they
[34:26] are the ones who created it matter
[34:28] Facebook and also that means that these
[34:30] models were designed with one thing in
[34:33] mind which is efficiency they were
[34:35] designed to be extremely efficient which
[34:38] means they are strong when it comes to
[34:41] performing even at smaller scales
[34:44] compared of course to other large models
[34:47] out there so they're very efficient and
[34:50] perform really well now when we look at
[34:53] then the parameters side of things it
[34:54] says here 3.2b and we saw that sometimes
[34:58] it fluctuates to 1.2 or3 or 1 and so
[35:02] forth different numbers these are the
[35:05] internal weights and biases that the
[35:08] model learns during training but what
[35:11] they do really is that they determined
[35:14] essentially how the model processes
[35:16] input data and how it generates output
[35:19] because that's the whole point okay now
[35:22] when we say
[35:24] 32b for instance what does that mean
[35:26] well that stands for 3.2 billion
[35:30] parameters now you can imagine even if
[35:32] you don't understand exactly what those
[35:34] means that means if you have 3.2 billion
[35:37] parameters in this case these weights
[35:40] and biases everything neurons and nodes
[35:42] and everything that means you have a lot
[35:45] inside of this neural neural network
[35:48] which means there's a lot of information
[35:51] that can be passed around the
[35:52] relationships and so forth which in
[35:54] terms mean that this model the the more
[35:58] billions or the more parameters it has
[36:02] which means has more connections and
[36:03] more interactivity inside that means the
[36:07] more accurate it is when it comes to
[36:10] getting the results now it's important
[36:13] to understand also that the number of
[36:16] parameters does reflect the complexity
[36:19] and the capacity of the model as
[36:21] explained but there's also what we call
[36:23] a tradeoff which means that more
[36:26] parameters can improve performance of
[36:29] course but it require more computational
[36:33] resources so if you have a large
[36:36] language model that is 3B or 5B or 7B or
[36:40] 8B well that means it's going to be
[36:43] amazing in terms of performance but also
[36:46] means it is going to require a large
[36:48] amount of computational resources so
[36:51] when we have something like
[36:53] 32b this kind of strikes the balance
[36:56] between per performance and resource
[36:59] consumption so let's summarize this
[37:01] simply when we talk about parameters
[37:03] talk about 3B or 2B or 10p or 7p and so
[37:07] forth these are numbers inside a neural
[37:11] network that it adjusts to learn how to
[37:14] turn inputs into correct outputs so the
[37:18] largest number the more the better in
[37:20] this case but we have drawbacks in that
[37:23] you also need uh a lot of computational
[37:26] resources to run it so next we have the
[37:30] context length so this number refers to
[37:33] the maximum number of tokens which
[37:35] essentially are pieces of text that the
[37:38] large language model can ingest in a
[37:41] single input so when we have something
[37:44] like
[37:45] 131,072 this is an exceptionally long
[37:49] context length what that means is that
[37:52] it can handle very long documents
[37:55] capturing dependencies across large
[37:57] large spans of text which is really good
[37:59] that means you can take big large books
[38:03] and article lengthy articles and
[38:05] extensive conversations and it can deal
[38:08] with it with no problem when we talk
[38:10] about the embedding length we're talking
[38:12] about the size of the vector
[38:14] representation for each token in the
[38:17] input text so when we say 372 what we're
[38:22] saying is that we have
[38:24] 372 dimensions in the embedding space so
[38:28] the larger number again the more
[38:30] Dimensions we have which means the more
[38:31] relations you're going to have in this
[38:33] Vector space so what does that mean well
[38:36] that means that we have what we call the
[38:37] semantic richness which means higher
[38:40] dimensional embeddings can obviously
[38:42] capture more nuanced meanings and
[38:45] relationships between words when it
[38:48] comes to a large language model then
[38:49] that means this will reflect the model's
[38:52] ability to understand very complex
[38:55] language patterns so the large ler we
[38:57] have this High dimensional the more the
[39:00] model is going to understand complex
[39:04] language patterns now there are some
[39:06] implications again is that we have this
[39:09] competition load so larger embeddings
[39:11] increases of course the compettition
[39:14] requirements and also if we have that
[39:17] that's going to improve the model's
[39:18] ability to generate contextually
[39:21] relevant and coherent
[39:23] responses okay and the last one we have
[39:26] here is quantization now there's books
[39:28] and we can spend hours and hours talking
[39:31] about this but in general quantization
[39:33] essentially is a technique used to
[39:35] reduce the size of a neural network
[39:38] model by in this case reducing the
[39:40] Precision of its weights this number
[39:43] indicates that the model's weights are
[39:46] quantized or
[39:47] quantized to four bits so translating
[39:51] essentially we just saying we now have a
[39:54] smaller model and faster processing
[39:57] and lower memory usage so now we have a
[40:01] more efficient um
[40:03] model so going back to our Lama 3.2
[40:06] model page here to see more information
[40:09] now you understand what 3B means 1B
[40:11] means and the implications and so forth
[40:14] and one thing you will realize also is
[40:16] that at the bottom we have what we call
[40:18] benchmarks now I don't really trust a
[40:20] lot of these benchmarks because anybody
[40:22] can inflate B the benchmarks or deflate
[40:24] them to follow certain agenda but
[40:28] nonetheless it's nice to look at and see
[40:30] The Benchmark but one thing I want to
[40:32] show you here is the implication of the
[40:34] sizes the parameters if I go back
[40:37] perhaps and let's go back to models here
[40:39] real quick and let's look for something
[40:42] different let's look for Lama 3.1 you
[40:44] can see with Lama 3.1 you even have 8B
[40:49] 70b
[40:51] 45b let's click on
[40:53] 405b well let's look at this one you can
[40:57] see that if we go to
[40:59] 405b that's 229 GB of space you have to
[41:03] have in your hard dis that's a lot and
[41:07] even if you're able to have that
[41:09] internally locally that that means you
[41:12] also have to have the computational
[41:15] capacity for you to be able to run these
[41:17] models so this is something to keep in
[41:19] mind you could have the capacity of
[41:21] having 229 gabyt locally but will you
[41:25] have the capacity of the computational
[41:28] resources that are needed to run this
[41:31] now keep in mind as you learn now this
[41:33] is probably one of the best models
[41:36] versions right because it has everything
[41:39] it's large billions and billions of of
[41:42] nodes and neurons and
[41:43] everything that's something to uh keep
[41:46] in mind and so for things that most of
[41:48] us are able to do want to do 8B even 70b
[41:52] should be totally fine okay just
[41:55] something to keep in mind and as you go
[41:58] through each one of these different
[41:59] models they have different things okay
[42:02] you can see of course we have more
[42:04] information here 45b 7B 8B and so forth
[42:07] as we talked about and they tell us the
[42:09] evaluation and what they're good at when
[42:12] should you use it and all these
[42:14] different informations as well as human
[42:16] evaluation as you see here so this is
[42:18] something to always go ahead and read
[42:20] and perhaps go to uh some links here to
[42:23] read more about it now I want you to
[42:25] always have this inquisitive mindset
[42:27] whenever you're looking for models so
[42:29] that you know exactly what you're
[42:31] getting um and contract that with what
[42:34] you want to accomplish that's how you
[42:36] need to uh go by uh when it comes to
[42:39] using AMA and different models here it's
[42:42] not just one model fits all that's not
[42:46] the idea the idea is that you find
[42:47] something that will work for you okay so
[42:51] models are real fun and you can always
[42:53] um go ahead and test them out now now
[42:57] let's go ahead and run through some
[42:59] commands uh with our Lama that way we
[43:02] are well situated and we understand how
[43:04] to use some of the most important
[43:06] commands Okay Okay so let's go ahead and
[43:08] get started so we know that we always
[43:10] start with AMA we can say AMA list this
[43:13] will list all of the installed models
[43:18] internally locally so you can see we
[43:20] have
[43:21] 32b 32 callon 1B here it's the size and
[43:27] when it was modified and we have other
[43:30] ones that I have here in your case you
[43:31] should probably most likely just have
[43:34] one let's say I want to delete or remove
[43:36] a model I can say
[43:38] AMA remove also I can say AMA help which
[43:42] is going to give me some of the commands
[43:45] that I can use we have serve have create
[43:47] Show run stop pull push list we've seen
[43:51] before PS and RM for remove so let's
[43:57] start with I'm going to go back say AMA
[44:00] list to show the list of all of the
[44:02] models that we have here let's say I
[44:04] want to delete the this one here so I'm
[44:07] going to copy that and I'm going to say
[44:09] clear AMA RM to remove and pass in the
[44:15] model name hit enter and it's going to
[44:17] delete that model now if I go ahead and
[44:19] say AMA list again we should see that we
[44:22] no longer have that model very good and
[44:25] if you want to delete this
[44:27] other one here you can also do like that
[44:30] now if we want to go ahead and pull in a
[44:32] new run we can say
[44:35] AMA and I'm going to say pull and the
[44:38] name of the model as we've seen before
[44:41] so again we can go back to our models
[44:46] website all right AMA Library models and
[44:49] let's say we want to get Jamma okay so I
[44:52] go back here you can see models all
[44:55] these models let's say I want to get
[44:58] something different let's say we want
[45:00] this code Gemma if you click here you
[45:03] can see that we have different flavors
[45:06] let's say I want to be the smaller so
[45:09] you can see we can run this one here I
[45:11] can either run all of this or I can just
[45:13] pull first this one and then run it okay
[45:16] so two ways to do the same thing so I
[45:19] can say AMA pull like that it's going to
[45:21] go ahead and
[45:23] pull our model now remember this is 1.
[45:27] gigabytes and may take a little
[45:29] bit okay so once that is done I can say
[45:34] AMA list we should see that we have code
[45:37] Jamma so I can now run it right so let's
[45:42] go ahead say all Lama run coama so it's
[45:45] going to now run coama so we can
[45:48] interact with it go back here I can say
[45:50] that Co Jamma is a collection of
[45:51] powerful lightweight models that can
[45:53] perform a variety of coding tasks like
[45:56] fill in the middle code completion code
[45:58] generation so forth so let's see we can
[46:01] test it out right can see if this works
[46:03] so I can say for instance write me a
[46:06] python
[46:08] function that returns hello
[46:13] world okay there we go you can see
[46:16] Define function your name return hello
[46:19] world your name and there we go all
[46:21] right very good so you can see that
[46:23] we're able to of course download a new
[46:27] model and test it out so what I can do
[46:30] again I can just say for slash buy to
[46:33] get out of here and then I can say
[46:37] AMA
[46:39] list to see that and I can copy this and
[46:43] remove it so I'm going to say
[46:46] AMA RM to remove that and it's
[46:50] deleted and
[46:52] AMA list again you can see that we no
[46:55] longer have Jemma
[46:58] okay so back to our models here I can go
[47:00] ahead and look at models and imagine
[47:02] that we want to find a model that deals
[47:05] with images or that is able to read an
[47:09] image and tell us what's going on
[47:11] there's a model called
[47:14] lava so lava is a novel endtoend trained
[47:17] large
[47:19] multimodel that combines a vision
[47:21] encoder and funa for general purpose
[47:24] Visual and language understanding so
[47:26] this is is a good candidate for what we
[47:29] call a
[47:30] multimodel model which means is that
[47:32] it's able to deal with images as well as
[47:35] text and so forth and other types okay
[47:37] so we have just this one and then we
[47:39] have this lava Lama 3 so depending on
[47:42] the situation we can read more to figure
[47:44] out which one is the one we should
[47:46] probably look into so let's click on
[47:48] this
[47:49] lava and you can see we have 7B 13B and
[47:53] 34b so you know which one we're going to
[47:55] be using so I'm going to go ahead head
[47:57] and get at least the
[47:59] 7B which is this one here latest or 7B
[48:02] doesn't matter it's about 4.7 gigabytes
[48:05] okay so let's go ahead and pull this one
[48:07] in I'm going to copy that and let's go
[48:08] ahead and just paste it all in and get
[48:12] our lava
[48:13] 7B all right so after a little while you
[48:16] can see that we downloaded a few things
[48:18] here but most importantly we have this
[48:20] 4.1 gab so we have now our model is
[48:23] running and you can see we have our
[48:25] shell I'm going to just say what is your
[48:29] name see what's going to say I don't
[48:31] have a name of course it's just a large
[48:33] language model okay so now let's see
[48:36] what we can do one thing I'm going to do
[48:37] actually is I'm going to move over to a
[48:41] code editor vs code to make this simple
[48:44] okay so I'm just going to say byy for
[48:46] now and then I'm going to open up here
[48:49] to see vs code so I have this
[48:53] flower1 PNG we're going to use this with
[48:56] of course our lava model to see if it
[49:00] can read this image and tell us what
[49:02] it's all about okay so let me go ahead
[49:05] and open the terminal here real quick
[49:07] and so what I'm going to do first let's
[49:08] go ahead and say AMA I'm going to
[49:11] run lava
[49:15] 7B okay now it's running very good so
[49:18] what I'm going to do now is I'm going to
[49:20] say what is in this image
[49:27] and I'm just going to
[49:29] pass the image name which is
[49:32] flower one. PNG let's see what's going
[49:35] to
[49:38] happen so now it's going to think and
[49:40] tell us exactly what it sees the image
[49:43] shows a bunch of small purple white
[49:45] flowers with yellow centers which
[49:47] appears to be a type of pansy they're
[49:50] growing in what seems like soil or a
[49:53] potting medium surrounded by what looks
[49:56] like green foliage so that's pretty good
[49:59] right for if you go back here and look
[50:02] yeah looks exactly that's exactly what
[50:05] it's describing which is really really
[50:06] awesome now let's test something here
[50:09] I'm going to test to see if this model
[50:12] has this history capability meaning it's
[50:15] able to save the conversation for
[50:18] context I'm going to say and
[50:21] now write me a
[50:24] short poem
[50:27] about the about that let's see what's
[50:29] going to
[50:30] happen ah there we go so it was able to
[50:33] look look at that a purple and white
[50:35] with yellow Hearts Define they add A
[50:38] Touch of Beauty to the world around
[50:40] little patch and so forth so it knows
[50:43] now we know if we're testing that this
[50:46] model is able to save the context the
[50:50] history of the things that we've been
[50:53] asking uh the context of our
[50:55] conversation
[50:57] tell me more
[50:59] about those flowers let's see what it's
[51:03] going to
[51:05] say okay there we
[51:07] go it knows exactly what flowers we're
[51:09] talking about which is what we had here
[51:13] okay I can ask can you tell me where
[51:16] they grow
[51:21] best and keep in mind again this is all
[51:25] local large language model that we
[51:27] running we don't have to go and send
[51:29] request to a network somewhere it's all
[51:33] local very good so pansies are native to
[51:36] the Mediterranean region and all that
[51:38] great stuff very good so we see that it
[51:42] is indeed working so we're able to pull
[51:45] in a new large language model in this
[51:47] case lava which is a multimodel model
[51:51] which allows us to deal with text as
[51:53] well with images and so forth and we ask
[51:55] it to tell us what it sees right in this
[52:00] image here and it was able to give us
[52:03] the description of the image and we ask
[52:06] some follow-up questions about that
[52:09] image and it
[52:11] works so I've changed to
[52:15] AMA run Lama 3.2 model right now so I
[52:20] can do it again you can see since it's
[52:22] loaded then we can see that we are able
[52:24] to go to the
[52:27] shell so what I want to be able to do is
[52:30] show you that we can also do many many
[52:32] different things here right so because
[52:34] we are running on a large language model
[52:37] it's a model that does a lot of things I
[52:39] can perhaps say I have a sentence that
[52:41] I'm not sure the sentiment of that
[52:44] sentence right I can say tell me what is
[52:47] the
[52:48] sentiment of the
[52:51] following
[52:55] sentence I
[52:57] not willing to pay you back let's see
[53:02] what's going to happen okay again it's
[53:05] very verbose which is okay but
[53:07] essentially says that the sentiment of
[53:09] the sentence I am not willing to pay you
[53:11] back is negative and confrontational it
[53:14] implies that the speaker is unwilling or
[53:17] unable to repay a debt all right so this
[53:19] is really good now one thing we can do I
[53:21] can also say can you please be less
[53:27] verbos okay there we go so the beauty
[53:31] here is that you're talking or you're
[53:32] interacting with a large language model
[53:34] and you can actually prompt it to do
[53:37] what you want it to do in this case I
[53:40] told it to always be prompt to always be
[53:44] less verbos at least in this case and
[53:46] you can see says the sentiment of I am
[53:48] not willing to pay your back is negative
[53:50] and confrontational it implies
[53:52] resentment defensiveness and obligation
[53:55] aversion okay now the beauty here is
[53:57] also we can modify or in this case
[54:01] customize our model because we can
[54:03] actually add certain pieces of metadata
[54:06] to the model that cater to what we want
[54:09] it to be like what I mean by that is
[54:12] that we can actually create a file here
[54:15] right click I'm going to click here and
[54:17] this is what we call a model file so it
[54:20] has to be something like this model file
[54:23] no extension whatsoever and inside of
[54:26] this file file this is where we are
[54:27] going to add a few lines of text code
[54:31] essentially so at the top here I'm going
[54:33] to say all caps say from because we are
[54:38] modifying it from a different from a
[54:41] base model it's going to be from Lama
[54:46] 3.2 in this case if you have something
[54:48] different you add that and then here is
[54:51] where we're going to set a few things so
[54:53] we can set a lot of things but now I'm
[54:54] going to show you how to set in this
[54:57] case the temperature what is a
[54:59] temperature temperature is what allows
[55:01] the model to be more creative or more
[55:03] direct and matter of fact so the higher
[55:06] the number in this case from 0 to one
[55:09] the higher the number closer to one the
[55:11] more elaborate the more um creative the
[55:16] model is going to be okay so I can
[55:18] change to 0 point three for instance if
[55:21] I want it to be less
[55:23] creative and then I can add for instance
[55:27] the system message so this is just the
[55:28] prompt right that it needs to know of so
[55:33] in this case here put that into let's
[55:36] say triple quotes like that to say
[55:39] inside here we can put text a very smart
[55:44] assistant who answers questions
[55:49] succintly and
[55:51] informatively
[55:53] okay something like this so this
[55:56] is our model file which is going to
[56:00] allow us to modify or customize our
[56:03] model okay so we can add more parameters
[56:06] here more things to Aid our model but
[56:10] this is okay for now so how do we do
[56:12] this now to make it so that our model
[56:14] indeed will comply with what we put
[56:17] together here in this model file well
[56:20] it's very simple all we have to do now
[56:23] I'm going to let's go say byy real quick
[56:26] here
[56:27] all we have to do now is we need to use
[56:29] the create command from AMA to create
[56:32] some sort of a new version of our model
[56:36] which is going to be specified by this
[56:39] model file very easy really so I'm going
[56:42] to say AMA and then say create and I can
[56:45] give it a name just
[56:47] say James okay whatever you want and
[56:50] then we're going to say f to say to the
[56:53] file it's going to be under the model
[56:56] file notice the model file is actually
[56:58] the same level here so I can just go
[57:00] ahead and that so if I hit enter what's
[57:03] going to happen now let's say enter okay
[57:05] so says success you can see that it went
[57:07] ahead and transferred model data so
[57:10] using the layers and everything in the
[57:12] background and so now if I save for
[57:15] instance list you can see that we have
[57:17] this James so it really created a
[57:20] replica of the main
[57:23] model and now this is going to be
[57:26] different from the Lama 3 3.2 right
[57:30] because it will have a little bit more
[57:32] of a modified customized piece to it
[57:35] very good so now I can use Mario or
[57:38] James I should
[57:39] say to ask
[57:42] questions so look now I can say
[57:46] orama first
[57:48] run James look at that and we should run
[57:52] James how beautiful is that so look what
[57:55] will happen I can say
[57:56] what is
[57:58] your name look what's going to happen my
[58:01] name is James and look how suc it is it
[58:05] goes straight to tell to answer what
[58:08] needs to be answered are you smart let's
[58:11] see I designed to process and provide
[58:14] accurate information quickly and
[58:15] efficiently making me highly
[58:17] knowledgeable on a wide range of topics
[58:19] and all of that comes from this because
[58:23] we said that your assistant that was
[58:25] very smart
[58:26] ask answer questions suly and
[58:29] informatively very good very good you
[58:31] the power things you can do right so now
[58:34] I can ask questions let's see let's say
[58:36] tell me about all tell me about the
[58:42] oceans let's
[58:45] see okay so this is pretty good because
[58:48] it's such an open ended question it went
[58:51] ahead there are five oceans gives me all
[58:53] that information oceans play a crucial
[58:55] role in regulating the earth's climate
[58:58] weather patterns and ecosystems they
[59:00] also provide all of this information
[59:02] here okay so it is to the point that's
[59:04] pretty good this was just an example to
[59:07] show you the things you can do you can
[59:09] actually modify customize your model to
[59:15] be and do certain things that you wanted
[59:17] to do this of course was very very
[59:19] simplistic but you can see for the
[59:21] system here we can prompt it even better
[59:24] with more complex information and change
[59:27] temperature and all these different
[59:29] things let's go ahead and just say byy
[59:32] and because we're done let's go ahead
[59:34] and get rid of it so that we save some
[59:37] space so llama
[59:41] RM James okay so Lama list we should not
[59:47] see James and while I'm here let me go
[59:49] ahead and actually get rid of some of
[59:50] these as well so that we have more space
[59:59] okay very good so far we've been using
[60:02] the CLI the command line interface with
[60:06] AMA which is attached with AMA that
[60:08] governs everything in a back end but we
[60:10] can also use the rest API in fact most
[60:14] of the things that we're going to be
[60:16] using later are based on the rest API
[60:19] which means there is an endo that we can
[60:22] hit and run in this case the models
[60:26] so how does that work one thing to keep
[60:29] in mind is that in the back end as you
[60:31] see well you can't quite see here but at
[60:34] the top my bar here I do have this well
[60:37] you probably see this quid AMA but there
[60:39] is this AMA that is running in the back
[60:41] end so that means in the background it's
[60:43] running which means we're able to do all
[60:45] sort of things that we're doing right
[60:47] the all Lama application per se it's
[60:49] running but it's actually being served
[60:52] at an endpoint locally and all of that
[60:55] is being served at Local Host
[60:58] 11434 what does that mean well that
[61:01] means then we can generate a response
[61:03] using the olama rest API because it has
[61:06] it attached to it and it's running in
[61:09] the back end in the
[61:11] background okay so to do so it's very
[61:14] simple all you have to do you have
[61:16] access to all of this we can curl
[61:18] something like this so in this case you
[61:20] can see it's a curl and this is the end
[61:22] point so Local Host 1143 for API gen
[61:28] generate and then dasd and then we pass
[61:31] the payload here so here we're saying
[61:33] the model is going to be Lama 3.2 right
[61:37] and the prompt we passing along it why
[61:38] is the sky blue so if I were to run this
[61:41] or hit enter what will happen is it's
[61:43] going to hit that end point and of
[61:45] course it's going to show me this now
[61:48] this is not very helpful mainly because
[61:51] it just shows a lot of gibberish and
[61:53] there is this stream that's happening
[61:55] but if you look look closely you can see
[61:57] that every time this run extremely it's
[62:00] going to show a certain word in response
[62:04] so combining all these words will be the
[62:07] response so that means then when we
[62:10] write the actual rest API payload we
[62:14] need to also add something else to it to
[62:17] tell it that we don't want it to stream
[62:19] as it is right now so that we just get
[62:22] result what that means now I can let me
[62:27] go back down let's clear so now what we
[62:31] need to pass is this here that says
[62:34] stream false so say we don't want it to
[62:36] stream just give us the response if you
[62:39] hit enter you can see it will take a few
[62:41] moments and then voila we have this
[62:43] model everything all the metadata about
[62:47] the model that we're using in this case
[62:49] is say tell me a fun fact about Portugal
[62:52] and then says here here is one response
[62:56] did you know that the town of obid obid
[62:59] in Portugal is often called the fairy
[63:02] tale Town due to its medieval
[63:04] architecture and picturesque streets
[63:07] right it gives it some other information
[63:10] all right so that is a difference here
[63:11] if we put stream false then we get that
[63:14] if this is not there it assumes that
[63:17] stream is true that's why we get what do
[63:19] you saw earlier now in this case you can
[63:21] see the end point is generate now we can
[63:24] also get the chat with the model the
[63:27] chat end point okay so generate the
[63:30] difference here generate it just goes
[63:32] ahead and predicts what needs to happen
[63:34] as you see just gives us result but we
[63:36] can also pass in the chat endpoint so
[63:39] let me go and clear to do so I have that
[63:42] you will have access to all of this
[63:44] anyway so no worries and there we go we
[63:46] do the same thing curl now the end point
[63:49] here is API for SL chat and then DD and
[63:53] then we pass here the payload so so we
[63:56] pass the model the messages messages
[63:59] plural that's why we have here a list
[64:02] okay so we pass the role it's going to
[64:04] be user and the content is going to say
[64:06] tell me a fun fact about mosm Beek let's
[64:09] go ahead and run this so there we go and
[64:12] now this is a different endpoint it
[64:15] gives us here some here's one fun fact
[64:18] about gorang gorza Peninsula which has
[64:20] one of the largest Coral in the world um
[64:23] I think this is actually not correct
[64:26] they have to remember that sometimes
[64:29] these large lar models are not always
[64:31] correct so there's no such thing as
[64:33] going a peninsula I know this because
[64:35] I'm from there but anyway you can see
[64:37] that it's giving us some results large
[64:39] ler models and are always correct so as
[64:42] always we can also pass more information
[64:45] or more specification to our endpoint in
[64:49] our payload so imagine that we want to
[64:51] for instance we want to request a Json
[64:55] mode we can do so by requesting it by
[64:58] adding Json right so you can see here we
[65:02] are getting the generated API and we
[65:06] have the same thing model prompt now
[65:09] here we even said in our prompt make
[65:12] sure that we want to get the response as
[65:16] Json but also we want to say here as one
[65:20] of the parameters that format it has to
[65:22] be Json to make sure that indeed the
[65:25] large language model aders to this okay
[65:27] let's go ahead and
[65:28] run and you can see doesn't look that
[65:31] great but we want to believe that this
[65:32] is actually a Json okay that is coming
[65:35] in so you can see the curly Bryce is or
[65:38] opening their model and then llama 3
[65:41] created and all that information and we
[65:43] should have an actual Json but you can
[65:45] see response Day morning Sky color and
[65:50] so forth the beauty here is that if you
[65:52] go to this link here gith habo Lama docs
[65:56] API end points or so many other end
[65:58] points we can go ahead and hit um using
[66:02] the rest API okay you can always come
[66:05] here and look at some of them we can do
[66:07] essentially the same thing we were able
[66:09] to do in our s ey that's the whole point
[66:11] we can do that using the rest API for
[66:14] instance we can copy a model show model
[66:17] information you've seen that before okay
[66:19] you can just curl and then do that so
[66:21] I'm not going to do that you can do that
[66:22] yourself and then this is the result
[66:24] that you get so all these things that
[66:26] we've done before you're going to be
[66:27] able to do them as well using the
[66:30] endpoint API the rest API endpoints
[66:33] we've learned a lot so far there are a
[66:35] lot of things that we've learned about
[66:37] the CLI we learn how to use the CLI of
[66:40] course which is attached to orama to do
[66:43] all sort of things to pull in different
[66:46] models to run them to remove them to
[66:50] even modify the model or in this case
[66:52] customize the model by creating the
[66:54] model file file and then use that to
[66:59] create some sort of a copy or a modified
[67:02] version customized version of that model
[67:04] which is very handy as you see for use
[67:07] cases that you may have okay so a lot
[67:10] that I've thrown at you I hope this is
[67:12] making sense I hope you are practicing
[67:14] and seeing the power and most
[67:16] importantly here remember is that all of
[67:18] that is local you're not having to do
[67:21] anything uh passing something to the
[67:23] network to get anything back in that way
[67:27] which means it's free you're just using
[67:29] your own resources which is the power of
[67:31] AMA so as a quick full summary here AMA
[67:35] as we know is a platform that allows you
[67:37] to run large language models locally
[67:41] which is really awesome the great thing
[67:43] is that it supports various models
[67:45] tailored for different tasks including
[67:47] text generation code generation and
[67:50] multimodel applications so essentially
[67:53] AMA model support these tasks here text
[67:57] generation such as Lama 3 x could be 3.2
[68:00] 3.10 or 3.11 depends Moll and so many
[68:04] others uh we have also code generation
[68:07] one of the example was code Lama okay
[68:10] and multi model application in this case
[68:13] text and images and we saw in this case
[68:15] was lava the lava model now this is just
[68:19] to show you the breath of things or
[68:22] models that ama provides Okay so again
[68:26] your job is really to figure out what
[68:29] models is going to give you the best
[68:32] results for your use case Okay so keep
[68:35] in mind that you always have to test
[68:37] figure out which one will work for you
[68:40] all right so to create large language
[68:42] model applications using AMA we saw that
[68:46] first of all uh we can use large
[68:49] language models that come with Alama
[68:51] because Alama hosts per se all these
[68:54] different large language models we can
[68:56] use now we also learned that there are
[68:59] different ways to interact with AMA and
[69:02] its models so we saw that the main way
[69:06] is through the CLI the command line
[69:09] interface so it's easy straightforward
[69:12] and that's what we've been looking into
[69:14] and then the second way would be in this
[69:16] case to use UI based interface which
[69:19] we're going to look into next so
[69:21] essentially have a user interface where
[69:24] the back and we can put or apply a model
[69:29] AMA model and then we have a nice user
[69:33] interface that users or yourself can use
[69:36] to interact with the
[69:38] model now we've just finished looking at
[69:41] the rest API so this is the base of
[69:44] essentially everything else we're going
[69:45] to be doing after the next UI based
[69:48] interface way of interacting with the
[69:50] models okay so essentially we're able to
[69:53] use rest API curl and then hit the end
[69:57] point and get information that way about
[70:00] generating text or um in this case
[70:04] deleting things deleting models or
[70:06] looking up everything related to our
[70:08] models so essentially the same things
[70:10] that we're able to do using the CLI we
[70:13] have a way of doing that using the API
[70:16] the rest API and later we'll see we're
[70:18] going to use AMA python library now this
[70:21] is where we're going to be able to then
[70:23] use the tools in Python in code because
[70:27] now we have more customization more
[70:29] freedom per se to start building the
[70:32] actual large language models
[70:34] applications that we so want to do okay
[70:38] okay so now let's go ahead and look at
[70:40] the UI based interface so if you go to m
[70:44] thata MST that app as you see here this
[70:48] is what you will see so you can read
[70:49] about it say Lama 3.2 Vapor mode all
[70:52] this great stuff say the easiest way way
[70:55] to use local and online AI models so
[70:59] without Misty painful setup endless
[71:01] configurations confusing UI Docker and
[71:04] all these other things with MTI one app
[71:06] one click setup no Docker no terminal
[71:09] offline and private unique and Powerful
[71:12] feature so that's the reason why I chose
[71:14] this of course you can choose any other
[71:16] that you might find out there that I
[71:18] mentioned so this is what I'm going to
[71:19] be using and you can see there's a lot
[71:21] of logos here saying that it works with
[71:24] uh meta stuff with chat open Ai and many
[71:27] many others so it's really good so you
[71:30] can see Windows Mac and Linux and they
[71:34] do have a user interface so you can see
[71:36] exactly how it will look like so
[71:38] essentially it's going to be like having
[71:40] chat GPT uh but now we are running our
[71:44] own models so click here and I am on Mac
[71:48] of course I'm going to go and go that
[71:50] way if you're on Windows you have the
[71:52] option to do the 64 or 64 CPU only MD or
[71:57] Nvidia if you're Mac you have apple
[71:59] silicon or Intel chip if you're Linux of
[72:02] course you have these flavors as well so
[72:04] pick the one that works best for you so
[72:06] I'm on Mac I'm going to go to Apple
[72:08] silicon like that so it's going to go
[72:11] ahead and
[72:12] download okay so it's now downloaded I'm
[72:15] go let's go ahead and open it and I'm
[72:17] going to double click to start the
[72:19] installation so I'm going to go ahead
[72:21] and drop it there very good and let me
[72:25] go ahead and open it real
[72:28] quick okay so I'm going to say open all
[72:31] right and there you have it now once you
[72:33] open it this should happen and the
[72:36] beauty here is that you can see how
[72:39] would you like to get started you can
[72:40] set up locally AI I remote modeles
[72:43] provided and so forth but if you look
[72:45] down here look at this Gama get started
[72:48] quickly using AMA models all right and
[72:51] even found it under users meama and all
[72:55] the models they found it locally because
[72:58] it detected that we have Ama installed
[73:01] in our in our uh on our machine so I'm
[73:04] going to go ahead and say continue down
[73:07] here okay that's it just one click and
[73:12] we are done and you can see here at the
[73:14] bottom here make see I can make this
[73:16] larger yeah I can make this larger
[73:18] that's very
[73:19] good okay at the bottom here you can
[73:21] click smaller you can click you can see
[73:24] we have
[73:25] different levels or different models we
[73:28] have lava 7B which you remember and I
[73:31] have Lama
[73:32] 3.2 all right because those are the ones
[73:35] that we have installed at least I have
[73:36] installed locally if you have more than
[73:39] one then you're going to see all of them
[73:40] sure here and this is beautiful because
[73:43] we didn't have to do anything just one
[73:45] click indeed like they promised and we
[73:47] are there so let me choose the Lama 3.2
[73:50] and the moment you do that that's it and
[73:52] you can just start chatting just like
[73:54] that I'm going to say how old are you
[73:58] just silly to see what's going to
[74:00] say so this is being powered Again by
[74:03] our own model so I was released to the
[74:07] public in 2023 very good can you tell me
[74:12] a funny short
[74:16] story and just like that look how fast
[74:18] this is one day a man walked into a
[74:20] library and asked the librarian do you
[74:22] have any books on puff loves dogs and
[74:25] scroller cats scroll danger I think
[74:28] that's what you say the libran replied
[74:30] it rings a bell but I'm not sure if it's
[74:32] hair or
[74:34] not okay that's very cool so we have
[74:37] that so you can start talking you can
[74:40] start chatting with this long large
[74:43] language model and if I wanted to I can
[74:45] also go and attach a document let's
[74:47] click here and I'm going to add one
[74:50] image that we've seen before drop it
[74:53] there
[74:55] now image is attached let me go back
[74:58] here and I can
[75:00] ask tell
[75:02] me about the image now here's the thing
[75:06] the reason why it's saying this is
[75:08] because the model doesn't CU we're using
[75:10] llama 3.2 it doesn't really know how to
[75:12] deal with the images however if I change
[75:14] here go to Lava 7 and ask the same
[75:17] question tell me about the
[75:21] image Aha and now because lava is a mul
[75:25] model multimodal model it will know how
[75:29] to answer questions about the image
[75:33] there we go now it's working the image
[75:34] shows two purple flowers with yellow
[75:37] centers okay very good so it is
[75:39] describing it took a little bit of doing
[75:41] here perhaps it should have reset
[75:42] everything but it's okay you can see
[75:45] that now it is actually working because
[75:47] we're using lava 7 and it will tell me
[75:50] exactly what it sees now I can change to
[75:53] Lama 2 or
[75:56] 3.1 and if I say tell me
[76:00] about the image it's going to give me
[76:02] some issues of
[76:04] course okay very good so it can't do
[76:07] that but the other thing I can do here I
[76:09] can also add let me just delete this I'm
[76:11] going to add a document let's say for
[76:14] instance I'm going to add this document
[76:17] that we've seen before which is the b o
[76:19] i PDF there we go I'm when I say
[76:25] can you tell
[76:28] me about the PDF file okay so now I have
[76:34] some issues here the best way to do this
[76:36] is you go to attach knowledge Stacks so
[76:39] click here and I'm going to say add your
[76:41] first knowledge stack so I'm going to
[76:43] say my
[76:45] tester and the thing is because we are
[76:49] now invoking want to create a rag system
[76:53] we need to have an embedding model so
[76:56] that it can create those embeddings of
[76:59] the PDF file of the document that way it
[77:02] can be saved and then the large language
[77:04] model is able to Converse that way okay
[77:06] so we're going to just use whatever they
[77:08] give us nomic uh embedding test text I'm
[77:13] going to add that it's very good and at
[77:16] this point let's go ahead and add an
[77:18] actual file so I'm going go ahead and
[77:21] drop it again in this case here just
[77:23] drop it like that all right
[77:25] so once you do that you have to hit
[77:27] compose so that it a it's able to pick
[77:30] it up do all the things it needs to do
[77:32] to actually get information from our
[77:35] file okay embeddings and everything say
[77:39] compose so it's
[77:41] composing okay knowledge stack saved and
[77:44] composed successfully you can now use it
[77:46] for chatting very good so let me get out
[77:51] of here and I'm going to go here so
[77:54] click again here and what I want to do
[77:56] here is I want to be able to say click
[78:00] on my tester but this is my knowledge
[78:03] base I want to include in my chat here
[78:05] so make sure you click that and that
[78:08] will what I will do is we'll let this
[78:11] know that indeed we have included that
[78:14] knowledge base which includes all the
[78:16] information about our PDF file so say
[78:20] Give me a summary of the PDF
[78:27] file okay let's
[78:30] uh says there's no PDF file mentioned in
[78:33] our conversation so far the text we
[78:34] discussed earlier was from beneficial
[78:36] ownership and everything so it took a
[78:38] little bit uh because I'm asking PDF it
[78:40] doesn't really know about the PDF
[78:42] because all of that information was
[78:43] actually transformed into text and
[78:47] embeddings and everything for the large
[78:49] language model so I just can say Give me
[78:51] a summary
[78:55] of the
[78:59] document okay there we go so it says the
[79:02] document provide instruction on how to
[79:03] fill out certain items including
[79:05] identifying documents issues a company's
[79:08] images and everything and it is indeed
[79:10] beneficial ownership all of that so
[79:12] we've seen this before I can keep asking
[79:15] what are
[79:17] the
[79:20] penalties for not
[79:23] filing and and there you go it gives me
[79:27] exactly the penalty and everything that
[79:29] will happen if I don't file right I can
[79:33] just say give me the
[79:36] penalties maybe that's a better question
[79:39] perhaps okay that's better because now
[79:42] it's giving me exactly what I need okay
[79:44] based on the text from beneficial
[79:45] ownership information blah blah blah and
[79:48] everything is there failure to comply
[79:50] with beneficial ownership reporting
[79:51] requirements Cil penalty 10,000 fine up
[79:55] to 500,000 and so forth
[79:59] okay what are
[80:02] the deadlines for
[80:11] filing okay so you can see that this is
[80:13] working we're able to converse with our
[80:17] data our own document now the beauty
[80:20] here if you AR haven't realized yet is
[80:23] that all of this again is internal it's
[80:25] our own large language models we can of
[80:28] course go and spin out a different large
[80:30] language model OKAY adding new provider
[80:33] and so forth and we can use
[80:35] interchangeably as you see here and we
[80:37] passed in a knowledge base which means
[80:40] we said here are some documents in this
[80:42] case which is one document and use that
[80:45] to create a knowledge base so a rag
[80:47] system essentially and so that we can
[80:48] chat and ask questions about that
[80:51] knowledge base okay and just like chat
[80:54] GPT we can go ahead and create a new
[80:55] chat we can do all sort of things as you
[80:57] see here the point is not for me to go
[80:59] through Misty thoroughly the point is to
[81:03] show you what's possible to create a
[81:05] user interface or to use a user
[81:07] interface that is guided or is being
[81:11] fueled by our own AMA model how cool is
[81:16] that so what I want you to do is to play
[81:18] around with this and you can see it's
[81:20] just to show you how amazing this is
[81:23] because we have our own box that we can
[81:26] uh pass in sensitive documents and all
[81:29] those things without worrying about
[81:32] prices you know having to call an API
[81:35] anything external put this in a cloud
[81:38] base or having to pay for usage so it's
[81:41] all here and we can use as we see
[81:44] fit so we're able to see all of the ways
[81:47] in which we can interact with and its
[81:50] models so we looked at CLI command line
[81:53] interface and we saw that it's really
[81:55] easy because it's the fastest access to
[81:58] our models it's very easy but as you
[82:01] know it's not scalable in sense that
[82:03] you're not going to build a full-fledged
[82:05] application using that and then we went
[82:08] and looked at the API the rest API which
[82:11] is essentially the same but now we have
[82:14] a different back door per se we we using
[82:16] rest API to pass in certain payloads and
[82:21] get information so we're doing the same
[82:22] thing that we can do with CLI hitting
[82:24] the same functions per se but now we're
[82:26] using different end points with the rest
[82:30] API and we just finished looking at the
[82:32] UI based interface so this is way easier
[82:35] for us to be able to have a UI based
[82:39] interface and actual interface such as
[82:42] chat DBT that way we're able to ask
[82:45] questions uh change manually quickly
[82:49] different models that we may have and
[82:52] start chatting with our models so it's
[82:54] really really easy to put uh together or
[82:56] to have that working using the M that
[83:01] app and of course there are different
[83:02] flavor different tools out there that
[83:05] you can use that will do the same thing
[83:07] right that would give you this UI based
[83:10] interface so I chose MTI because it's
[83:13] just easy as you saw to install and get
[83:16] started so it wasn't about how to use
[83:18] these tools but I was about to give you
[83:20] the knowledge and tools that you can
[83:23] then uh use
[83:25] on your own and explore more okay so now
[83:29] we're going to go to the fun part which
[83:31] is we're going to use now sort of a a
[83:34] backend combination of the API rest API
[83:37] through the python Library the AMA
[83:40] python library because truth be told we
[83:43] want to be able to create local large
[83:46] language model applications using AMA
[83:49] models and so for that we need a way for
[83:53] us to be able to use python this code in
[83:55] this case code or any other language but
[83:57] in this case going to be python to be
[83:59] able to take advantage of these models
[84:02] that ama provides and so that's what
[84:04] we're going to be doing next which is
[84:06] we're going to get started with AMA
[84:08] python Library so we can use code to
[84:11] interact with AMA
[84:14] Models All right so I have this folder
[84:17] here we've seen before called AMA we got
[84:20] a few things here let's get started here
[84:22] simply by using the AMA library but
[84:26] actually I'm going to do the hard way
[84:30] first to show you that we can actually
[84:31] use code to get to the end point just
[84:34] like what we saw with the restful API so
[84:38] real quick so we can see what's going on
[84:40] so let me go ahead and create a new file
[84:41] here so we're going to have access to
[84:43] all of these code so no worry so I'm
[84:45] going to say start one. P1 or py
[84:49] okay and so what we'll do here first
[84:52] actually is I'm going to go ahead and
[84:54] make sure that we have an actual virtual
[84:58] environment for our python project so
[85:01] I'm going to say Python
[85:03] 3 okay like that so we should have a
[85:07] virtual environment there that's very
[85:09] good and let's go ahead and activate it
[85:12] so Source bnv if you're on Windows of
[85:16] course there's a different way to do
[85:18] this activate and voila so now we have
[85:21] our virtual environment AC activate
[85:25] active I should
[85:26] say very good so what is the first thing
[85:29] we're going to do well first of all I'm
[85:30] going to go and import let's say pip
[85:33] install I'm going to install
[85:36] requests real fast here so we have
[85:39] that okay very good because we're going
[85:41] to use that to do what we are looking
[85:44] for it to do all right so I'm going to
[85:47] go ahead and import requests so we have
[85:51] that and also I'm going to import Json
[85:53] so we have that as well okay so I'm
[85:56] going to create a URL here which is
[85:58] going to be the URL where is the end
[86:00] point it's going to be HTTP for our
[86:04] Local Host here so it's not going to be
[86:05] that it is 1 4
[86:10] 34 and in this case I want to go to the
[86:14] generate endpoint like this
[86:18] okay and for that remember we need to
[86:20] pass some payload data when we pass
[86:22] along so I'm going to say data just
[86:24] create a little uh dictionary
[86:27] here and for a dictionary I'm going to
[86:30] pass the model that we need it's not
[86:32] going to be gpt3 or
[86:34] gpt2 we're going to call this llama it's
[86:37] going to be
[86:38] Lama 3. two and make sure that llama is
[86:43] running for this to work
[86:45] always and then I'm going to pass the
[86:48] [Music]
[86:49] prompt I'm going to pass something that
[86:52] says tell me
[86:55] a short story and make it funny like
[87:01] that okay so there we go this is our
[87:04] payload that we're going to pass along
[87:07] using the rest API so I'm going to send
[87:10] the request so I'm going to say
[87:12] response pass in the request. post pass
[87:15] in url and Json and what I want to do
[87:18] here
[87:20] is I want to say stream
[87:24] to true to say I want this to be
[87:27] streamed okay as you will see in a
[87:29] little bit okay maybe this a little bit
[87:32] smaller so you can see everything all
[87:34] right okay so then let's go ahead and
[87:36] check the response first so I'm going to
[87:38] say check the
[87:41] response status so it's going to say if
[87:44] response status is as you see there I
[87:47] already have all these code so I'm not
[87:49] going to Bor you with all of the
[87:51] intricacies
[87:54] I'm going to put it all there okay so
[87:55] what we're doing now is that we're going
[87:57] to check if response code is 200 which
[87:59] means all is good and then we're going
[88:01] to go ahead and start generating thing
[88:02] so we're going to go and iterate over
[88:05] the streaming response because we said
[88:07] the streaming to true as it comes the
[88:10] response that comes in okay and so what
[88:12] we're doing here we're decoding the line
[88:14] and parsing everything until we actually
[88:17] print the generated text as we go
[88:19] through very good so let's go ahead and
[88:21] see if this works I'm going to go ahead
[88:22] and run this make sure that of course
[88:25] the
[88:27] AMA icon is running is out there
[88:30] otherwise this is not going to work so
[88:32] let's go ahead and run let's run this
[88:35] looks like I'm having
[88:36] issues right so I'm need to pass API
[88:40] like that okay my bad that was my
[88:44] problem let's go ahead and give it a try
[88:50] again okay and there we go you can see
[88:52] that is indeed working and it's really
[88:55] really fast as you see here generat text
[88:58] here's a short silly story for you once
[89:01] upon a time there was a chicken blah
[89:03] blah blah and there you have it all
[89:06] right very good so this is indeed
[89:08] working we're able to use code to
[89:12] actually interact with the our llama 3.2
[89:17] model locally right albe it we are not
[89:20] using the AMA uh python Library but you
[89:24] can see that this is actually working go
[89:27] ahead and play around with this and I'll
[89:29] see you next all right so this was
[89:31] really nice we're able to use the rest
[89:34] API in code here the restful API should
[89:38] sayama restful API in code to interact
[89:42] with Lama 3.2 this is fine but now I'm
[89:45] going to show you how to use the AMA
[89:48] Library the AMA python Library so we can
[89:53] go straight into code we don't have to
[89:54] hit these we don't have to explicitly
[89:58] hit these end
[90:00] points all right let me go ahead and
[90:02] create a new file here let's call this
[90:06] start
[90:07] to.py remember you will have access to
[90:10] all this code so do not worry okay so
[90:14] the first thing we need to do here of
[90:15] course is to make sure that we import or
[90:20] we get the right dependency so going to
[90:24] say pip install we need to install or
[90:28] Lama as such very
[90:31] simple and there we go now once we have
[90:35] a Lama we can start working at right so
[90:40] first let's go ahead and import
[90:42] AMA there we go and real fast here I'm
[90:46] going to show you how simple this is so
[90:49] the same thing we did before with the
[90:51] CLI and even with the rest API we can do
[90:55] the same thing here using
[90:57] this dependency right using this SDK
[91:01] perite so now I can say put inside of
[91:04] response and I'm going to say all Lama
[91:07] and I can say list so this function here
[91:10] as in name imply is going to list list
[91:14] it should list all of
[91:17] the you guess it all of the things that
[91:19] we have in this case all of the models
[91:21] so let's go ahead and save and give it a
[91:23] quick run run so real
[91:25] quick okay I ran and you can see that we
[91:28] have this Json that comes up it says
[91:33] llama 7 oh name lava 7 and that is the
[91:37] model and I should have gives all that
[91:40] information and I should also have the
[91:44] Lama
[91:45] 3.2 and some other things all right this
[91:49] is very cool very good all right
[91:55] very good so you can imagine we can say
[91:58] AMA list we can also say
[92:01] AMA let's say chat chat response client
[92:05] create and delete embed and all these
[92:08] other different things right so that's
[92:10] very very exciting I just want to show
[92:13] you that I'm going to just call comment
[92:14] that out so we don't have a lot of
[92:16] things here all right so let's go ahead
[92:19] and hit the chat endpoint or in this
[92:22] case the chat API
[92:25] well as you know it's very simple so
[92:28] let's go ahead and get started here all
[92:31] right so I'm going to say res we can put
[92:34] anywhere orama chat
[92:38] so. chat and then I'm going to pass a
[92:41] few things and you can see that if you
[92:43] have over it says it respects at least
[92:46] to have the model name so the model I'm
[92:49] going to say llama
[92:56] 3.2 okay just add the one that you have
[93:00] and then here because it's chat I'm
[93:01] going to pass messages okay so this
[93:04] comes as a list and I'm going to pass
[93:07] the role as a user and the content is
[93:09] going to say something I can say
[93:11] something different that perhaps why is
[93:14] this sky
[93:18] blue something like this okay so now I
[93:22] can actually print
[93:24] what comes back in this case from our
[93:27] chat we pass the model and the messages
[93:30] could be more than one message okay we
[93:32] can also say uh for our role here
[93:36] another one that says let me just put
[93:38] like that I can say the context pass the
[93:41] context pass all different things that I
[93:44] may want to pass okay but so far this is
[93:47] okay let's go ahead and see how this
[93:49] works I'm going to go ahead and just run
[93:51] from
[93:52] here okay to a little bit and you can
[93:54] see we have the results so you can see
[93:57] the model and everything but most
[93:58] importantly we have the content says
[94:00] content is the sky appears and
[94:02] everything but to make it even better I
[94:05] can just go ahead and get the content so
[94:07] let's go to
[94:09] result uh let's say I think I have to go
[94:12] to
[94:13] messages and they get the content like
[94:17] that
[94:20] okay let's go ahead and run again all
[94:23] not messages
[94:26] message
[94:32] okay all right and there you go so this
[94:36] guy appears blue because of phenomenom
[94:39] because of phenomenon called scattering
[94:41] and all of that great stuff so we have
[94:43] all the information here now you noticed
[94:45] before when we
[94:47] just look at showing the entire payload
[94:52] it comes in with a lot information about
[94:54] the model when it was created of course
[94:56] the content the role and all that
[94:59] information at the bottom we have some
[95:02] other pieces of data metadata so done
[95:05] true total duration how long it took and
[95:08] low duration prompt evaluation and uh
[95:12] also evaluation count prompt evaluation
[95:15] duration all these little
[95:18] things that we may want to keep track
[95:21] of all right so just like that ladies
[95:24] and gentle ladies and gentlemen we're
[95:26] able to use the chat endpoint here using
[95:30] uh the ol Lama python Library so go
[95:34] ahead and play around with this and see
[95:36] what we can do because this is the
[95:37] beginning of a very exciting thing I
[95:42] think now one thing you can also do you
[95:44] should know this is that we can pass
[95:46] other things in our chat function here
[95:50] right see we have model messages tool
[95:54] stream so we can say we want this to be
[95:56] streamed and format as well very cool so
[95:59] what you can do I'm going to give
[96:00] another example here for streaming let
[96:03] me go ahead and comment this out so we
[96:05] have that for streaming it's essentially
[96:08] the same thing notice I passing messages
[96:10] we're all user now I have a different
[96:12] question why is the ocean so
[96:15] salty and I can say stream to true
[96:19] because I want it to be streamed when I
[96:21] do that then whenever I want to print
[96:23] that out I have to put that through a
[96:25] for Loop or a loop of some sort that way
[96:28] we can get all these pieces that are
[96:31] being streamed so we can see them being
[96:33] flashed out in our command line okay so
[96:36] let's go ahead and run real quick so you
[96:38] can
[96:41] see just run like
[96:46] this takes a second and there you have
[96:50] it okay so there we go if you want to
[96:53] have the this capability of streaming
[96:55] and showing those messages as they come
[96:58] in this is how you would do using Code
[97:02] all right so go ahead and play around
[97:04] perhap add different messages or
[97:06] different questions here and see how
[97:09] this behaves and what you can also do
[97:11] you can use a different model whichever
[97:13] model you may have running
[97:16] okay so one thing to keep in mind is
[97:19] that all we've seen now this AMA python
[97:22] Library West DEC I guess I could call
[97:24] that it's all based or designed around
[97:28] the AMA rest API so internally when we
[97:32] call orama that chat internally we're
[97:35] essentially doing what we did before I
[97:37] showed you here so it's hitting this
[97:39] endpoint URL and passing in the API in
[97:43] this case go to chat or generate or
[97:45] create and so forth it's just like an AB
[97:48] it's just an abstracted way of hitting
[97:51] this end point so we don't have to
[97:53] explicit L do this so I made sure to put
[97:56] it here so that you have that okay now
[97:59] we can hit now we can use another now we
[98:02] can also use the same now we can go and
[98:06] use a different function or end point
[98:08] for instance like generate if want to
[98:10] generate something right we can do that
[98:12] what I mean by that is I can say for
[98:15] instance AMA generate and pass that and
[98:18] pass that prompt I can even pass all of
[98:21] that I don't think I can okay I don't
[98:23] need to pass any of that so we can also
[98:27] just generate a response instead of
[98:29] chatting as such so nothing new here I'm
[98:32] just going to leave it as is so you can
[98:34] see the other thing we can do is we can
[98:37] say for
[98:39] instance we can
[98:42] show by saying AMA let's just print and
[98:47] say
[98:48] AMA and then that
[98:51] show and we want to show what want to
[98:53] show anything about
[98:56] llama three like that that two okay
[99:00] let's go ahead and
[99:02] see and there you go after a few moments
[99:04] you can see it tells everything we need
[99:07] to know about our model OKAY a lot of
[99:10] information that's for sure okay there
[99:12] we
[99:15] go next thing we can do is we can
[99:18] actually use the model file as we did
[99:20] before but all in code to create a
[99:24] separate model that has Specific
[99:27] Instructions so it's very simple I have
[99:31] some parts of the code here so we create
[99:33] in this case the model file we don't
[99:36] have the actual model file but we can
[99:38] put all in line is here as you see here
[99:41] and the other thing I can do is I can
[99:43] pass the temperature here let's go back
[99:47] to this one to see how you do that just
[99:50] copy this and pass temperature 0.
[99:53] 1 for instance and then all we need to
[99:56] do now that we have this information is
[99:59] as we saw before we are going to go
[100:01] ahead and create the actual model
[100:07] so and then we're going to pass the new
[100:09] name know it
[100:11] all and then pass the model file which
[100:15] is this one here so we can make this
[100:17] more robust if we want for instance we
[100:19] can say you are very very suce and
[100:25] informative something like that all
[100:27] right okay so now that we have created
[100:30] our new know it all so we can go ahead
[100:33] and generate something right use the
[100:36] generate message or generate function so
[100:40] I'm going to put into a variable so
[100:44] AMA generate and I'm going to pass the
[100:47] model that I'm going to be using yes
[100:49] going to be know it all this one here
[100:51] same name and The Prompt there so I can
[100:55] just say for instance sure that's fine
[100:59] okay and we can go ahead and print the
[101:06] result need to go into response that's
[101:10] how it comes
[101:12] as okay and there we have it so let's go
[101:15] ahead and give it a run and
[101:18] see so it's generating creating
[101:21] everything and some sometime soon we
[101:24] should have there we go the ocean
[101:25] salinity is primarily due to the Earth's
[101:28] water
[101:30] cycle very good it looks great and it
[101:34] works so now if I come here and just
[101:39] say AMA list we should see that we do
[101:43] have this new know-it-all latest which
[101:46] is two gabes was created about 20
[101:49] seconds
[101:50] ago all right very good so these are are
[101:53] the things that we've done before using
[101:55] the CLI as well as using the rest API
[101:59] but now we're using
[102:01] the AMA we're using the AMA python
[102:05] library osdk to do the same thing
[102:07] because I just want to show you what's
[102:09] possible so this is more flexible as you
[102:11] see because now we actually use encode
[102:13] and the power of the rest API in the
[102:16] back end to create very simple
[102:18] applications of course this is just to
[102:20] show you what's possible and soon enough
[102:22] we're going to be able to create those
[102:23] large language model applications that
[102:25] are based on locally using AMA
[102:30] models so once we done that we can of
[102:33] course
[102:35] delete models okay so can say all delete
[102:40] know it all and if I run this it's going
[102:43] to go ahead and run again and go ahead
[102:45] and delete our friend know it
[102:49] all okay so I have the response again
[102:53] but most
[102:54] importantly if I go back and list we
[102:58] should see that we no longer have no at
[103:00] all okay so go ahead and play around
[103:02] with this and uh I'll see you later okay
[103:06] so now that you have all the tools you
[103:08] understand what Lama is what problem
[103:10] comes to solve and how to actually build
[103:14] a simple uh useful applications using
[103:17] AMA AI based applications using AMA now
[103:20] it's time for us to dive in and start
[103:22] build building real world use cases
[103:26] using and let's go ahead and get started
[103:29] all right so now it's time for us to
[103:30] start building something useful things
[103:33] that are actually useful so the first
[103:35] thing we're going to do here is we're
[103:37] going to build a grocery list organizer
[103:41] okay or categorizer so the idea is you
[103:44] have for instance a grocery list file
[103:46] text file that has all these different
[103:49] items okay and so what we want to do is
[103:51] we're going to use the line 3.2 the
[103:54] model with Allama of course and see if
[103:57] we can recreate this entire grocer list
[104:01] and actually create a new grocery list
[104:03] from this but categorized into
[104:07] categories so for apples and all these
[104:09] things could be produce and then we have
[104:11] snacks and so forth okay that's what
[104:14] we're going to do so you will have
[104:16] access to all this code of course but
[104:17] you notice that I now I have this data
[104:20] folder here which will have this grocery
[104:22] list that here
[104:24] with all of these items and so let's go
[104:26] ahead and create a new file here called
[104:30] categorizer
[104:32] py and this is where we're going to
[104:34] start playing around first thing going
[104:37] ahead and import
[104:40] AMA and then we're going to import OS
[104:42] because we are going to interact with
[104:45] the directories and finding things and
[104:48] at the top we're going to create I'm
[104:49] going to just add this model so we can
[104:52] use it so it's going to be
[104:53] be llama
[104:56] 3.2 okay so let's go ahead and set up
[104:58] the paths and I already have them set up
[105:01] so let me go ahead and get that so
[105:04] essentially for input path it's going to
[105:06] be data grocery list so essentially I'm
[105:09] going to data get this grocery list and
[105:12] output is going to be data and it's
[105:14] going to be called categorize grocery
[105:16] list at txt so it's going to be under
[105:19] data again once all is done
[105:24] all right so the first thing is we're
[105:25] going to go ahead and
[105:28] check paths if the input file Exist by
[105:33] using OS exist and pass this input file
[105:37] path like that and I'm going to print
[105:39] input file if it's not found right and
[105:42] then we're going to get out of there and
[105:44] then we're going to read these ancored
[105:47] and then we're going to read the
[105:49] uncategorized grocery items from the
[105:52] input file
[105:53] to do so we just open the file through
[105:57] the input file the path and read as F
[106:01] and then we take all those items we read
[106:04] everything strip everything into our
[106:07] items so these items now should have all
[106:09] of the items from our grocery list text
[106:12] file and as we know to direct our large
[106:16] language model we need to pass a prompt
[106:18] so to say this is what you need to do
[106:21] this is how you're going to be doing it
[106:22] right right so kind of prompted to give
[106:24] it some directions so it knows not to ve
[106:27] off somewhere else so for that we need
[106:30] to have this prompt that I already
[106:31] created so there is the prompt so you
[106:35] are an assistant that categorizes and
[106:37] sorts grocery items so we specify
[106:39] exactly what this model does okay here's
[106:43] a list of groceries so items going to be
[106:45] all of the items that we received there
[106:48] please one categorize these items into
[106:51] appropriate categories such as produce
[106:53] Dairy meat Bakery beverage beverages Etc
[106:57] and two sort the items alphabetically
[107:00] within each category so in each category
[107:03] you put every every item into make sure
[107:06] you sort um alphabetically and three
[107:09] present the categorized list in a clear
[107:12] and organized manner using bullet points
[107:15] or numbering that's it so we can make
[107:17] this more complicated as we wish but
[107:20] this should be fine all right so now we
[107:23] have the prompt we
[107:25] have of course now we have the file we
[107:28] have all of the items inside of this
[107:29] variable here from the text file and we
[107:32] have the prompt okay we're passing in
[107:35] that information through the prompt so
[107:37] we can use that prompt then to send the
[107:40] prompt and get the response okay so now
[107:43] what we do here put inside of a try
[107:46] catch essentially so we are generating
[107:49] we're using AMA generate we pass in the
[107:52] model and the model remember is what we
[107:55] set up here
[107:57] okay and then we pass in the prompt
[107:59] remember the prompt will have the items
[108:03] and of course the prompt to make sure
[108:05] that the large language models knows
[108:07] what to do and then we have generated a
[108:09] response or text we get that information
[108:12] if we have something or we just get
[108:15] nothing okay return nothing okay so now
[108:19] we are writing the categorized list to
[108:21] the output file because we want to get
[108:23] that what I can do also here is I can
[108:26] print real quick here the generated text
[108:29] so which is going to be the actual
[108:31] response so we can do that here just for
[108:33] fun I can just
[108:36] say generated text or
[108:39] generated or let's say categorized
[108:45] list and so then we write into a file
[108:49] and that's it and we categorize grocery
[108:52] list has has been saved and we show the
[108:54] output file the path that's it all right
[108:58] okay that's that is pretty much it so
[109:00] now we're going to go ahead and run this
[109:02] so I'm going to open this different ways
[109:04] of running just run here or you can
[109:06] write the command for running but that's
[109:09] okay so let's go ahead and what's what's
[109:11] going to happen so things are happening
[109:13] in the background and voila you can see
[109:16] that we see categorize list has been
[109:19] saved data categorize list and there it
[109:22] is but also we wanted to show it here
[109:26] look at this categorize list after
[109:28] categorizing and sorting gral items I
[109:31] present to you the following list so
[109:33] produce apples all of these items that
[109:36] seems correct we have dairy cheese eggs
[109:39] milk and yogurt very nicely done meat
[109:41] Seafood chicken ground beef salmon
[109:44] shrimp and sausages Bakery bread flour
[109:48] pasta rice Pantry almonds and all of
[109:51] these other items beverages that sounds
[109:53] great Frozen ice cream snacks chips
[109:56] chocolate and everything and now if I go
[109:59] to see as I said we have here the
[110:02] categorized grocery list if I click it's
[110:05] going to be exactly the same where we
[110:06] saw printed out there but now it is
[110:09] indeed a txt file now this is very small
[110:14] and very simple but I hope it's showing
[110:17] you the things that you can do and keep
[110:18] in mind that all of this is locally we
[110:21] don't have to pay for chat DBT or open
[110:24] API or anything else this is something
[110:27] that is local to us and keep in mind
[110:29] we're using llama 3.2 but if you after
[110:33] doing some testing and realize that for
[110:35] whatever you trying to do there's
[110:38] another large language model on another
[110:40] a model that will fit with what you're
[110:43] trying to do then obviously it's easy to
[110:45] swap that large language model to use
[110:48] that that's the beauty because remember
[110:51] here I'm saying we're using Lama 3.1 but
[110:53] I can go ahead and pull another large
[110:56] language model another model to use and
[110:59] nothing will change all will change is
[111:01] changing this to a different model that
[111:04] is the beauty of a
[111:06] Lama all right so what I want you to do
[111:09] is perhaps to make this even better or
[111:11] maybe you have some ideas that you may
[111:14] want to implement here because you have
[111:16] the large language model and you have
[111:18] some data maybe you have a list of uh
[111:21] something or
[111:23] a text that has something you want to
[111:25] summarize it you want to see what is the
[111:27] sentiment of that text and so forth
[111:30] there's so many things that you can do
[111:32] all of that because we now have this
[111:35] capability through AMA of picking and
[111:38] choose whichever model that we want to
[111:41] use incredible all right play around
[111:44] with this and I'll see you next so now
[111:46] it's time for us to look at something a
[111:48] little bit more complex something that
[111:50] you will find in the real world we which
[111:52] is building rack systems with AMA so
[111:57] with AMA of course we can build more
[111:59] complex large language model
[112:01] applications as I have mentioned so rag
[112:05] what is this thing rag stands for
[112:08] retrieval augmented generation so
[112:10] essentially is a way for us to converse
[112:13] or to talk or to chat with our own
[112:16] documents our own data
[112:19] internally now the thing to keep in mind
[112:21] is that large language models are
[112:23] actually very limited in terms of
[112:25] knowledge so they're really bombarded
[112:27] with a lot of different information
[112:29] textual audio and everything that's how
[112:32] they were trained on but they don't know
[112:34] what they don't know meaning they only
[112:36] know what they were trained on and so it
[112:39] makes it really difficult for us to
[112:41] build applications that are catered or
[112:43] that are customized to our own needs so
[112:47] rag solves that problem because now
[112:50] instead of the large language models to
[112:52] know only what it knows we can actually
[112:55] use it because it's really good at
[112:57] predicting things at summarizing and
[113:00] kind of brainstorming and getting pieces
[113:02] and pieces and putting together and get
[113:05] a response we can use that to solve this
[113:08] problem by injecting quote unquote
[113:11] injecting our own documents so that is
[113:13] one problem that it solves but also
[113:15] we're solving the issue of hallucination
[113:18] because any large language models you've
[113:21] encountered it makes stuff up because it
[113:24] knows a lot but if you ask questions
[113:26] sometimes just doesn't know but it
[113:27] doesn't say doesn't know it just says
[113:30] things that makes no sense that may seem
[113:32] true but they're false so this is what
[113:34] we're call hallucination that's what rag
[113:36] comes to help large language models and
[113:38] in turn help us when building rag
[113:42] systems now let's go and look have a
[113:45] deep dive into rag I think this is
[113:47] important especially for those of us who
[113:49] don't really know what rag is or or knew
[113:52] new to rag even if you already know what
[113:54] rag is I will just go real fast so you
[113:58] understand what really happens an
[114:00] overview so the idea is very simple is
[114:02] that we have these documents right so
[114:05] PDF files text files whatever and those
[114:08] are actually passed through this
[114:10] processing of parsing and pre-processing
[114:13] them what that means is that all of
[114:15] those are chapped into small tiny chunks
[114:19] documents okay and those in turn are
[114:23] then put through an embedding model what
[114:26] that does is it takes this textual
[114:30] information these documents chopped up
[114:33] and now they're going to be converted
[114:35] into a vector space into Vector into a
[114:39] number representation of those chunked
[114:42] documents that's what embeddings mean
[114:45] essentially and once that happens the
[114:48] vectorization of that that's what
[114:50] happens right it's a different format
[114:52] now of this data and that is saved into
[114:56] a vector database okay so all of this is
[114:59] what we call indexing this whole process
[115:02] we call it indexing what happens then is
[115:04] that a user comes in with a question
[115:06] query so that query say how old is is
[115:11] the universe for instance is also taken
[115:13] in and because of this whole ecosystem
[115:16] now is into um is transformed into a
[115:20] vectorized system that also has to be
[115:23] embedded so the query is also embedded
[115:26] into a model uses embedding model to
[115:30] embed that to vectorize this query and
[115:32] so that is indeed what it is actually
[115:35] either passed onto the vector database
[115:38] or it's used for searching because now
[115:41] you can imagine if we are searching
[115:43] using text and this realm on this other
[115:46] side here it's a vector database or
[115:48] vector store is saving everything mostly
[115:52] in Vector we need to have this
[115:54] correspondence right between we have the
[115:57] question but it has to be transformed
[115:59] into the same format as what we have
[116:01] here so that way it's easier to do this
[116:05] searching and so what happens now is
[116:07] that these systems have what we call
[116:09] retrieval um capabilities so they
[116:13] retrieve the pieces that are
[116:15] needed for answering the question
[116:18] meaning if the query is about how old is
[116:21] universe is going to find all the pieces
[116:24] all the documents inside of the vector
[116:26] store that perhaps are close to that
[116:30] question right and packages are all up
[116:34] and that's what's passed along with the
[116:36] prompt right the documents these pieces
[116:39] that we
[116:41] retrieved we call them relevant
[116:43] documents and the query itself and that
[116:46] is indeed what is passed through the
[116:48] large language model the model so that
[116:51] the large language model model has this
[116:53] contextual idea Sense on how to answer
[116:57] that question so it generates an answer
[116:59] which then we get a response so in
[117:02] nutshell ladies and gentlemen that is
[117:04] what rag is and is very powerful for the
[117:07] reasons that I've just shown you here
[117:09] now what do rag systems need as you saw
[117:12] well they need a few things first of all
[117:14] they need a large language model because
[117:16] that is essentially what drives most of
[117:18] the things essentially because we get
[117:20] all that information the documents
[117:23] pieces of documents related documents
[117:26] for what we are trying to ask for and
[117:29] the prompt and everything that's what it
[117:31] takes and the large language model has
[117:33] so much knowledge but also has these uh
[117:36] reasoning capabilities to know how to
[117:39] package it all up and answer that
[117:41] question coherently and also we need
[117:44] this document Corpus so essentially it's
[117:45] the knowledge base what we passing
[117:47] through the document embeddings is very
[117:50] important for the reasons I saw there
[117:52] because you take these documents you
[117:55] break them down into embeddings small
[117:58] pieces of vectors and if we need
[118:01] somewhere to save those vectors these
[118:03] are not just normal databases this what
[118:06] we call Vector databases these are
[118:07] vector stores so there are many out
[118:09] there uh we have phas Pinacle uh pine
[118:12] cone and chroma DB and then we need to
[118:15] have this retrieval mechanism and most
[118:17] Vector databases have in them some
[118:20] retrieval mechanism to retrieve retrieve
[118:22] what needs to be retrieved to then be
[118:24] passed along Downstream when we want to
[118:27] build large language model applications
[118:30] we can Brute Force everything and do
[118:32] everything our own on our own but it's
[118:35] always nice to have tools that kind of
[118:38] help us and Abstract out all the
[118:41] complexities of building large language
[118:43] model applications and so that's when we
[118:45] use tools such as Lang chain so it's a
[118:48] tool that makes it really really easy to
[118:50] deal with large language models and
[118:52] build robust large language model
[118:55] applications Lang chain is a an amazing
[118:58] framework actually and there's a lot of
[119:00] things that it does but one of the
[119:01] things that it does well is that it's
[119:02] able to load and parse documents easily
[119:05] we can use classes and wrappers to split
[119:08] documents and chunk them up um generate
[119:11] embeddings and many many many other
[119:13] things okay so it provides this unified
[119:16] abstraction for working with large
[119:19] language models and building llm
[119:21] application
[119:22] so tools are always welcome okay so I
[119:25] thought probably would be a good idea to
[119:27] look into Vector store and embeddings as
[119:30] an overview now it says Here length
[119:32] chain but essentially this is just in
[119:35] general how things work when it comes to
[119:37] Vector store embeddings large language
[119:40] models how to build a rack system and
[119:42] how everything actually works in the
[119:43] back end okay so just relax and let's go
[119:47] through this okay all right so this is
[119:50] the idea the main idea really as we saw
[119:52] before is that there's got to be some
[119:53] way of loading documents so we need to
[119:56] load the documents that come in in
[119:58] various forms Sometimes they come in as
[120:01] URL PDF files text file databases and so
[120:04] forth all of that is pulled together
[120:07] through as documents okay this is
[120:09] document loading and then goes through
[120:11] the process of splitting those documents
[120:13] as we saw and then once these are split
[120:15] of course go through the process of
[120:17] embeddings and everything and that is
[120:18] put into storage right this is a vector
[120:21] space or factor store but it has to have
[120:24] some retrieval as we saw because this is
[120:27] a way of getting information related to
[120:30] the question the query that comes in
[120:33] okay so the retriever goes through the
[120:35] vector store finding things are related
[120:38] to the question and then get relevant
[120:42] splits relevant documents and that is
[120:44] all what's put together with the prompt
[120:46] and large language model and pieces of
[120:49] documents and then large language model
[120:51] is able ble to give us the answer the
[120:54] result that's what we just saw so dis
[120:57] spitting what happens we go through the
[120:59] process of splitting and the splitting
[121:03] documents or the chunks are then passed
[121:06] through the embedding model so embedding
[121:08] model embeds creates this factorized or
[121:12] this number system or number format
[121:16] representation of those texts or those
[121:19] pieces of information Okay so so it
[121:22] looks something like this literally the
[121:24] text is converted into vector embeddings
[121:27] and looks something like that okay so
[121:29] embedding Vector keeps the content and
[121:32] the meaning of the text
[121:34] intact now this text with so the ideas
[121:38] that DET Texs with similar content and
[121:40] meaning will have similar vectors and so
[121:43] if you look at this diagram here this di
[121:46] this diagram hopefully puts everything
[121:48] into perspective we said that text with
[121:51] simp similar content and meaning will
[121:53] have similar vectors now remember that
[121:56] in this Vector space High dimensional
[121:58] space there's a lot of things going on
[122:00] lot of high dimension right so this is
[122:03] very simplified so that means then if
[122:05] you look at cat and kitty in this Vector
[122:08] space in this High dimensional space you
[122:09] can see that they are really close in
[122:11] meaning because a cat and a kitty
[122:13] they're essentially the same thing but a
[122:15] cat and run not the same thing right and
[122:20] so that is the representation but if you
[122:21] look at the actual representation in the
[122:25] vector space vectors as vectors numbers
[122:29] you will see something similar like this
[122:30] because cat and kitty are pretty much
[122:32] the same in terms of the semantic
[122:36] demeaning they will have some sort of
[122:38] similar numbers at some somewhere okay
[122:42] that's STA and that is actually what is
[122:45] stored into a vector database these
[122:48] numbers these vectors okay those are
[122:51] embeded things now the great thing is
[122:53] that Vector databases can be used in
[122:56] various cases now you can use all of
[122:59] this in searching for instance If You're
[123:02] Building A system that wants to find
[123:04] relevant uh results to the query string
[123:08] recommendation systems for instance
[123:10] finding items with related text strings
[123:14] uh that are recommended and
[123:16] classification to classify strings uh by
[123:19] most relevant and similar labels okay so
[123:23] this is some these are some use cases
[123:25] that are used when we talk about Vector
[123:27] stores and Vector databases and vectors
[123:30] embeddings and so forth now the whole
[123:33] embedding process works as follows
[123:35] number one of course we have documents
[123:38] that are split into chunks of text we
[123:40] saw this and then those are transformed
[123:44] into embeddings which then are added or
[123:49] saved as embeddings into a vector
[123:52] database now the great thing here is
[123:54] that most Vector databases they not only
[123:57] save the embedding Vector they also
[123:59] embed or save the original text split so
[124:03] on one side we're going to have the
[124:04] actual embedding Vector as you see here
[124:06] and on other side the corresponding
[124:08] original text so that's how most Vector
[124:12] databases store things so this whole
[124:14] process is extremely important because
[124:17] when we then are ready to query the
[124:19] vector store for embeddings in this case
[124:22] this is what happens first thing is we
[124:24] have here as you see let's put it all
[124:28] together we have the question this is
[124:29] the indexing part of thing the question
[124:31] goes through the embed process because
[124:33] it has to be transformed into embedding
[124:35] so that the whole system understands
[124:37] right and then what happens is that we
[124:39] go through the process of indexing here
[124:41] so it knows essentially to look for what
[124:45] is going to be similar to this embed
[124:48] which is part of this question so search
[124:50] and compare
[124:52] all of the entries in the vector store
[124:54] and then it picks out the most similar
[124:56] entry and once those are picked those
[125:00] most similar entries those are the ones
[125:03] that are packaged up with everything
[125:05] through the large language model so the
[125:08] large language model what it will do is
[125:10] going to take the most similar entry
[125:12] found and pass them into the to with the
[125:15] query the question the pieces of
[125:17] documents and everything to then get the
[125:19] answer the result so that how it all
[125:22] works so I wanted to show you this so
[125:24] that you have an idea on how Vector
[125:27] store and the large language model how
[125:30] everything comes together and rack
[125:32] systems so now that we understand rack
[125:34] systems we understand how embeddings
[125:37] large language model how this whole
[125:39] thing comes together this is the simple
[125:42] rack system handson or at least the
[125:44] overview of what we are going to be
[125:46] putting together so the idea here is
[125:47] very simple is that we're going to use
[125:49] Lang chain to pass through the PDF file
[125:52] to load that and then use l chains
[125:56] character text splitter to split that
[125:58] into small
[125:59] chunks as we have discussed and then
[126:02] we're going to use AMA embeddings this
[126:04] is the important part AMA also has a
[126:07] model for embeddings that is going to
[126:10] allow us to embed all of this
[126:12] information and put that into a chroma
[126:14] DB database and then what will happen is
[126:17] that we're going to pass in a query and
[126:18] we're going to use again Lang chains
[126:20] multi-query retrieve
[126:22] that will be able to do all things that
[126:24] it needs to do to transform this query
[126:26] into embeddings of course and search the
[126:29] chroma database with all those
[126:31] embeddings and then pull out the uh most
[126:35] similar pieces of information or data or
[126:38] documents and then put it all together
[126:41] relevant docs query and prompt and pass
[126:44] that through a large language model
[126:45] which in this case again you know is
[126:47] going to be Lama 3.2 and of course
[126:50] generate the resp response that we so
[126:52] desperately need so this is a r system
[126:56] now using
[126:58] AMA embeddings and using Lama 3.2 which
[127:01] is part of AMA now key Point here is
[127:04] that for all these embeddings AMA
[127:07] embeddings models and this large L model
[127:10] we can swap these with whatever else
[127:12] that we might want to use and the whole
[127:15] system will still work again I'm
[127:17] repeating myself I know that is the
[127:19] beauty of AMA is that once we have the
[127:22] system set up we can swap the embeddings
[127:25] we can swap the large language model
[127:28] these models with whatever else we want
[127:30] and it will still work okay so this is
[127:32] what we're going to be putting together
[127:34] next all right so first things here I
[127:37] have this B oi PDF we've seen this
[127:40] before so it should have access to all
[127:42] of this anyway it's just these
[127:45] beneficial ownership information report
[127:47] that I found online and that's what
[127:49] we're going to be using for our RX
[127:51] system okay now one thing I've also
[127:54] added here is this requirements. txt so
[127:57] this has all of the requirements all of
[127:58] the things the dependencies that we need
[128:01] for all these thing for all these
[128:04] project that we're going to be working
[128:05] on so I kept it here so that we can run
[128:09] so Al that you have it here that way
[128:12] it's easier you don't have to watch me
[128:14] always go ahead and type things and
[128:16] importing things and so forth so you
[128:18] have I may still do that but you have
[128:20] this work at txt with all the
[128:23] dependencies so the first thing I want
[128:25] to do of course is go ahead and run that
[128:27] real
[128:29] quick so pip install. R
[128:36] requirements txd like that so it's going
[128:39] to go ahead and get all the dependencies
[128:41] that was pretty fast that's that's very
[128:43] nice and we should be good all right
[128:47] very
[128:48] good and next you can see that I have
[128:50] also created this PDF rag. py this is
[128:54] what we're going to be working in so I
[128:57] have a few things that we're going to do
[128:58] here so I'm going to copy and paste in a
[129:02] few things so first we need to ingest
[129:05] the PDF files and then we need to
[129:07] extract text from that PDF file and
[129:09] split into small chunks so essentially
[129:12] the same things that we have been
[129:14] talking about in our presentation and
[129:16] then we're going to send those chunks to
[129:17] embedding model saved embeddings perform
[129:20] similar research retrieve and do all
[129:22] sort of things okay so that's the
[129:24] overview of what needs to be
[129:26] done all right so let's go ahead and get
[129:28] started now the first thing is I'm going
[129:30] to import a few things here so this is
[129:33] from Lang chain Community now this is
[129:35] working now because I
[129:38] installed a Rand requirements TT which
[129:41] included link chain community and other
[129:44] things so important that this happens
[129:47] all right so we getting the unstructured
[129:49] PDF loader and this online pdf loaded
[129:51] here now we may use one over the other
[129:53] but you have both and next what I'm
[129:56] going to do here I'm going to have the
[129:58] doc path and the actual models so I'm
[130:02] the model is going to be Lama 3.2 and
[130:05] the path through document just have one
[130:08] here is data Boi
[130:12] PDF that's what I'm
[130:15] doing okay so let's load the PDF file
[130:18] okay so I use the unstructured PDF
[130:20] loader and pass the doc path which is
[130:23] this one here and then data is going to
[130:27] get all of the loaded information right
[130:31] okay so now I can go ahead and actually
[130:35] print for preview so we see that things
[130:38] are actually working and you know that
[130:40] things are actually working I'm going to
[130:42] print just about 100 items from our
[130:45] content from data let's go ahead and run
[130:48] this fast
[130:53] so in the back end we should be able to
[130:56] do all the things like getting the
[130:59] information and load the
[131:03] document and I'm going to should be able
[131:05] to see soon here
[131:08] something okay very good so we went
[131:11] ahead and showed a few hundred words or
[131:16] 100 parts of the content that comes in
[131:19] in our content data which is the data
[131:22] that we loaded which is our PDF file
[131:25] very good so we see here P beneficial
[131:27] ownership information and that is good
[131:30] so just to test it out to make sure
[131:32] everything is working okay so that was
[131:34] the end of PDF ingestion so we're able
[131:36] to get that PDF next we're going to
[131:39] extract the PDF and do some chunking
[131:43] essentially okay so for that we need a
[131:47] lot of help of course so I'm going to go
[131:49] ahead and get all of the import so what
[131:51] happens here is that first of all we're
[131:53] getting ama ama embeddings this is a
[131:56] wrapper of AMA through L chain that's
[131:59] the beauty of Lang chain because we have
[132:02] a lot of wrapper classes and methods and
[132:04] so forth and then we have this recursive
[132:06] character text splitter which will help
[132:08] us to recursively split our text and
[132:12] then of course we have the chroma DB
[132:14] through Lang train all right so first of
[132:17] all let's go ahead and split things here
[132:21] keep in mind all we're doing here is
[132:23] independent of large language models
[132:25] we're using this is across the board
[132:28] this is how you would use you would do
[132:30] anytime you want to create a rack system
[132:33] all right so we splitting everything the
[132:36] chunk is 1200 so we're saying for each
[132:39] 1,200 chunks we're going to split so
[132:41] we're saying each chunk will have about
[132:44] 1,200 so we're saying each chunk size
[132:46] will have about 1,200 and the overlap is
[132:49] going to be 300 so the greater the
[132:51] overlap the better because that means we
[132:53] are conserving the context okay very
[132:57] good and then chunks we pass this text
[133:01] splitter which should have all that
[133:03] information actually this is all from
[133:05] the data that we loaded from here and
[133:10] then we use text splitter and split the
[133:11] document so now we have those chunks and
[133:13] if you want to see we can go ahead and
[133:16] print things out real quick here so you
[133:18] can see
[133:22] okay so you can see that we have the
[133:26] number of chunks first says here
[133:28] somewhere number of chunks is
[133:31] 42 and we have the actual
[133:35] chunks table of contents and all of that
[133:38] so this is good that means things are
[133:40] working and the beauty of using lank
[133:43] chain is that it adds metadata to all of
[133:46] these pieces you can see now we have
[133:48] this metadata was added called source is
[133:51] going to be data Boi PDF which is really
[133:54] good okay otherwise we will have to do
[133:56] all of that manually okay so we know
[133:59] this is working let me go ahead and
[134:01] comment this
[134:02] out okay now it's time for us to add
[134:05] into our Vector database because we have
[134:07] those chunks and we need to add them but
[134:09] how do we add to Vector database well
[134:11] first we need to do something first we
[134:14] need to create those embeddings if you
[134:16] go to
[134:18] ama if we go to ama.com and go to
[134:21] models you find many many different
[134:24] models one of the models called
[134:26] embeddings let say embeddings you can
[134:28] see that we have this nomic embedding
[134:31] embed text we also have this MX Bay
[134:35] embed large and many others okay we can
[134:37] use whatever want but this is the one
[134:39] we're going to be using so click here
[134:41] and you can see it's called it's a high
[134:43] performing open embedding model with a
[134:46] large token context window which means
[134:48] that means it allows you to put a large
[134:51] amount of text for embeddings to for it
[134:54] to be able to embed okay so this is what
[134:56] we're going to be using for our example
[134:59] here that means we'll have to pull this
[135:01] in code and so let's go ahead and do
[135:05] that first I'm going to go ahead and
[135:07] import
[135:08] AMA and then I'm going to
[135:11] incode pull in this embedding now I
[135:14] already have this so it's just going to
[135:17] go ahead and update I'm just put this in
[135:19] way so you have and then what we need to
[135:21] do is to actually create the
[135:24] vector database and it's very simple all
[135:27] we do we say chroma from documents and
[135:30] pass the documents chunks remember the
[135:32] chunks are here and the embedding now
[135:36] we're using this AMA embeddings and
[135:38] passing the mo the model nomic embed
[135:40] text to generate the embeddings so all
[135:44] of that is paying p through our cha from
[135:47] documents function and of course we
[135:51] collection name we just pass name simple
[135:53] rag that's
[135:54] fine
[135:56] okay all right so continuing let's go
[135:58] ahead and do the retrieval side of
[136:00] things so how do we retrieve things and
[136:03] so forth so first let's go ahead and
[136:06] pass in a few things here now we need to
[136:10] some help with L chain where we can
[136:12] create prompt templates as you see and
[136:15] also way of parsing whatever comes in or
[136:18] goes through this rack system we'll see
[136:20] in a second here so that's why I'm
[136:23] passing importing all of these and while
[136:25] we're here let's go ahead and also
[136:27] import a few other things such as the
[136:30] chat AMA because that's what we're going
[136:32] to be using that model
[136:34] to as our large language model and then
[136:38] now we have this runable pass through
[136:41] this is part of what we're going to see
[136:43] later part of chaining that happens with
[136:46] AMA with L chain we have this multi-
[136:50] quare retriever as you see it's going to
[136:51] help us with a lot of things because
[136:54] we're going to be able to retrieve
[136:55] documents going to be able to give a
[136:57] query and use a large language model to
[136:59] sort of write a set of queries and pick
[137:02] up the best one to pass through the
[137:05] large language model so this is all
[137:07] optimization under the hood okay let's
[137:09] go ahead and set the model that we're
[137:11] going to be
[137:13] using so this is how set model now this
[137:15] is a wrapper or Lama wrapper and we pass
[137:18] the model which is Lama 3 to all right
[137:21] so we have the large language model
[137:23] already now as you know we're going to
[137:25] go ahead and use the simple technique to
[137:28] allow us again to generate multiple
[137:30] questions from one single question and
[137:32] then retrieve documents based on those
[137:35] questions which allows us to get the
[137:37] best of both worlds so this is the query
[137:41] prompt that I'm going to use here so you
[137:42] can see here I'm using the prompt
[137:43] template the reason why is because we
[137:45] can then pass things like input
[137:46] variables it's going to be the question
[137:49] that internally we'll know how to pass
[137:51] around all these pieces of data what you
[137:53] can see here the templates say you are
[137:55] an AI language model assistant your task
[137:58] is to generate five different versions
[138:00] of the given user question to retrieve
[138:02] relevant documents from a vector
[138:05] database by generating multiple
[138:07] perspectives on the user question your
[138:10] goal is to help the user overcome some
[138:14] of the limitations of distance based
[138:16] similarity search provide these
[138:19] alternative questions separated by new
[138:21] lines original question and it pass the
[138:23] question which is going to be passed
[138:24] along so this prompt template here it's
[138:27] going to be passed along soon through
[138:29] the Retriever and it will know how to
[138:32] deal with all of that this is beautiful
[138:34] because we're using lank chain and it
[138:37] does everything automatically for us all
[138:39] right so now let's go ahead and actually
[138:42] create our retriever so our retriever
[138:44] here we're going to use the multi-query
[138:46] retriever from large langage model and
[138:48] we pass a few things okay we have the
[138:51] vector database so we need of course the
[138:53] vector database that we created here and
[138:55] then we're going to transform it into a
[138:57] retriever so retriever will know how to
[138:59] retrieve things from database it's going
[139:02] to return something and we need to pass
[139:05] the large language model because it
[139:06] needs to know how to take that item all
[139:10] those information retrieved and how to
[139:12] process that and with that we also need
[139:14] the prompt so you remember in the
[139:16] diagram when it comes to once we have
[139:18] everything set up we retrieve
[139:20] information
[139:21] we retrieve pieces in this case we're
[139:23] retrieving pieces that are similar to
[139:25] The query that we're passing along which
[139:27] is inside of a prompt and then we need
[139:29] the large language model which is going
[139:31] to drive everything it's going to use
[139:33] all that information compact everything
[139:35] and be able to answer questions all
[139:38] right very good and next we're going to
[139:40] use the create the rag prompt so let's
[139:43] go ahead and do that real quick so
[139:45] essentially it's another prompt that we
[139:47] can use this is our question that we
[139:49] pass along okay rag prompt this is just
[139:51] another template we're going to pass
[139:53] says answer the question based only on
[139:55] the following context so we're going to
[139:57] pass the context and the actual
[139:59] questions this context and everything is
[140:01] going to passed along as you will see
[140:03] later right internally with blank chain
[140:06] okay so once we have that we need to
[140:08] create an actual chat prompt from all of
[140:10] that so we use chat prompt from template
[140:13] and pass this template here so these
[140:15] wrappers here internally will know how
[140:17] to pass in this context how to pull in
[140:19] the question and so forth cuz these come
[140:21] in as just variables okay and once we
[140:25] done that that's when we then put all of
[140:28] that into a chain which is going to go
[140:30] through the process of pulling in all
[140:33] the all the pieces that we put together
[140:34] so that we can get the result so
[140:36] essentially what it means what is
[140:38] happening here is that we passed the
[140:39] context we know the context is going to
[140:40] be the retriever retriever will know as
[140:43] you saw here how to get things from
[140:47] database knows the large L metal model
[140:50] that's going to be driving everything it
[140:52] also knows the prompt which we created
[140:54] here which has a few input variables
[140:57] we're passing along so we're putting all
[140:58] together and then we have the question
[141:00] runnable pass through let's just say
[141:02] just pass through don't mind much and
[141:04] then we have here a chain to pass in a
[141:07] prompt and then we need the large
[141:09] language model again and then at the end
[141:11] once we have the result we're going to
[141:13] pass that through a string output parsel
[141:16] essentially to parse everything out to
[141:18] make everything clean and nice all that
[141:21] is done we can just start asking
[141:23] questions so I'm going to comment this
[141:25] out let's start with the main question
[141:28] at the top
[141:31] here say what is the document about for
[141:34] instance let's go ahead and run this so
[141:35] this is it this is our fullon rag system
[141:39] let's go ahead and run and see if this
[141:40] works okay so done loading done
[141:43] splitting document done adding to Vector
[141:46] database very
[141:49] good okay
[141:51] it says here the question the answer
[141:53] says the document appears to be a set of
[141:54] instruction for reporting beneficial
[141:57] ownership information especially the
[142:00] Boi filing instructions provide guidance
[142:03] and blah blah blah very good that's
[142:05] exactly right and so now we can pass
[142:08] questions so that's one question let's
[142:11] see if I go and comment this out I want
[142:14] this second on what are the main points
[142:16] as a business owner I should be aware of
[142:19] let's go ahead and run
[142:25] again now this is very inefficient
[142:27] because in a way it goes back and does
[142:30] the splitting and adding everything but
[142:31] that's okay just to show you how things
[142:33] are done okay look at this as a business
[142:36] owner you should be aware of the
[142:37] following main points and gives us all
[142:40] the
[142:41] points and let me do another one here
[142:56] okay based on provided text here's how
[142:58] to report beneficial ownership
[143:00] information Boi to finsen complete
[143:04] filing and there we
[143:08] go all right so it is indeed working we
[143:11] were able to put together a full rag
[143:14] system be it we could improve improve in
[143:17] terms of repetition of you know
[143:20] splitting into all the things that's
[143:21] okay but you can see that it this works
[143:24] we use large language model in this case
[143:26] one large language model from our own in
[143:29] this case ol Lama uh llama 3.2 and then
[143:33] we used embeddings also from all Lama
[143:37] just like that again we can just split
[143:40] around we can change the models that
[143:42] we're using and leave everything as is
[143:45] and it should work very good very good
[143:47] indeed so I hope you're seeing uh the
[143:49] benefits here number one again remember
[143:52] is that we don't pay anything we can run
[143:54] this as many times as we want we're not
[143:56] paying anything for any hits to the API
[144:00] in this case open AI or anywhere else
[144:02] because this is local which is really
[144:04] really awesome if you ask me so I hope
[144:07] this is helpful and uh take this code
[144:10] and change it around make it your own
[144:12] maybe you have a different PDF file that
[144:16] you want to uh converse with or whatever
[144:19] else that you need so go ahead and play
[144:21] around this is your code you have access
[144:23] to it and so just just do it all right
[144:27] so we finished this PDF rag it works
[144:30] fine and so I decided actually to show
[144:33] you here I have the other piece of code
[144:35] which is exactly same that's the same
[144:37] thing but I call this PDF rag clean so
[144:40] essentially it's the same thing as what
[144:41] we've seen it's just that it's a little
[144:43] bit cleaner and looks better okay so
[144:46] essentially at the top here we have some
[144:48] configurations and blah blah blah and
[144:50] have this doc path we have this model
[144:53] name so this created constants at the
[144:56] top here and Vector name as well okay so
[144:59] we have a function here called injest
[145:00] PDF it's essentially that we just pass
[145:03] the doc path and it does all the thing
[145:05] that it needs to do we have the split
[145:07] documents function this is how exactly
[145:10] how you should do it right split
[145:11] everything into modules and then we have
[145:14] this create database and it returns a
[145:17] VOR DB and we create Retriever and
[145:19] everything
[145:20] and we're passing along the templates
[145:24] and all of that so essentially like I
[145:25] said what we've seen before but now in a
[145:28] more cleaner way and then we have this
[145:31] create chain which creates the chain for
[145:33] us as we saw before the code is still
[145:35] the same and then we have this main so
[145:38] now we just call for instance get the
[145:40] data by calling injust PDF and then we
[145:43] get the chunks VOR DB instantiation the
[145:46] llm instantiation and Retriever and then
[145:49] we have to check and then we have our
[145:51] question right so we can add different
[145:53] question here let's go ahead and run
[145:55] this we should see the same thing
[145:57] essentially but just different code
[146:01] which is
[146:02] cleaner okay so it's doing all the
[146:07] things okay so there's a lot of
[146:10] good logging that's happening and voila
[146:13] so at the end response according to
[146:15] instructions this is what we have so
[146:17] nothing out of this world the same thing
[146:19] but this is is way better because it's
[146:21] way organized so you have both and you
[146:24] are set and because it's always nice to
[146:27] have some sort of a user interface I
[146:29] have another version you have access to
[146:32] called PDF Rag streamlit and in this
[146:35] case when we run this we should be
[146:38] having we should be able to have a user
[146:41] interface using streamlet so essentially
[146:43] obviously it's the same code it's just
[146:45] that we have at the bottom here
[146:47] streamlet let look see
[146:50] setting up the title and user input and
[146:54] everything else okay you can look at
[146:55] this code but it's it's not uh out of
[146:59] this world so let's go ahead and run
[147:00] this so I can show you real
[147:05] quick okay there we go so now I
[147:11] can let's see so it's going to go
[147:14] generate response verus running the load
[147:16] Vector DB and in a few moments hopefully
[147:19] we should see an
[147:22] answer okay so to file for beneficial
[147:25] ownership information there we go all
[147:28] that information very good very good
[147:31] indeed and I can
[147:33] say what's the
[147:38] penalty for not
[147:42] filing there we go and it gives us the
[147:46] information all right so there you have
[147:48] it now you have different versions and
[147:51] this is the user interface version that
[147:54] you can see that we are conversing with
[147:57] our rack system with our document so
[148:00] quick break here I just want to let you
[148:02] know that I thank you for being here for
[148:04] learning this awesome tool with me here
[148:08] I do have something for you as a way of
[148:10] me saying thank you for being here with
[148:12] me and I appreciate you watching this
[148:13] video this mini course I have an olama
[148:17] pack that you can download that includes
[148:19] source SCE code code templates and many
[148:22] other things that it's going to be a
[148:24] surprise and you can get that for free
[148:26] obviously you go in the link in the
[148:28] description and you can go and get this
[148:30] AMA starter pack all right okay let's
[148:33] continue all right so the next
[148:34] application that we're going to be
[148:36] building we're going to still use AMA
[148:38] Lama 3.2 but you can always change the
[148:41] model and this application is going to
[148:44] be a full-fledge application that allows
[148:46] us to actually use swarm framework which
[148:49] is is a new open AI framework that
[148:53] allows us to create agents agents that
[148:56] will actually be able to delegate
[148:58] certain tasks to other agents and do
[149:02] different tasks so this is going to be
[149:04] full-fledged application an AI solution
[149:07] application that you can actually use in
[149:10] real world most of the ones that we've
[149:12] been doing you can use in real world of
[149:14] course but this one is something that is
[149:16] actually I would say that it's usable so
[149:19] if you want to get a job usually you
[149:21] have to find somebody who will have that
[149:24] job for you okay and usually you have to
[149:26] go through recruiters and recruiters
[149:28] usually work in the agency and it's a
[149:31] slew of process a lot of people that are
[149:33] involved usually to get you matched with
[149:37] an employer and so in this video I'm
[149:39] going to show you how to take this whole
[149:43] agency with different people working
[149:46] these are recruiters how to actually
[149:48] create AI agents that will do exactly
[149:52] what the real recruiters do these
[149:55] recruitment agencies have a lot of
[149:58] moving parts and one of the things that
[150:00] can be very difficult is that when they
[150:03] need to vet people they actually have to
[150:06] of course read the resumés and they are
[150:08] getting many many of those resumés and
[150:10] have to go through each one of those and
[150:13] contact people and try and trying to
[150:15] figure out their qualifications and how
[150:18] would they match them to an employer
[150:20] what would be a solution well you know
[150:22] this drum roll yes the solution would be
[150:25] AI agents with AI agents they're going
[150:28] to be able to pass along different tasks
[150:31] to other AI agents to do exactly that
[150:34] from extracting information and then
[150:37] analyze the resume and maybe start
[150:40] figure out how to connect or how to
[150:42] match to different employers or
[150:45] different positions and then have a full
[150:48] report with recommendation
[150:50] so that the whole agency knows what is
[150:53] the process that went through to get to
[150:55] that conclusion where X person or X Y
[150:58] and Z people need to be matched to to a
[151:02] certain company and so that is exactly
[151:04] what we're going to be doing in this
[151:06] video I'm going to show you how to use
[151:08] large language model in this case AMA
[151:10] Lama
[151:11] 3.2 to Aid Us in this on top of that
[151:15] we're going to use swarm framework that
[151:17] allows us to create agents AI agents
[151:20] very quickly simply and to the point so
[151:24] let's go ahead and let's go ahead and
[151:27] build a recruitment agency all with AI
[151:30] agents so here's what we're going to be
[151:32] building so here is the user interface
[151:35] so it is indeed a UI recruiter agency so
[151:38] you can go ahead of course and upload
[151:40] things but most importantly you can see
[151:41] we have this about that we click it
[151:44] gives us some about this whole thing now
[151:46] of course here is saying AMA 3.2
[151:49] the framework swarm framework and all
[151:51] the things so it says here our system
[151:53] uses specialized AI agents to do what
[151:57] well to extract information from rums
[152:00] and analyze candidate profiles and also
[152:03] match with suitable positions screen
[152:07] candidates and then provide details
[152:09] recommendations so this is going to be a
[152:12] full-fledged agency that we're going to
[152:15] be building using code and llama 3.2
[152:19] okay of course the front end is going to
[152:21] be stream it which is what you see here
[152:23] so the idea is very simple and let me go
[152:26] back to the upload so here what we can
[152:28] do is we can go ahead and upload files
[152:31] now I already have a resume that I'm
[152:33] going to go ahead and just drop it right
[152:36] there so you can imagine now things are
[152:38] going to be happening in a back end so
[152:40] the analyzing of the resumé is happening
[152:42] well first of all different agents are
[152:45] doing what they need to do to get
[152:47] started so the first agent is going to
[152:49] go ahead and and pull information from
[152:52] this PDF file which is a resume and then
[152:54] it's going to pass on to different
[152:56] agents that will analyze and do all sort
[152:58] of things so this is going to take a
[153:00] little bit and at the end the main agent
[153:02] is going to be able to give us
[153:05] recommendations through analysis and
[153:07] there we go we can see that we have now
[153:09] this analysis complete so we have
[153:11] recommendations so so we have analysis I
[153:14] can also close this tab menu there so
[153:17] scale analysis analysis for Janine
[153:19] Nail's resume and gives us sales
[153:22] engineering industrial engineering
[153:24] project manager a management and all
[153:27] these great great stuff so it pulls in
[153:29] all the information from the resume and
[153:31] at the end here you can see
[153:32] recommendation to enhance J Na's profile
[153:35] consider following add more experience
[153:38] develop technical expertise increase
[153:40] Education and Training confidence is
[153:45] 85% wow it gives us the confidence that
[153:48] this person could potentially do well in
[153:51] a certain industry and of course I'll
[153:54] show you that results also have been
[153:56] saved locally but you can see also have
[153:58] other things here we have this job
[154:00] matches it tells us exactly this person
[154:04] matches well 80% in New York for this
[154:07] data scientist position all right and we
[154:10] have the software engineer position and
[154:12] says it the match is 70% and is remote
[154:16] so this is exciting that's exactly what
[154:18] we're going to be building and also we
[154:20] have the screening here screening
[154:22] results gives us a summary of everything
[154:24] qualifications and the percentage uh
[154:27] experience relevance and all of this
[154:30] information all right and of course we
[154:33] have another tab the final
[154:34] recommendation they give us the
[154:37] recommendation and all of this ladies
[154:39] and gentlemen is built using AI but most
[154:43] importantly using AI agents you can see
[154:46] how helpful how useful this tool can be
[154:49] by the way if you are interested in
[154:51] being a part of a community where you
[154:53] find like-minded people to build AI
[154:56] tools from scratch and you will have
[154:58] templates we have weekly monthly calls
[155:01] that we have for Q&A and many many other
[155:04] things in this community where people
[155:06] are actually building AI Solutions
[155:09] you'll have everything you need to build
[155:12] AI Solutions using code and AI tools is
[155:15] a place where I'm very proud about and
[155:17] there's a lot of people who are enjoying
[155:19] this community and I would love for you
[155:21] to join the community where you'll be
[155:23] able to dive deeper into building AI
[155:27] solutions that actually solve problem
[155:29] okay so let's go ahead and get started
[155:31] all right so I already have the code and
[155:34] because of the nature of this project uh
[155:37] there's a lot of moving parts and uh
[155:39] there's a lot of code as well not too
[155:41] much and so what I will do I will just
[155:43] walk through the code and no worries
[155:45] you're going to have access to all this
[155:47] code but what I want you to keep in mind
[155:49] is that you can take this and apply to
[155:52] something else so that is the beauty of
[155:54] using agents that's the beauty of
[155:56] building these Solutions AI Solutions so
[155:59] I have here inside AI recruiter agency
[156:03] and we have this folder called agents
[156:05] let's focus on that real quick but at
[156:07] the center of everything the way I
[156:09] wanted to organize this code is that
[156:11] because each agent will have to have
[156:15] access to the large language model in
[156:17] this case we're using AMA that's another
[156:19] Point here is that in this case we're
[156:21] using AMA Lama 3.2 which means you don't
[156:24] have to pay for inference or pay for an
[156:27] NPI key which is really great and it's
[156:31] really good because then you can build
[156:33] these systems build these AI Solutions
[156:36] locally and test them out before you can
[156:39] push them to production so that is one
[156:42] of the great things about using AMA
[156:45] models but it is still the same if you
[156:47] want to use different models Okay so at
[156:49] the center of everything the way I
[156:51] organize this code is a little bit more
[156:53] organized in a sense where you would
[156:55] find in a production application now
[156:58] there are a few things here that you
[156:59] probably will have to kind of work
[157:03] around to make it production ready but I
[157:05] put the code so that it is a little bit
[157:07] more organized we're using classes and
[157:10] everything now if you don't know a lot
[157:12] about classes bython it's okay um I have
[157:15] a lot of videos where I talk about
[157:17] python um but this is how you would
[157:19] usually want to put together a
[157:21] potentially production ready application
[157:24] at the center of everything we have this
[157:26] base agent here what is this well it is
[157:29] indeed a class make this smaller so you
[157:30] can see as you can see here it's a class
[157:33] called Base Class that we pass in a few
[157:37] things in this Constructor so when we
[157:39] call Base Class which is going to be
[157:41] that all agents are going to inherit
[157:43] from which means they're going to model
[157:46] after um this is what we have so each
[157:49] agent of course has a name instructions
[157:52] so this is the model uh way that we do
[157:55] with
[157:57] agents using swarm that is what we're
[157:59] using here and you pass in of course the
[158:02] opening eyes now because I have another
[158:05] video where I talk about how to use
[158:08] swarm and opening ey but override
[158:10] opening eye in this case so that we can
[158:12] use l 3.2 or AMA any AMA model okay so
[158:18] this is how you override that I have
[158:19] another video where I talk about that
[158:21] you can find somewhere here or you can
[158:23] just search on my channel okay so what
[158:25] we do here we pass a base URL and
[158:28] instead of having in this case the base
[158:31] URL that would have been for openi we
[158:34] have our own base Ur So this tells us
[158:37] that this is going to be locally using
[158:40] AMA in this case pointing to a llama and
[158:44] AMA model okay and then we pass API key
[158:48] here just AMA it's not requ required but
[158:50] it needs to be passed around so at this
[158:52] point here when we instantiate this base
[158:55] agent we're going to know that we
[158:57] actually going to be using instead of of
[158:59] open AI because we're using swarm
[159:01] framework we're going to be using our
[159:04] local AMA models okay and then we have
[159:07] this function here that is defined
[159:08] called run so this will get of course
[159:12] the message which going to be a list and
[159:15] then we have this query or Lama so at
[159:18] this point we're going to pass a a
[159:19] prompt as a string and then we're going
[159:21] to go through the process of calling our
[159:24] client or Lama client which was created
[159:27] here and then we say hit the chat in
[159:30] completions that create so in this case
[159:33] here you notice we passing the Lama 3.2
[159:36] and we have messages in this case we
[159:38] have system and we pass the self
[159:40] instructions so these instructions will
[159:42] have passed along through the agent okay
[159:46] so this is how you structure it I know
[159:48] for some of you this may be a little bit
[159:49] Advanced but it's a good it's a good
[159:52] exercise and we're going to pass
[159:53] temperature and tokens and all of that
[159:55] all right and then we return the
[159:57] response so whenever we say query orama
[160:01] this is what's going to happen we're
[160:03] going to pass the model and then the
[160:05] messages all of that is going to be
[160:07] passed along also the prompt and the
[160:09] instructions okay okay so here is this
[160:12] helper function here par Jon safely this
[160:15] is just going to bring get the text that
[160:18] is coming in as I said safely pars Json
[160:21] from text handling potential errors
[160:23] that's all we're doing here so that is
[160:25] the base agent so now once we have the
[160:28] space agent what we need is to create
[160:31] the actual agent swarm opening ey agents
[160:35] and if I go back here you can see that I
[160:37] have a bunch of Agents here so these
[160:39] agents we have for instance the
[160:41] extractor agent as name imply is going
[160:43] to be responsible for extracting
[160:46] information from the PDF file and then
[160:49] we have the match agent which is going
[160:51] to match whatever we got from the
[160:54] extraction in this case the PDF and try
[160:57] to match it with the company or the
[160:59] position that the person may be a good
[161:02] fit for we have the orchestrator and we
[161:05] have profile enhancement recommander and
[161:07] Screen agent let's start with screen
[161:09] agent because all of these agents will
[161:12] essentially in terms of code codewise
[161:15] they will look pretty much the same now
[161:17] this is important look how beautiful
[161:18] this is so the screen agent here we are
[161:21] creating a class just to make things
[161:23] simpler and scalable and then it's
[161:26] inheriting inheriting from the base
[161:29] agent right because the base agent as we
[161:31] saw has all the properties all the
[161:33] functions or methods that needs for it
[161:35] to actually become an agent so that's
[161:38] what we're doing here we're initializing
[161:39] our things so we are calling
[161:41] initializing the names to be screener
[161:43] and we have the instructions here to say
[161:46] screen candidates based on
[161:48] qualifications alignment experience and
[161:50] everything so we're passing the actual
[161:52] instructor instructions for this agents
[161:55] we say provide comprehensive screen
[161:57] reports and all that and we have the run
[162:00] so this is just going to go print to say
[162:02] okay screener conducting initial
[162:03] screening that's all that is this uh the
[162:06] Run function is going to do and then we
[162:08] have work workflow context here we are
[162:10] evaluating getting information from our
[162:12] messages that we getting and then get
[162:15] the content so essentially we're going
[162:16] to get what the large language model in
[162:19] conjunction with the agent the results
[162:22] and pass that into the query AMA and if
[162:25] you remember if I command click query
[162:27] AMA what does it do query AMA model with
[162:30] the given prompt so now we get that
[162:32] information we got and then pass through
[162:35] AMA this agent here well it's a call
[162:38] essentially to this endpoint so that it
[162:41] knows what to do and then we get the
[162:43] response and of course we return an
[162:46] object or in this case a dictionary
[162:48] which is going to have screening report
[162:51] and we're passing into date here we
[162:52] could make this more Dynamic but this is
[162:54] okay and screening score 85 in this case
[162:59] right so let's go ahead to another agent
[163:01] let's go to extractor agent so what does
[163:03] it do well this extractor agent as name
[163:06] apply it just extracts the information
[163:09] from a PDF file okay we importing a few
[163:12] things here but the idea is very simple
[163:15] again it's base agent is the base agent
[163:17] that we pass right that's going to to be
[163:19] the model for this extractor agent and
[163:22] we have the name instruction extract and
[163:25] structure information for from reses
[163:27] focus on personal information work
[163:29] experience education skills and
[163:31] certification and all that provide
[163:33] output in a clear structured in format
[163:36] so it's important that we make these
[163:39] instructions very direct and very
[163:41] concise okay we have to run again so
[163:44] this case is going to show us this
[163:46] extractor is processing resume and we do
[163:48] the exact the same thing as we saw
[163:50] before so like like I said earlier most
[163:53] of these are going to be the same in
[163:55] terms of the code right and we extract
[163:58] everything and also we um extract the
[164:01] text in this case from the PDF file
[164:03] because we're going to pass that along
[164:05] because we take of course the raw text
[164:07] that we should have at this point and we
[164:10] pass along to query AMA as you know what
[164:12] it does and then of course we return
[164:14] again the raw text the structure data in
[164:18] this case the exu
[164:19] structured I should say information and
[164:22] the results so we're just returning a
[164:24] dictionary so we can see what's going on
[164:27] all right so it's pretty much the same
[164:28] let's go to matcher the matcher here you
[164:31] know what it's going to do it's going to
[164:33] match whatever it was extracted from
[164:36] resume trying to match with certain
[164:38] positions that they might have so it's a
[164:41] matcher the same thing again that's the
[164:43] beauty and the instructions here is that
[164:45] match candidate profiles with job
[164:48] positions consider skill match
[164:50] experience level location and everything
[164:53] return matches in Json format with title
[164:56] match score and location Fields okay the
[164:59] same thing as you see here this run it's
[165:02] going to print matcher finding suitable
[165:05] jobs and all the great stuff now here is
[165:09] where we just passing sample jobs so
[165:11] this is where you would perhaps hit a an
[165:14] API that might have somewhere with a lot
[165:17] of different jobs or or a PDF file or a
[165:22] CSV file whatever you have that would
[165:24] have different jobs so this case here we
[165:26] just have the samples jobs with with a
[165:29] few titles and everything so that's the
[165:31] beauty you can just plug in the source
[165:34] where you'll have a list of different
[165:37] jobs with requirements locations and
[165:40] everything okay very good so that's it
[165:42] that's what's going to be used for the
[165:44] matcher to be able to match it's going
[165:46] to look at the information that it's
[165:47] received from other people this case
[165:50] other agents and find the match okay so
[165:54] match response so this is where the
[165:56] matching results from a Lama come in so
[165:59] what we do we call the query or Lama and
[166:02] we pass this F string here analyze the
[166:04] following profile and provide jobs
[166:05] matching valid Json okay so we have the
[166:08] profile here it's going to be and passes
[166:10] in okay the analysis because in order to
[166:13] analyze in order to do the matching it
[166:15] needs to have the pieces that it needs
[166:18] in this Cas anal analysis results that
[166:20] were gotten from a different agent okay
[166:24] and then we pass the available jobs is
[166:26] this so here's return only adjacent
[166:29] object with this exact structure so it's
[166:33] very important to usually guide the
[166:35] large language models when it comes to
[166:38] the formats that you want okay and there
[166:40] we go so this is the format that we have
[166:43] here make it smaller so you can see
[166:45] everything and then we can parse the
[166:48] response to make sure that we get
[166:50] everything in this is PA Jon safely
[166:53] function that we saw in the um agent I
[166:57] believe the base agent okay we have some
[167:00] fullbacks here uh for errors and
[167:02] everything and then we just return the
[167:04] par response now before we go to the
[167:07] orchestrator here let's go to the en
[167:10] profile enhancer agent so what does it
[167:12] do well this is going to kind of enhance
[167:15] the profile as the name apply it's it's
[167:18] going to go ahead and pass in the
[167:20] extracted information it's going to copy
[167:22] that information and enhance essentially
[167:25] the user the profile of that candidate
[167:28] and return that enhanced profile
[167:30] essentially to extract what needs to
[167:31] extract and put all nicely okay and then
[167:35] of course uh the profile agent here
[167:38] that's what we pass we call agent
[167:40] profile agent and pass Lama 3 and we say
[167:43] enhance the candidate's profile based on
[167:45] the extracted information so this is
[167:47] where we actually call this is where
[167:49] then we pass this function that we
[167:52] created here
[167:53] okay and pass through the agent because
[167:56] you will know that is the function it
[167:58] needs to run very cool all right let's
[168:01] continue here and let's go to the
[168:03] recommender agent so recommender agent
[168:05] as the name imply it's going to be the
[168:07] same thing it's going to recommend right
[168:09] so we have the instructions here
[168:11] generate final recommendations
[168:13] considering extract profile skills
[168:15] analysis and provide all these
[168:17] information so again nothing new here
[168:20] and of course we're printing everything
[168:23] when we run and in this case here we do
[168:26] the same thing we pass we call the query
[168:29] AMA and pass the workflow context so
[168:32] essentially is the information that
[168:34] we're getting from the evaluation of all
[168:36] the messages come in okay and then we
[168:38] return the recommendation this is going
[168:40] to be sort of a a dictionary with a time
[168:44] stamp and confidence level in this case
[168:47] we just say hi okay of course you can
[168:49] add more things to make it actually um
[168:52] realistic but this is pretty good okay
[168:54] let's continue let's go to screener
[168:56] agent now the screener agent I think we
[168:58] looked at it well that's exactly what it
[169:00] does it screens to make sure everything
[169:02] is good and pass it along all right so
[169:06] that is what it does it's very simple
[169:08] the same thing again and there you have
[169:11] it now the most important thing here
[169:13] again to keep in mind is that everything
[169:15] is base agent so that means all these
[169:17] agents have a parent class that sets
[169:20] everything up so that is the best way
[169:23] for you to think about when you're
[169:24] building real world applications or
[169:26] Solutions all right okay let's go ahead
[169:29] and look at the orchestrator the
[169:31] orchestrator in this case is the one
[169:33] that orchestrates everything that is the
[169:35] the the myestro if you will okay and so
[169:39] it has a lot of things of course we have
[169:41] to import all of those but again it is
[169:44] still an agent so it has to um inherit
[169:47] from base agent so we give it a name we
[169:50] give it instructions Now instructions
[169:51] are very important for agents okay says
[169:54] it coordinate the recruitment workflow
[169:56] and delegate tasks to spe uh specialized
[169:59] agents ensure proper flow of information
[170:02] between extraction extraction analysis
[170:05] matching screening and recommendation
[170:07] phases phases maintain context uh
[170:10] important maintain context and aggregate
[170:13] results from each stage okay so we got
[170:16] self that set up agents so the a
[170:19] function that is been called that goes
[170:22] ahead initializes all the specialized
[170:25] agents all the ones that we' talked
[170:27] about look at that okay very good
[170:29] because as an orchestrator it has to
[170:31] know which agents that it has access to
[170:34] to start
[170:36] delegating we have the run again the
[170:38] same thing we've seen and then we have
[170:40] this process application so essentially
[170:42] it's going to just go ahead and pass
[170:43] some information here along and print
[170:46] out orchestrator starting application
[170:48] process and so forth so we extract the
[170:51] resume so it's going to go ahead and run
[170:54] to go and extract resume update in this
[170:56] case the work context workflow context
[171:00] and then we analyze the candidate
[171:02] profile okay called self analyzer to
[171:05] start the
[171:07] analysis right so this case we're
[171:10] calling all of those agents right you
[171:12] can have over can see it is indeed an
[171:14] analyzer agent okay and then we app
[171:18] upate the workflow context here so each
[171:21] step of the way as we're calling these
[171:23] different or the orchestrator is calling
[171:25] the these different agents is going to
[171:27] do what it needs to do okay delegate and
[171:30] create in this case add to the workflow
[171:33] context okay and does with matching jobs
[171:36] screening candidates and then we
[171:39] generate the recommendation all right so
[171:41] it calls the recommander and that's
[171:43] exactly the same thing and we have some
[171:45] exception just in case things don't work
[171:47] and that's pretty much it now we have
[171:49] other things here logs and everything
[171:51] but that's something that you can take a
[171:53] look yourself but this this is it and so
[171:55] once we have that we also have of course
[171:58] the app that why a app that py should
[172:02] say so this has some code with streamlet
[172:06] which allows us to look at something
[172:09] that is nice to look at as you saw
[172:11] earlier okay so at this point here let's
[172:13] go ahead
[172:15] and do a quick run so we say stream late
[172:18] run
[172:19] app.py and what will happen is going to
[172:22] go ahead and open our agency AI
[172:25] recruiter agency we have a broken link
[172:28] here but that's okay we can go to about
[172:30] for instance and tells us about this
[172:33] agency it's very very cool what it does
[172:36] and everything and all of this was
[172:38] created using streamlet and we go to
[172:40] upload now we can actually browse and
[172:42] upload something a resume so I do have a
[172:45] resume that we can go ahead and upload
[172:47] I'm going to upload here real quick and
[172:49] the moment you upload that drop it and
[172:52] it's going to start the process and the
[172:55] cool thing as this happens I have here
[172:57] logs that are actually being sent out
[173:01] and also I have what's happening in the
[173:03] whole orchestration uh showing here
[173:06] orchestrator starting application
[173:08] process and is analyzing everything and
[173:12] it's working it will take a little bit
[173:14] because there's a lot of things are
[173:16] happening and you can see all of that
[173:19] Happening Here Right tells us
[173:22] orchestrator starting analyzer did that
[173:25] matching screener and recommender and
[173:30] it's happening so we're going to take a
[173:32] little moment here until hopefully we
[173:34] get the final results okay so after a
[173:37] little bit you can see that then we have
[173:40] here analysis complete ah how cool is
[173:43] that so we have different tabs here with
[173:45] different information right so we have
[173:47] skills analysis is it tells us exactly
[173:50] what happened key skills and expertise
[173:52] level and goes over everything about
[173:55] this candidate two years of experience
[173:58] and it gives it so it's extracted
[174:00] everything from the resume and
[174:03] qualifications um and also even some
[174:06] subjections right however the fact that
[174:08] her master degree is not explicitly
[174:10] mentioned in her work experience section
[174:12] raises question about how much of her
[174:14] degree was applied in real world real
[174:16] world scenarios so this is these are
[174:18] agents uh just like you know real world
[174:21] agents real agents who would have to
[174:23] think about um whatever they are
[174:26] thinking whatever they are analyzing so
[174:28] career progression potation red flags
[174:30] look at that all of this okay and the
[174:34] confidence uh score is
[174:36] 85% and here if I go back to next tab
[174:40] look at this job matches data scientist
[174:43] 75% New York Senior software engineer
[174:47] about 60% all right we can look at
[174:50] screening so screening results 85% of uh
[174:54] screening results with confidence based
[174:56] on the provided information I will
[174:58] perform a comprehens comprehensive
[175:00] screening of Janine Nel's resume
[175:03] qualification alignment it goes through
[175:05] all of that so all these all these
[175:08] agents were able to work together to
[175:10] come to a certain uh conclusion okay and
[175:13] we can go through and all that it also
[175:16] says results are under result results
[175:18] analysis so we also create a textual
[175:20] file txt file with all these
[175:23] analysis okay and I can go
[175:26] recommendation and based on this
[175:27] analysis I would recommend that Janine
[175:30] now develop more explicit examples
[175:33] expand her Professional Network and all
[175:35] of these different things all right
[175:37] there you have it so just like that
[175:38] ladies and gentlemen if you not excited
[175:41] about this I don't know why really will
[175:42] make you excited this is something that
[175:45] you can actually use it's usable right
[175:48] you can imagine that if you have in fact
[175:50] here's what I will do if you have a
[175:52] friend of yours who is a recruiter show
[175:55] them this video and see if they could
[175:57] use something like this right don't make
[176:00] them feel like their jobs are in
[176:02] Jeopardy no but just some ask them if
[176:04] they could use such thing how much time
[176:07] they will save them um daily from
[176:10] vetting people and all that all right
[176:13] well thank you so much for going through
[176:15] this course and I really hope that this
[176:17] course was helpful and that you can see
[176:19] the power that we get when use o Lama
[176:22] because now we're able to pick and
[176:25] choose different models and we can build
[176:27] really good AI Solutions locally without
[176:30] having to spend a dime at least um you
[176:33] will have to have a computer of course
[176:35] you don't have to pay for opening ey
[176:37] keys and all of that so I hope this was
[176:40] useful I hope this was helpful to you
[176:42] the thing also I've been hinting
[176:44] throughout this entire course is that I
[176:47] do have a surprise for you which is what
[176:50] I call Ama startup pack which will have
[176:53] everything that you need to get started
[176:55] with AMA from code templates um and
[176:58] other things that are surprise so you
[177:00] should see a link in the descriptions if
[177:03] you don't see it you can you can just
[177:04] leave a comment as well I'll show you
[177:06] the link where you can go and get all of
[177:07] that again thank you so much for your
[177:10] time and I really hope this was helpful
[177:12] if you have any questions at all just
[177:14] leave them below and I'll do my best to
[177:18] um serve you to answer them thank you
[177:20] again till next time be well