[00:00]
in this short video I'll teach you
[00:01]
everything you need to know to get up
[00:03]
and running with AMA which is a
[00:05]
fantastic free open-source tool that
[00:08]
allows you to manage and run llms
[00:10]
locally rather than having to pay for
[00:12]
Chad GPT or use these hosted services
[00:15]
online you can actually run all of these
[00:17]
models locally on your own computer so
[00:19]
you get privacy security and best of all
[00:22]
they are completely free so with that in
[00:24]
mind let me show you how to set this up
[00:26]
get it running and I'll also explain to
[00:28]
you how you can utilize this through
[00:29]
code because olama provides an HTTP
[00:32]
server which means you can call your
[00:34]
models from really any type of
[00:35]
application so first things first we do
[00:37]
need to install AMA so to do that you
[00:39]
can go to the website which is ama.com I
[00:42]
will link it in the description and you
[00:44]
can simply press on download and then
[00:46]
select your operating system in my case
[00:48]
I'm using Windows but of course you have
[00:50]
the command for Linux and then the
[00:51]
installation for Mac now once that's
[00:53]
downloaded simply double click it and
[00:55]
install it and then I'll show you the
[00:56]
next step so once you've installed AMA
[00:59]
there's a few different ways to run it
[01:00]
first of all you can just open the
[01:02]
desktop application so if you're on
[01:03]
Windows you can go here to the search
[01:05]
bar and just search for ama if you're on
[01:07]
Mac you can simply go to the spotlight
[01:09]
search same for Linux and just run the
[01:11]
application now when you do that
[01:12]
nothing's going to appear on your screen
[01:15]
and the reason for that is this just
[01:16]
starts a backend server that's running
[01:18]
the AMA service now the other way to do
[01:20]
this is to open up a command prompt or a
[01:22]
terminal so you can see I'm in command
[01:24]
prompt here on Windows and then to
[01:26]
simply just type O llama if you do this
[01:28]
you should get some kind of out put and
[01:30]
if you see that it means that you've
[01:31]
installed o llama correctly at this
[01:33]
point I'll assume that you have this
[01:34]
installed correctly and that this
[01:36]
command gave you some kind of output and
[01:38]
now what we can do is start running
[01:40]
models so the first thing to look at is
[01:42]
the different models that we have access
[01:43]
to with olama now truthfully you have
[01:46]
access to pretty much any open-source
[01:47]
model you want and you can even write
[01:49]
some custom configurations to use your
[01:51]
own models or things you pull in from
[01:53]
something like hugging face now if you
[01:55]
go to the AMA GitHub repository which I
[01:57]
will link down below you can see some of
[01:59]
the common mods mod that you may want to
[02:00]
download now keep in mind that since you
[02:02]
are running these locally you will need
[02:04]
to download the entire model and you'll
[02:06]
need to have enough space on your
[02:07]
computer you can see some of these are
[02:09]
43 GB for example and enough RAM to run
[02:12]
and load the model depending on how
[02:14]
large it is so you see if we look
[02:15]
through here it defines the number of
[02:17]
parameters for these models if we look
[02:19]
at something like llama 3.1 we have 231
[02:22]
GB and 45 billion parameters and if you
[02:26]
go down here to this note it specifies
[02:28]
how many gabt of ram you should have
[02:30]
based on the different model parameters
[02:31]
so even on my computer which has 64 GB
[02:34]
of RAM it would be difficult to load the
[02:36]
new llama 3.1 model with the 405 billion
[02:39]
parameters so keep that in mind when you
[02:41]
are choosing the models that you want to
[02:42]
use for now I'm just going to go with
[02:44]
the standard llama 2 model because this
[02:47]
is older and it's not as large and I
[02:49]
know that I can run it and most of you
[02:50]
should be able to run it as well so I'm
[02:52]
going to show you how we can pull that
[02:54]
but if you're looking for a list of all
[02:56]
of the models you have available you can
[02:57]
go to the oama library so if you go to
[03:00]
ama.com library and you can scroll
[03:02]
through here and you'll see there are
[03:03]
hundreds of different models you can
[03:05]
sort them you can filter and you can
[03:07]
find even multimodal models except
[03:09]
things like video photos voice Etc so
[03:12]
once you've decided on a Model that
[03:14]
you'd like to run it's very simple to do
[03:16]
so all you need to do is type a llama
[03:18]
run and then the identifier of that
[03:20]
model now in my case I just want to run
[03:22]
llama 2 I know this is an outdated model
[03:24]
I'm just doing it because it's smaller
[03:26]
so I can simply type O llama run llama 2
[03:29]
and if this model is not already
[03:31]
installed on my system then it will
[03:33]
download it and install it for me if it
[03:35]
is already installed it's just going to
[03:37]
bring up a prompt where it allows me to
[03:39]
actually start typing to the model and
[03:41]
messaging with it so notice here that
[03:43]
it's just loading and it kind of gives
[03:44]
me these three arrows and I can just
[03:46]
start typing something to the model and
[03:47]
get some kind of response and you can
[03:49]
see it's pretty much instant because
[03:50]
there's no latency it's running on my
[03:52]
own machine now again if this wasn't
[03:54]
already installed it would start pulling
[03:56]
the model for you and then you would
[03:58]
have to wait for it to finish it would
[03:59]
install then you can run the model and
[04:01]
you can start using it now after some
[04:03]
experimentation it's told me that you
[04:05]
can type slash bu to get out of this so
[04:07]
if I type slby you can see that it will
[04:09]
enclose this window and then if we want
[04:11]
we can type amaama and then list and we
[04:14]
can list the different models that we
[04:15]
have available on our system in this
[04:16]
case you can see I have llama 2 which is
[04:19]
the latest version if I had any other
[04:20]
models they would show up here so that's
[04:22]
the basics on running models using oama
[04:24]
but there's a lot more to show you so
[04:26]
make sure you stick around after a quick
[04:28]
word from our sponsor today's video is
[04:31]
sponsored by SEO writing a tool that's
[04:33]
transforming content creation across
[04:35]
different niches and industries their
[04:37]
new brand voice feature lets you
[04:39]
generate content that matches your
[04:40]
Unique Style whether you're writing
[04:42]
tutorials reviews or even industry
[04:44]
analysis one click generates a complete
[04:47]
blog post with AI generated images and
[04:49]
relevant videos embedded automatically
[04:52]
potentially saving you hours of manual
[04:54]
work what sets SEO writing apart is
[04:56]
their deep web research with built-in
[04:58]
citations when you need accurate
[05:00]
up-to-date information the platform
[05:02]
pulls from reliable sources and adds
[05:04]
citations automatically their humanized
[05:06]
text feature helps your AI generated
[05:08]
content stand out while their external
[05:10]
linking feature intelligently connects
[05:12]
to relevant resources and for all you
[05:14]
WordPress users out there there's a
[05:16]
gamechanging feature that lets you
[05:18]
connect your site and autopost content
[05:20]
directly this feature allows for
[05:22]
consistent scheduling while focusing on
[05:24]
other projects now if you're ready to
[05:26]
try it for yourself then use my code TW
[05:28]
wt20 5 for a 25% discount click the link
[05:32]
in the description and see how SEO
[05:34]
writing can fit into your content
[05:36]
strategy all right so we are continuing
[05:37]
here and I want to show you what happens
[05:39]
when you pull multiple models so again
[05:41]
if we go back to the library we can
[05:43]
start looking through different models
[05:44]
that we may want to utilize maybe I can
[05:46]
even just go back here to the GitHub if
[05:47]
I want to find them a little bit easier
[05:49]
and maybe I want to use the mistal model
[05:51]
as well if that's the case I can just
[05:53]
copy this command or the name mistl I
[05:55]
can go back here I can simply run the
[05:57]
command AMA run mistl it will then pull
[05:59]
that manifest for me pull the model once
[06:02]
that's finished I'll be able to use this
[06:03]
and I'll show you how so looks like this
[06:05]
has been downloaded and now I can start
[06:06]
using the model if I want I can exit out
[06:09]
of this and if I want to switch between
[06:10]
the two different models again I just
[06:12]
type O llama run and then I can specify
[06:14]
the model that I want to use so if I
[06:15]
want to go back to llama 2 I use llama 2
[06:18]
if I want to go back to mistl I just
[06:21]
type mistl and now I can start using
[06:23]
Mistral so you can have as many models
[06:25]
as you want and again you can list them
[06:27]
by typing ol llama list and if you want
[06:29]
all of the commands you can use simply
[06:31]
type AMA and then it will show you which
[06:33]
ones you have access to there's a lot of
[06:34]
them for example you can also remove a
[06:36]
model if you want to do that copy a
[06:38]
model there's also customizations you
[06:39]
can make to them which I'll show you in
[06:41]
just one second all right so all of that
[06:43]
is great but we probably want to know
[06:44]
how to utilize these models from
[06:46]
something like code from our
[06:47]
applications sure they're great to use
[06:49]
here in the terminal but a lot of times
[06:50]
you want to integrate them with some
[06:52]
kind of software especially if you're a
[06:53]
programmer and you watch this channel so
[06:55]
the interesting thing about olama is
[06:57]
that it actually exposes an http API on
[07:00]
Local Host that means that anything we
[07:03]
just did here with commands we can
[07:04]
actually trigger through the API so we
[07:06]
can send request to this from something
[07:08]
like curl Postman something like python
[07:11]
code really any code at all that can
[07:12]
send some type of HTTP request Now by
[07:15]
default if you're running aama you
[07:17]
should be able to see this if you're on
[07:18]
Windows in kind of like the I don't know
[07:20]
what you would call this Services bar
[07:22]
wherever it's showing the running
[07:23]
applications and you can see I have this
[07:25]
little AMA logo now when olama is
[07:27]
running as the desktop application by
[07:30]
default that Port is going to be open so
[07:32]
you'll be able to access the HTTP API
[07:34]
but if for some reason this isn't
[07:36]
running so for example if I quit this
[07:38]
what I can do to trigger that to run is
[07:40]
I can simply type AMA serve in my
[07:43]
terminal if I do this it's now going to
[07:45]
start running the HTTP API in this
[07:47]
terminal instance and now I'll have
[07:50]
access to it and here it will also show
[07:51]
us what port it's running on although it
[07:53]
should be standard and you can see if we
[07:54]
look through here it gives us the exact
[07:56]
Port so it's on
[07:58]
11,434 so if you wanted to you can copy
[08:01]
that and save it for later so that we
[08:02]
can use it in our code regardless now
[08:05]
that the olama serve or the olama HTTP
[08:07]
API is running we're able to call it and
[08:10]
again just to clarify if you're running
[08:12]
this as the desktop application it will
[08:14]
already be running in the background but
[08:15]
if for some reason you want to manually
[08:17]
invoke this to run then you can run the
[08:19]
command ol llama serve where it will
[08:21]
give you all of this output and you'll
[08:22]
be able to view all of the requests to
[08:24]
the HTTP server so now that the server
[08:26]
is running we can use something like the
[08:28]
following python code here to send a
[08:30]
request to it now this is done manually
[08:32]
very intentionally I'm going to show you
[08:34]
an easier way to do this in 1 second but
[08:36]
it's just to illustrate that you do have
[08:37]
kind of complete control over this if
[08:39]
you want so you can see here in Python
[08:41]
I'm using the requests and the Json
[08:43]
module now just by the way if you want
[08:45]
this to work on your machine you will
[08:46]
need to install the request module so
[08:48]
you can say pip install request or pip
[08:51]
three install requests and I'm going to
[08:52]
leave this code in the description uh
[08:54]
Linked In A GitHub repo in case you want
[08:56]
to check it out now what we do is we
[08:58]
Define our base URL this is the URL of
[09:01]
the server and then
[09:02]
/i/ chat there's a lot of other
[09:04]
endpoints that you can use here and you
[09:06]
can even control deleting models adding
[09:08]
models Etc but in this case we just want
[09:10]
to chat with one of our models then we
[09:12]
can define a payload this is the model
[09:14]
that we want to chat with so in this
[09:15]
case I've gone with mistl and then we
[09:17]
can Define different messages here's a
[09:19]
standard message with the role of a user
[09:22]
next we can send a post request here
[09:23]
using request. poost to our URL with our
[09:27]
Chason payload which is this right here
[09:29]
and enable the streaming mode which
[09:30]
allows us to grab all of the responses
[09:33]
as they are typed this way we can grab
[09:35]
them in real time and we can show the
[09:37]
model actually typing the response
[09:39]
rather than waiting for the entire
[09:41]
response to be generated and then
[09:43]
viewing it now here's a little bit of
[09:44]
code just to handle that streaming data
[09:46]
for us so we're going through all of the
[09:48]
lines that are returned from this
[09:50]
response and then we are simply kind of
[09:52]
printing them out okay so I'm going to
[09:54]
show you what happens when I run this so
[09:55]
we already have requests installed and
[09:58]
if I go python sample request dopy just
[10:01]
wait one second here it will stream in
[10:03]
all of the data and then print it out so
[10:05]
you can see that it's kind of printing
[10:06]
it out line by line for us here as it
[10:08]
gets it and there you go python is a
[10:09]
high Lev language blah blah blah gives
[10:11]
us the answer if we go back to the API
[10:13]
we can see that the request was sent
[10:15]
here it took 4.1 seconds to process and
[10:18]
it returned to us that data sweet so
[10:20]
there you go that is how you utilize the
[10:22]
API manually but a lot of you probably
[10:24]
don't want to write all of this code so
[10:26]
instead we can use a very simple module
[10:28]
from python called you guessed it ol
[10:31]
llama so if you're using python or
[10:33]
JavaScript there are packages that will
[10:34]
do this for you so you can simply pip
[10:37]
install olama or pip three install olama
[10:41]
in your systems that you have this
[10:43]
module and now you have access to the ol
[10:45]
module you can simply create a client
[10:47]
you can Define your model you can Define
[10:49]
some kind of prompt and then you can use
[10:51]
client. generate specify the model and
[10:54]
the prompt and then you can grab the
[10:56]
response okay so I'm going to quickly
[10:58]
show this to you I can run this code
[11:00]
with python package. piy and you will
[11:05]
see here in just one second that we
[11:06]
should be able to get the response okay
[11:09]
and there you go we get the response and
[11:10]
it gives us the answer so that is how
[11:12]
you use the HTTP API now I'm going to
[11:15]
show you how you can do some
[11:16]
customizations to the models in ama so
[11:19]
moving on I'll show you a quick
[11:20]
customization that you can make to any
[11:22]
of the models that you can pull with AMA
[11:24]
so you can see on the right hand side of
[11:26]
my screen that I've created something
[11:27]
called a model file now I've just put
[11:29]
this in a directory that's on my desktop
[11:31]
you need to put the file in a location
[11:33]
that you know and that you're able to
[11:34]
access from your terminal and for the
[11:36]
model file I've used this very simple
[11:38]
syntax that I just took directly from
[11:40]
the AMA website all you do is you
[11:42]
specify from and then you have some kind
[11:44]
of base model so in this case we're
[11:45]
using llama 3.2 but you can use any
[11:48]
model that you want that's available
[11:49]
with a llama you can do something like
[11:51]
set the temperature of the model you
[11:53]
don't need to do this but there's some
[11:54]
other parameters you can set as well and
[11:56]
then you're able to pass something like
[11:58]
a system message which is essentially
[11:59]
kind of instructing the model what it's
[12:01]
supposed to be doing and how it should
[12:03]
handle the upcoming messages so in this
[12:05]
case they've just written Ur Mario from
[12:07]
Super Mario Bros answer as Mario the
[12:09]
assistant only okay so we have this
[12:12]
model file written notice I don't have
[12:13]
any extension it's literally just called
[12:15]
Model file no. txt or anything and what
[12:18]
I've done is I've put my terminal in the
[12:20]
same directory where this file exists
[12:23]
now what I'm able to do is create a new
[12:25]
model based on this model file in olama
[12:29]
and have one that's set up as Mario so
[12:32]
to do that I can type AMA create I can
[12:35]
give this a name in this case I'll call
[12:37]
it Mario you can call it anything that
[12:38]
you want and then I'm going to specify
[12:40]
DF which stands for file and then the
[12:42]
location of my model file now in this
[12:45]
case it's just simply at@ slm model file
[12:49]
okay so I'm going to go ahead and create
[12:50]
this and you'll see that it says success
[12:52]
that's because I've already pulled model
[12:54]
llama 3.2 so now if I want to utilize
[12:57]
this customized model what I can do is
[12:59]
type a llama run and then the name of
[13:01]
the model which is Mario and now if I
[13:04]
say hello you'll see that it says it's a
[13:06]
me Mario and it kind of you know
[13:08]
simulates like how Mario would reply so
[13:11]
if you want to set up some custom models
[13:12]
where they have some system prompts they
[13:15]
have some different parameters set up
[13:16]
with them you want to tweak them somehow
[13:18]
you can do that using these model files
[13:20]
then you can simply create them in olama
[13:22]
now let's say you're done with this one
[13:24]
and you want to remove it you can say RM
[13:26]
or sorry AMA RM and then what is it the
[13:29]
name of this one Mario and it will
[13:30]
remove that so now if we type oama list
[13:33]
you no longer see it and also it's worth
[13:35]
noting that these uh models like Mario
[13:38]
you can utilize them from code so in my
[13:40]
python code here I can just specify
[13:42]
Mario once that's created and then I can
[13:44]
use that anyways guys that is it that's
[13:46]
all I wanted to show you I hope you
[13:47]
found this valuable if you did make sure
[13:49]
to leave a like subscribe to the channel
[13:51]
and I will see you in the next one
[13:54]
[Music]