[00:00]
Local models just got a massive upgrade.
[00:03]
Just over the past two months,
[00:06]
drop that are very capable
[00:09]
Even on just mid-tier hardware.
[00:11]
So in this video, I'm going to show you
[00:15]
and then connect them to tools like OpenClaw
[00:19]
of dollars per month and no longer need to rely on
[00:23]
Now, I am going to explain this extremely in-depth
[00:26]
You need to pick the correct local model,
[00:30]
It really depends on your use case,
[00:32]
what you're looking for, security,
[00:35]
Anyways, let's dive in.
[00:37]
So the experience that you're going to have running
[00:40]
on the model that you choose
[00:44]
Now I want to go over both.
[00:45]
And while it's going to seem a little bit
[00:49]
because making the wrong decision here
[00:53]
to go back to using these cloud based models,
[00:57]
Now, the first thing to understand
[01:00]
are open source,
[01:03]
What that means is that these are models
[01:06]
that are free to use, that
[01:10]
Okay.
[01:11]
So you're not going to be able to run,
[01:12]
you know, opus 4.7 locally on your own computer
[01:16]
And they want you to pay them a massive amount
[01:21]
Same with GPT 5.4 or whatever the newest model is.
[01:24]
You can't run that locally.
[01:25]
You have kind of a limited selection
[01:29]
And when you are picking those models,
[01:32]
that are going to be compatible with tool
[01:37]
like Open Claw,
[01:40]
now in order to determine
[01:43]
you do need to understand
[01:46]
These models are very demanding on the hardware
[01:49]
and specifically on the Ram and the graphics
[01:55]
So before we go much further, I'm
[01:58]
the specs of the computer
[02:02]
If you're working with a higher end
[02:03]
MacBook, what you're going to want to do
[02:07]
go to about this Mac, and you're going to be looking
[02:12]
Now assuming that you have a mac,
[02:15]
This is the unified memory that you have available
[02:21]
your GPU, and effectively
[02:25]
It's not going to be able
[02:27]
In this case, I have 32GB, and probably the highest
[02:33]
Again, we'll look at that in a second.
[02:34]
But what you want to find is, okay if I'm on Mac,
[02:38]
Again,
[02:41]
if you have a machine that's six, seven,
[02:45]
to run local models, and you're
[02:49]
just because they're going to be extremely slow
[02:53]
If you're on windows,
[02:56]
If you're on windows or even a Linux device,
[03:00]
or your graphics processing unit,
[03:04]
So the Ram in your computer doesn't really matter.
[03:06]
It's more about just the Ram. Again,
[03:09]
So if you're running a 4090, for example,
[03:13]
If you're running maybe an older GPU,
[03:17]
Now, typically speaking, these local models
[03:23]
So whatever that number is,
[03:27]
or performance of local model
[03:30]
Now we're going to get into all of the details here,
[03:34]
on my on for 99% of you it's going to be Mac
[03:38]
If you're on Mac,
[03:41]
If you're on a newer Macs on M-series Mac,
[03:44]
then you're going to be looking at the total amount
[03:48]
processing unit, assuming that that's an Nvidia GPU.
[03:51]
And again, you're ideally going to want
[03:55]
But even if you're running on a
[03:59]
available in the GPU or again on a newer series
[04:04]
Okay, so now that we have that out of the way,
[04:08]
setting this up on our machine.
[04:09]
So what we're going to do in order to run
[04:13]
called Ollama.
[04:14]
Ollama allows you to pull down different models
[04:19]
The only limitation for running these models
[04:23]
it's completely free.
[04:23]
You don't need to pay for anything
[04:26]
like Open Claw,
[04:29]
However, before we go any further, I want to make you
[04:31]
aware of a really cool opportunity,
[04:35]
It's called ClawComp.
[04:36]
Now Link Ventures is running it,
[04:39]
a free Mac mini to build with Open Claw,
[04:43]
You can keep it no matter where you finish.
[04:46]
No matter what happens,
[04:49]
Now here's how it works.
[04:50]
It's a multi-month build program, so it's designed
[04:54]
You apply with a team of 1 to 3 people.
[04:57]
You hang out in their discord, you talk in their
[05:01]
And then you get three months to actually build
[05:05]
So not a wrapper, not a demo that falls apart
[05:08]
in an actual automation with measurable outcomes.
[05:11]
Now the whole thing ends
[05:13]
This is from June 15th to June 18th in Cambridge,
[05:17]
At Link Studios.
[05:18]
They fly you out,
[05:19]
they cover the housing, and you spend four days
[05:23]
Now, founders and investors from the link ecosystem
[05:26]
Researchers from Harvard
[05:29]
companies are walking around meeting teams
[05:32]
Lock now the prize for $17,500
[05:38]
So if you're a student building something
[05:42]
that you've been meaning to take seriously,
[05:45]
for three months with a real deadline
[05:49]
It's an awesome opportunity. Again,
[05:52]
Link is in the description.
[05:53]
Go apply, get in the discord stand out
[05:58]
Okay, so that said, let's get back to the video here.
[05:59]
I want to go through the setup process.
[06:01]
So the first thing we're going to have to do here,
[06:05]
and if you're running on a virtual private server,
[06:08]
You're going to go over to your terminal
[06:10]
and you're just going to run this command
[06:13]
I'm going to leave a link to in the description,
[06:16]
You're going to copy the install command.
[06:17]
And even if you already have Ollama installed,
[06:22]
because you will need the newest version in order
[06:26]
Okay, so you open a terminal or command prompt.
[06:27]
You paste this command, you hit enter
[06:31]
Now like I said, Ollama is just a tool runs on
[06:36]
So we're going to wait for this installation
[06:38]
Once it's done, I'll be right back and then we're
[06:41]
And then once we have the local model running, we're
[06:46]
And good timing.
[06:47]
It looks like it's already installed.
[06:48]
While I was just doing that speech.
[06:50]
Okay, so we have Ollam installed now
[06:54]
We're just going to type the Ollama command.
[06:55]
If for some reason this Ollama command
[06:59]
and reopen it or close your command prompt
[07:03]
and you should just see something popping up
[07:08]
And then what you can do
[07:10]
Okay, get out of this interactive window.
[07:13]
Now if you're on a virtual private server,
[07:18]
you're going to want to make sure
[07:22]
that I talked about before, right?
[07:24]
So in the case of a Linux machine, you're going
[07:29]
If you have that, then you're going to be able
[07:33]
So the steps that I'm showing you here
[07:35]
work on any operating system,
[07:38]
whether it's on a VPNs,
[07:41]
hardware in order to have a decent experience
[07:44]
Okay, so now that we have Ollama installed,
[07:46]
what we need to do is select
[07:49]
So what I'm going to recommend is that we use
[07:54]
For now. There's a bunch of other models
[07:57]
At least when I'm filming this video.
[07:58]
This is the current, smallest and best model
[08:03]
But what we're looking for
[08:05]
is that they have the ability to cull tools
[08:09]
Specifically, you need this.
[08:10]
You don't just want a chat based model,
[08:14]
All of these different modes like Gemma for has.
[08:17]
And if you want to see all the different models
[08:22]
or just go to the models tab and you can look
[08:26]
For example Gwen 3.6.
[08:28]
Also another great model,
[08:31]
So I'm going to recommend
[08:34]
So once we've selected that Gemma four
[08:38]
Is this Ollama run or Ollama pull?
[08:41]
I'm going to show you the command and a second
[08:42]
Gemma for this is a command
[08:46]
But before you do that, you need to select
[08:51]
Now you'll notice that if we go to the models down
[08:55]
We have Gemma for latest, Gemma for 2 billion, Gemma
[08:58]
for 4 billion, Gemma
[09:02]
And then the cloud one
[09:05]
Now the billion value here
[09:09]
The larger the amount of parameters,
[09:10]
the better performance of the model,
[09:14]
So really the thing that you want to look at here
[09:18]
So we see nine gigabytes, seven gigabytes,
[09:22]
Now you want to make sure that this size is smaller
[09:26]
that you have in your computer
[09:29]
Again, based on what I talked about before,
[09:33]
I have 32GB of Ram, so I'm fine to go all the way
[09:39]
want to go much higher than that because the bigger
[09:43]
And again,
[09:47]
Now theoretically, you can run any model you want,
[09:51]
as long as you had enough hardware space.
[09:52]
But it's going to be so incredibly slow
[09:54]
that you're not going to be able
[09:57]
So that's why I'm emphasizing
[10:00]
that it fits into the Ram
[10:04]
So that's what we're looking for.
[10:05]
Specifically, you want to pick the biggest model
[10:09]
Okay.
[10:10]
So I'm going to go with let's just go with the 31 B.
[10:13]
Right. Because that's going to work for my machine.
[10:14]
Even though it might be a little bit slow.
[10:16]
And what I'm going to do
[10:18]
and I'm going to type the command Alama pull.
[10:21]
And then I'm going to paste this Gemma for 31 B.
[10:25]
Now, for most of you, you're
[10:27]
You might go with 4 billion right.
[10:29]
Or E 4 billion E 2 billion,
[10:32]
So you might put E to be right.
[10:34]
Whatever the name is that you see here,
[10:37]
what this is going to do
[10:40]
It's going to pull all nine, 15, 20GB, whatever.
[10:43]
And once that's done we're good to actually right now
[10:48]
So I'm not going to run this command.
[10:49]
But if you don't again you need to download it first.
[10:51]
It might take a few minutes
[10:53]
Now once it's downloaded,
[10:57]
that you have available is to type
[11:00]
When you do that, it's going to give you a list
[11:03]
You can see that
[11:06]
and this is the one that I'll end up
[11:09]
But you can see all of these.
[11:10]
You can also just go Ollama help.
[11:12]
And if you do that, it's going to give you a list
[11:15]
where you can create a new model, you can pull
[11:19]
There's all kinds of advanced stuff.
[11:22]
The point is, awesome is super
[11:25]
Okay, so now that the model is installed, we can just
[11:29]
And then we're going to go with whatever the name is.
[11:31]
So my case I'm just going to go Gemma for
[11:35]
installed and it should take a second here
[11:39]
So I'm just going to go hello world.
[11:40]
And then you can see immediately
[11:45]
You know I'm good.
[11:45]
How are you? Right. Whatever I'm doing. Well.
[11:47]
And then it gives me the response and you can see
[11:49]
this is actually quite fast
[11:52]
just the nine gigabyte model on my machine,
[11:56]
If you want to get out of this window,
[11:59]
This is just kind of a terminal based view
[12:02]
if you want to do that.
[12:03]
Okay, so now that we have the model installed
[12:08]
So first we need to make sure we have OpenClaw
[12:10]
And again
[12:13]
that virtual private server
[12:17]
And you would follow the same steps I just did
[12:22]
So what I'm about to show you,
[12:26]
Okay, I'm going to assume that you have it installed
[12:30]
Now what you can do right is we go to open
[12:34]
So I have OpenCL installed on my machine.
[12:36]
And literally all we need to do to get Ollama
[12:42]
configure.
[12:43]
Okay.
[12:44]
So assuming this is installed on our machine right
[12:48]
But for this tutorial I will show you doing it right
[12:50]
We're going to type OpenCL configure.
[12:52]
Maybe you guys have a mac mini or something.
[12:54]
So you're going to do that directly on there.
[12:56]
And what we're going to do is go through
[12:59]
So where it says model we're going to use our arrow
[13:01]
We're going to press enter.
[13:02]
And we're going to select down here.
[13:04]
Mine's a little bit laggy when it first pops up.
[13:06]
But we're going to go down
[13:10]
You should see it popping up as an option.
[13:12]
Now once it pops up
[13:15]
We don't want to use the cloud one.
[13:17]
I'm not going to get into that in this video.
[13:18]
Ollama has a cloud offering,
[13:21]
We're going to go local only.
[13:22]
We're just going to leave the base URL as it is.
[13:24]
We don't need to change this and continue.
[13:27]
And we're going to select the model
[13:30]
Now I apologize for the cut here,
[13:33]
for some reason Ollama is not appearing or showing,
[13:36]
and there's some issue, then what
[13:40]
You can hit Ctrl C
[13:44]
Now when you run Ollamaserve, this is just going
[13:48]
It's going to run directly in this terminal window.
[13:50]
So just don't close it.
[13:52]
Now you probably don't need to do that.
[13:54]
You also can just go to your spotlight search.
[13:56]
If you're on something like Mac, and you can just run
[14:01]
and you should see you get like a little Ollama
[14:04]
meaning that it's running in the background.
[14:06]
Okay, so you can see we have a bunch of options here.
[14:08]
I'm just going to check this “ollama/gemma4:latest”
[14:12]
You can select all these different models.
[14:15]
Because these are all the ones
[14:18]
And I'm just going to go ahead and press on confirm
[14:21]
So you just select the model that we just installed
[14:24]
For most of you it's going to be that gem of four.
[14:26]
Okay. So we're going to go ahead and press enter.
[14:28]
And now both those models or whatever ones
[14:32]
So I'm going to press on continue.
[14:34]
And then what we're going to do
[14:35]
is just restart the gateways
[14:39]
We're gonna go open core
[14:44]
Now when we run that command it's
[14:47]
Both these models should then be available.
[14:49]
And then what we can do is just start using them
[14:52]
Now, the same thing goes for windows.
[14:54]
You can run the Ollama serve command,
[14:57]
like application search bar or whatever
[14:59]
And then you can just run Ollama and it should run
[15:03]
Okay. So you want to make sure that's running
[15:06]
Now once we do that
[15:08]
Again I'm going to assume you know how to get here
[15:12]
cloud before.
[15:13]
And what we can do is just asking the question so
[15:17]
Can you tell me the meaning of life?
[15:18]
Right.
[15:19]
And you'll notice that by default,
[15:22]
using GEMA four.
[15:23]
And if we go enter here
[15:27]
And you can see we get the response very quickly.
[15:29]
Now if for some reason you have different models
[15:31]
probably want to set the default model
[15:35]
In order to set the default model,
[15:39]
I don't know exactly where the configuration
[15:43]
cloud dot json file, there is a way to set
[15:49]
And then you're kind of good to go.
[15:51]
Now, what I typically will do
[15:53]
when I'm using this is I might have a local model
[15:57]
And then I also configure a cloud model
[16:01]
more challenging,
[16:04]
because while these models are good,
[16:08]
or opus 4.7,
[16:12]
unless you purely just care about privacy
[16:18]
So my suggestion is you configure like an open
[16:22]
and then the local model,
[16:25]
and then you switch to the other model
[16:28]
Now what you can do is you can actually type
[16:31]
Right. And then you can change the model
[16:33]
So I just do slash models.
[16:34]
It will give me a list.
[16:37]
And then I can say slash model.
[16:38]
And then you know Gemma for right.
[16:41]
And then switch over to actually use that model.
[16:43]
In my case
[16:45]
But you also can just directly tell it, hey,
[16:50]
Can you switch the model and then open?
[16:52]
Close should be capable of just running the command
[16:55]
You can see it called the tool
[16:58]
Now in this case it's saying you can't do it again
[17:01]
By the way,
[17:01]
if you are wondering what I'm using to dictate here,
[17:06]
I can quickly click into it
[17:09]
if we go to the settings, but effectively
[17:14]
It uses AI in the background extremely fast,
[17:18]
They're free to try out and I just use them anyways,
[17:21]
You could see my words per minute here,
[17:23]
and I'll leave a link to in the description
[17:25]
but especially if you're working with a lot of these
[17:30]
rather than to type, right?
[17:31]
So if I want to say something, hey,
[17:33]
Here's a bullet pointed list of Abcde.
[17:36]
Right?
[17:36]
And then, you know, we go and it just gives me
[17:39]
it, fix the spelling, punctuation,
[17:43]
So anyways, that's pretty much all that I have for
[17:46]
Setting it up with OpenClaw is very easy.
[17:48]
It's a matter of having Ollama installed
[17:51]
If you want to go further,
[17:55]
as well as your agent start MD file
[17:59]
and you can tell it when to use the local model
[18:03]
You can install multiple local models. Right.
[18:06]
So maybe you want Gemma for for something.
[18:08]
Maybe you want Gwen for something else.
[18:09]
Maybe there's a specific image or video
[18:13]
You can go crazy with the configuration.
[18:15]
The important thing is really understanding
[18:18]
And again, the performance
[18:21]
is going to be dictated by that hardware.
[18:23]
In combination with that model selection.
[18:25]
Probably you just want Gemma,
[18:28]
and see what experience you get.
[18:30]
In my case, Gemma is much, much, much faster
[18:35]
that are currently out there.
[18:36]
So that's it guys. I'm gonna wrap up the video here.
[18:38]
If you enjoyed, make sure they like subscribe