---
title: 'Running LLMs Locally Just Got Way Better - Ollama + MCP'
source: 'https://youtube.com/watch?v=GAyNvq6Ayps'
video_id: 'GAyNvq6Ayps'
date: 2026-07-28
duration_sec: 1311
---

# Running LLMs Locally Just Got Way Better - Ollama + MCP

> Source: [Running LLMs Locally Just Got Way Better - Ollama + MCP](https://youtube.com/watch?v=GAyNvq6Ayps)

## Summary

This video demonstrates how to run a local LLM on your own machine and connect it to external tools like Google, Notion, and Facebook Ads using Ollama and the Zapier MCP server. The goal is to achieve tool-use capabilities similar to Claude or OpenAI, but completely free, private, and secure.

### Key Points

- **Introduction to local LLM with tool use** [00:00] — The video promises to show how to run a capable local model and connect it to external services for free, private, and secure tool use.
- **Using Ollama and Zapier MCP** [00:35] — Ollama is used to run local models, and the Zapier MCP server connects to over 8,000 integrations, allowing the LLM to access external tools.
- **LLM vs AI agent explained** [01:13] — An LLM is a chatbot that predicts text; an AI agent can take actions by calling tools. Connecting an LLM to tools turns it into an agent.
- **Installing Ollama** [02:26] — Download Ollama from ollama.com and install it. Update with 'ollama update' command if already installed.
- **Choosing a model based on hardware** [03:45] — Model selection depends on GPU and RAM. Macs with unified memory can use RAM for models; Windows relies on VRAM. Newer devices perform better.
- **Selecting a tool-calling model** [06:00] — Models must support tool calling. Example: Qwen 3.5. Parameter count affects performance and RAM usage; choose based on available RAM.
- **Pulling and testing the model** [08:21] — Use 'ollama pull <model>' to download. Test with 'ollama run <model>'. Larger models may be slow; adjust parameter size as needed.
- **Setting up MCP client for Ollama** [10:58] — Ollama doesn't natively support MCP; use 'ollama-mcp' bridge. Install via pip: 'pip install ollama-mcp'.
- **Configuring Zapier MCP server** [11:37] — Create a Zapier MCP server, connect tools (e.g., Notion, Google Calendar), generate a token, and copy the URL with token.
- **Running the model with MCP integration** [15:55] — Run 'ollama-mcp --mcp-server <URL> --model <model>' to connect. Test tool calls like reading Notion or creating calendar events.
- **Using local model in code** [19:42] — Ollama exposes a REST API. Use LangChain with MCP adapters to build agents in code. Example: query calendar events.

### Conclusion

Running a local LLM with tool integration is powerful and relatively easy to set up. The trade-off is speed vs. accuracy, but it's a huge unlock for privacy and control.

## Transcript

If you want to run a local model that is
free, private, and capable of connecting
to all of your external tools like
Google, notion, Facebook ads, you name
it, anything that you want, then keep
watching this video. What I'm going to
show you how to do here is how to run a
capable local model on your own machine,
but more importantly, how to connect it
to external services so that you can get
the same tool use and advantages that
you would by using something like Claude
or OpenAI, but completely for free,
completely private and secure on your
own machine so you control everything.
With that said, let's get into the
video. Okay, so the way that we're going
to do this here is we're going to use
Olama. I'll talk about how to install it
and set it up. We're going to use a
capable model. We'll discuss the
criteria for using that in a second. And
we're going to use something called the
Zapier MCP server. This allows you to
connect to over 8,000 different
integrations. It is free to use. You can
just create an account and you can set
it up. And this means that all of your
integrations can be handled in one place
and you just connect a single MCP server
to O Lama, which again I'll explain to
you how to do and then you're good to go
and you can run a local model and
effectively do anything you would with
the better ones. Now, there are a few
things you should understand before
doing this. So, let me quickly go
through them. Now, first I'd like you to
understand the difference between an
LLM, a large language model, which is
kind of what we've been talking about,
and an AI agent. Now, an AI agent is
something that's capable of actually
taking action. And the way that it does
that is by calling various different
tools. An LLM or a large language model
is just kind of a standard chatbot. What
it's capable of doing is predicting
text, in some cases, calling tools or
generating videos, images, etc. But what
allows it to do that is you connecting
it to something external. So if you just
have you know chatgbt or you have I
don't know claude or enthropic or some
base model whatever if you don't connect
it to anything it can't really do
anything. It can just give you some text
back. Now the exact same thing applies
here when we're running these models on
our own machine. It's very easy to run a
local model on your own computer. You
can type with it. It can give you a
response back. That's cool. But we want
to add that tool layer so it can
actually take some actions and be useful
to us. So that's the key difference. The
LLM is kind of like the brain. It can
chat with you. It can do things. But
what allows it to do something is
connecting it to these tools. So when we
make that connection now, the LLM goes
from a chatbot to something that can
actually go out in the real world and
take actions on your data, which makes
it extremely valuable. Okay, so in terms
of setting this up, first thing we need
to do is download and install OAL. I'll
leave a link to in the description, but
you can probably find it faster. It's
olama.com. What you're going to do here
is just press the download button.
Download it for your operating system.
And if you already have it, I'd
recommend just updating it. You can do
that by running this command right here
in your terminal. Okay. Now, once you've
got Olama installed, go ahead and open
up a terminal or a command prompt. If
you're on Mac like me, you can just
search for terminal in the spotlight
search. If you're on Windows, go to the
Windows search and search for PowerShell
or for CMD or even now I think they've
called it terminal as well. Doesn't
really matter. Just open it up. From
here, you're going to simply type Lama
like so. Let's zoom in and just make
sure that this is working. Okay. Now, if
it's your first time running this, you
may need to just make sure that Lama is
started on your machine. To do that, you
can also just search for the Olama
application and just double click to run
it. And then it should run in the
background. Okay. Okay, so once you've
run O Lama here, you might see a screen
that looks something like this where it
says like, "Oh, launch these different
things." This is like a brand new update
they just released. For now, to get out
of it, you can just hit escape because
we're going to kind of pause on this for
a second cuz what we need to do is find
the model that we want to run. Okay, so
from here, if you want to see a list of
models that you can possibly run, just
go to this models tab and you can see
there are a ton of different ones and
you can start searching through them.
This is where we need to talk about kind
of realistic expectations. The type of
model that you can run on your computer
really depends on the performance of
your machine. And the main thing that
we're focused on is the graphics
processing unit or CPU that you have
depending on your operating system, Mac
versus Windows, and the amount of RAM
that you have. Now, what I'm about to
tell you is pretty much exclusively for
newer devices. So, I'm talking about in
the last like four or five years. If
you're running a much older machine, you
still can run these local models, but
you're going to be really limited in
what's possible just based on the
current architecture of like the old
devices versus the new devices. So, if
you're running a newer Mac on any M
series chip, then you should have
unified memory. That means the RAM
available to your computer is also
available to your graphics card uh or
graphics processing unit, whatever you
want to call it or whatever you know
Apple is calling it now. So, if you have
32 gigs of RAM on your machine, then you
can run models that utilize maybe like
70 or 80% of that RAM. Now, if you're on
a Windows machine or a Linux machine,
it's unlikely that you have unified
memory. Actually, I don't even know if
that's possible on Windows. I don't know
the current architecture, but what I do
know is that usually you're going to
have these models running on your
graphics card. So, if you have a 4090,
for example, then you should have 24 GB
of VRAM. Now, this is the memory that's
exclusively available to your graphics
card and can be used for running these
models. And the models can use pretty
much all of that. So, if you're on Mac,
what you want to look at because you
need to know your amount of RAM is,
okay, how much RAM or unified memory do
I have? If you don't have unified
memory, you're looking at the memory for
your graphics card. And if you're on
Windows, you're almost exclusively
looking at the memory for your graphics
card. If you don't have a graphics card,
you can still run these models, but
they're going to be extremely slow. So,
you could always run these models,
assuming you have enough hard drive
space. It's just a matter of like are
they actually usable and do they give
you any kind of output that doesn't take
like five hours to run because you can
run them but they're just going to be
really really slow if you don't have
enough RAM. And the reason for this is
that typically you load the entire model
and all of its weights into the
computer's memory. So this is what we're
focused on. Again, if you're on Mac,
it's a little bit easier for you because
you just look at the RAM amount. If
you're on Windows, you need to know your
graphics card, the amount of VRAM, and
again, newer devices are just going to
perform better. So, with all of that
said, when we choose a model here, what
we need to be looking for is one that's
capable of tool calling. If I just
search for like the Llama model here,
and I'm using like an outdated model,
something like Llama 3, you'll see that
Llama 3 doesn't indicate that it has the
ability to call tools. So, if I download
this and use it, okay, cool. But I can't
actually call any tools with it. So,
it's just not really usable, right, for
doing the integrations. Now, you can use
as a chatbot that, but that's about it.
So, when you're going to pick your
model, you want to pick a model that has
this tools ability. So, the one that I'm
going to use in this video is Gwen 3.5.
Very good, new model updated 2 weeks
ago. And you'll notice that when you
start looking at these models, they have
a ton of different options. So, if we
scroll down, you'll see we have one that
is 8 billion parameters, 2 billion
parameters, 4 billion, uh 9 billion, 27
billion. Sorry, this is 0.8 billion. And
the number of parameters directly ties
to the space that the model is going to
take up on your computer in terms of
storage space, but also in your RAM. So
the more parameters, the better
performing the model is going to be.
However, the more difficult it's going
to be to run. So when you're selecting
the version of the model, and let's say
you're using Gwen here, you need to be
conscious of how much RAM you have
available for the model to use. Now, in
my case, because I have 32 GB of RAM on
my machine. Let's just quickly show you
about this Mac. And this is unified
memory on an M2 Max uh like MacBook
here, I can use most likely the 35
billion or the 27 billion parameter
model. The 122 billion one, I can still
use it, but it's going to be incredibly
slow and just not practical to run
because it uses 81 GB, which means in
order for it to be efficient, I would
have to load all 81 GB into RAM. and my
RAM is also still being used by some
other processes on my machine. Okay, so
that's what you're paying attention to.
I'm just explaining that because the
model selection is the most important
part. For most of you, running like a 9
billion parameter model will work.
Again, it's not going to perform as good
as Opus 4.6, but it will still give you
decent performance and do some basic
things that you need. So, the more
hardware you have, the richer you are,
right? The better models you can run.
But that's kind of how it works. And
again, if you wanted to connect to
something else like a tool, you need
this tool calling ability. If you go
look at the models list, a lot of them
now do have the ability to call tools or
to do thinking or all of these different
things. So check those out. Okay. And
Gwen 3 coder next is also a good model
that's a little bit lighter that you can
run as well uh on your own computer.
Okay. So now that we know the model that
we want to use, what we can do is we can
type pull and we can just go find the
name of the model. So mine I'm just
going to go with maybe 35 billion and
I'm going to paste this here. So Gwen
3.5 col 35B pull is now going to pull
the manifest down and download the
entire model and all of its weights.
This is going to take a long time. Uh
well my case may not actually incredibly
fast here which I'm surprised by. You
can see it's downloading 23 GB. So I'm
going to wait for this to finish. Once
it's done I'll be right back and then
we're going to start running the model
and then I'll show you how to connect
the tools. All right. All right. So,
this just finished and now the first
thing I want to do is just quickly test
the model. So, to do that, I'm going to
type run and then Gwen and this is going
to be 3.5 colon 35 billion. And then
this is just going to run it directly in
a lama. I'm going to show you a better
way to run it in 1 second, but at least
this will allow us to test it. Should
take a second to load. Again, it does
need to load up RAM here with all of the
weights. And then as soon as it does
that, we can start chatting with it and
get like textbased responses. Okay, so I
just gave it a random prompt like, "Hey,
tell me what you're good at doing." And
let's see kind of the response time that
we're getting here from the model.
Again, the larger the model, typically
the slower it's going to be on your
machine, unless you have like a really
fast good machine. And in my case, you
can see this is taking a while. So, I
probably g downloaded a model that was a
little bit too large for the specs that
I had. Again, this one, I believe, is
using 24 GB of RAM. If I go down to the
27 billion parameter one, I'm more at
like 17, uh, which is a little bit more
manageable. So, if this takes too long,
I'll switch over to the other one.
Anyways, just want to show you the
reality here. I don't really know
exactly what's going to happen. You have
to kind of test it out and see which
models actually work on your machine.
Okay, so trying to run the other one
wasn't really working. I wasn't getting
any response after like a minute. So,
I've just switched to use the 27 billion
parameter model, which now is fine. It
also doesn't help that I'm recording a
video right now. And you can see that
it's giving me its live thinking process
where we have quite a few tokens per
second as the output. Now, again, this
is going to take a second to run, right?
it's going to be slower than you running
it in Claude, but the point is you're
running it on your own machine and, you
know, that's a huge advantage. So, you
play around with the different models to
determine what it is that you actually
need. So, it's doing this whole thinking
process. Hopefully, it's going to give
me, you know, like a valid answer here
in a second. Uh, but generally speaking,
it's working. And now we can move on to
the next step where we start doing the
integrations. Oh, and there you go.
Okay, it's giving me the answer as we
speak. And I mean, that didn't take too
long. That's pretty decent. And it looks
like it's giving me a pretty
comprehensive answer. So it's not just
like a super basic model. All right. So
continuing here, let's now go to the
tool integration component. Now, in
order for us to actually run MCP servers
inside of Olama, we need to use
something called the MCP client for
Olama. Now, there's a few other tools
and abilities to do this. But Olama does
not natively support MCP. So effectively
what we need to do is use something
called a bridge which is going to
connect to the MCP server, discover the
tools and then share those with the
llama in real time and just act as a
proxy for doing the tool calling and
getting the results. Now the best one
that I found is mcb client for a llama.
I'll leave a link to in the description.
Of course it's open source and available
and I'll just show you the commands you
need. You don't need to read this whole
thing. Now in order for us to use that,
like I mentioned, we do need to use the
Zapier MCP server. Again, massive shout
out to them for sponsoring this video.
This is free to use. The way that it
works in terms of pricing is that yes,
if you do use it a massive amount, you
will need to pay for it, but it lends
the same actions from your Zapier plan.
So, if you're familiar with Zapier, it's
very popular for automations. I use it
in all of my businesses. And it does
something called zaps. So, it like zaps
an automation or zaps something to a
platform. And each one of those zaps is
like one credit, right? Or yeah, one
zap. So, when you use the MCP server,
every time you use it, it just uh acts
as like one zap towards your plan, which
is totally fine. and you can get, you
know, like thousands of them for free
effectively without you having to pay
anything. So, the way that this will
work is just go to this link. It'll
leave it in the description. And what
you can do is just press get started.
When you do that, it should bring you to
a page that looks something like this
where it shows Zapier MCP. And you can
see like the number of tasks, right,
that are being used and that I've
currently used so far. Now, from here,
what we're going to do is go to new MCP
server. from the MCP server. We're just
going to select other, but you also
could connect this to any of the other
uh AI agents that you're using. So, it
doesn't just need to be what do you call
it? Um, what am I using here? A llama.
Okay. So, I'm going to go other because
this is just show that on here. From
here, I'm now going to connect some
different tools that I have. So, for
tools, let's go ahead and connect
something like notion cuz that's like
pretty visual and easy to see. And I'm
just going to select all of the tools
and just make them all available. But
you can obviously safeguard which ones
you want. So, I'm going to go ahead and
now connect my notion account. It's
literally as easy as just using the
OOTH. And then all of the security is
handled for you. So, what I'm going to
do is just connect it to my personal
account so that I don't really care if
it does something wrong. So, let's just
connect it to new page and this second
channel page and maybe this December
2025 travel. Uh, you can see I have just
all my travel stuff in here. So, let's
just select all of it and at minimum it
can maybe summarize some of the stuff
that we have inside of here. Okay, so
notion is now connected. Editors, please
just make sure you're blurring my email
there. Let's add all of the tools here
now to this MCP configuration. And just
because we're at it, we can just add
another one. So, let's add something
like I don't know, there's literally
8,000 of them. So, you kind of just have
to figure out what do you want and then
you can start searching them uh to find
anything like I don't know. Do they have
meta? Uh let's see here. Yeah, they have
like 100 meta things. Okay, let's try
maybe Google Calendar. Um and let's just
same thing. Select all. Connect. And let
me just connect it to one of my
accounts. All right, so I've got that
connected. Obviously, I can go crazy and
add as many of these as I want. But for
now, we'll just stick with these two.
And now, if we want to actually connect
this to Olama, we're going to go to this
connect tab here, and we're going to
generate a new token. Now, when we
generate the token, we've got to make
sure we save this. And obviously, you
don't want to share it with anyone else.
So, you can see there's a full URL with
the token. This is the one that I want,
so I'm going to copy that. But you can
also just connect um what is it like
through a standard MCP configuration
using the authorization header. A few
different ways. Depending on the tool
you're using, you'll use a different
one, but we want the URL with the token.
And we're just going to save that. So,
put that somewhere safe uh cuz we're
going to use that in 1 second. All
right. So, as I mentioned, the next step
here is we're just going to install this
Olama MCP client connector thing so that
we can actually connect to the MCP
server. So, in order to do that, you can
do this directly with pip. So, pip
install upgrade OLCP.
Then you can run oolcp. You can also do
uvxol mcp or you can run this
installation step right here. So, for
most of you, running it through pip is
probably going to be the easiest. And
the way you do that is just make sure
you have Python installed on your
machine first. Okay. Now, from here,
there's a ton of configuration options
once you've got this installed. I
already installed it, but effectively
all you would do, right, is just run
this pip command or run this uvx
command. I'll leave the link to this in
the description in your terminal. So,
you just go here, right, if you have uv
installed, and you go uvx and then ol
mcp, and it should just run, right? And
install all the dependencies, and you
should be good to go. Now, once that is
installed, okay, so it's spun up and you
can see by default it's using Gwen 2.5.
I'll show you how we change the model in
a second. Let's just quit so we can get
out of that in the meantime. Uh there's
a bunch of options you can use to run
this. So you can do uh what is it? - MCP
server where you're specifically
specifying an MCP server, which is what
we're going to do. You can do
autodiscocovery where it's going to look
in your clouds configuration actually.
And then you can specify the model, the
host, the version, all of that fun
stuff. So what we want to do is we just
want to specify the model and the MCP
server. So I'm just going to show you
the command to do this and then we can
run it. Okay. So effectively we're going
to run olcp-mcp
server URL. We're going to take our
server URL. I just stored it in my
browser which I know is horrible but for
the video that's fine. And I'm going to
paste this in. So put the URL including
the token. So we've just done that. And
actually, I believe we need to put this
inside of quotation marks, a set of
double quotes, just so it identifies
this as one argument. So, let's do that
and go back here. And then we're just
going to specify the model that we want
to run. And it's going to run an olike
environment for us. So, we're going to
go d-model and then Gwen 3.5 colon and
then 27B or whatever parameter version
you were using. And go ahead and hit
enter. So, again, MCP--
MCP server URL. put the server URL you
want to connect to d-model and we're
good to go. You can connect to multiple
servers if you want for MCP. However, if
you have Zapier, you just need one
because it connects to all of them by
default. Okay, so we're now going to do
that. It's going to try to connect to
this URL. You can see it's connected. It
now has access to these various tools.
It's also in thinking mode and you can
see that you can change a bunch of stuff
here. So, if you want to change the
thinking mode configuration, you can
type a TM. If you want to show the
thinking, ST. If you want to show the
metrics, SM. If you want to view the
various tools, type tools, right? So,
let's type T and we should be able to
see all of the different tools here. And
you can see it's exposing all of the
tools that we connected with the Zapier
MCP server. So, I'm just going to press
Q to get out of that. Okay. And it's
going to bring us back here. Show us
that we're in thinking mode. And let's
try something basic. Can you tell me
where I was traveling in the last year
based on my notion documents? Okay. And
then we're going to go ahead and press
enter. And you can see that it begins to
work. And now it should hopefully
connect to our MCP integrations, start
calling and using those tools and giving
us the result. Okay, so you can see it's
actually prompting me if I want to
enable the tool call. You can see that
it's going to go travel and start
searching in notion to find this page by
the title. So let's go ahead and type Y.
Now again, I'm not going to lie to you.
This is slow. It's not speedy execution.
You're going to sit here for a little
bit, but this is the kind of tradeoff
when you're running on your own machine.
If I had a really highend machine, which
by the way I'm considering buying soon
because I can just run a ton of local
models. This would be a lot faster. I
can use a better parameter model. But
also I could just go down to like the
nine billion parameter model and then
this is going to be lightning fast. I
might get a little bit less accurate
replies, but that's the trade-off. So
play with them. Let me know in the
comments down below, but let's wait for
this to finish and make sure it can
actually kind of do what I'm saying.
Okay, so it did take a second here.
There was a lot of tool calls that it
ran through, but it was able to get all
the content. And we can see now that it
has all of the details here and it's
starting to show where I was traveling,
what flights I was taking, all of this
kind of stuff that I was doing back in
December 2025. So was able to go through
notion. It was able to do that. Let's
quickly test one with the calendar. And
then I want to show you how we can do
this from code. Can you create a new
booking today at 4 to 5:00 p.m. That is
just a time block saying eat lunch in my
calendar. Let's just test something
simple and just see if it can like take
the action rather than just reading some
content. Let's test it. Okay, so it's
telling me here that it did create the
event. Uh, let me go check my calendar
and see if that's true. Okay, so I did
find it. It is at 4 to 5:00 a.m., not 4
to 5 p.m. So, let's just tell it to fix
that. Hey, the events at 4 to 5:00 a.m.
Move it to 4 to 5:00 p.m. Okay, probably
just based on how it called the tool.
Again, we're not using the best model
possible, but if we scroll through here,
we can see all of the stuff. And I'm
also guessing that maybe this is a time
zone issue because it's probably doing
this in Eastern time, which my calendar
is set to and I'm currently 12 hours
ahead of that. Okay, so that's it
probably actually did do it at the
correct time. It's just uh in the wrong
time zone awareness area. So anyways,
we'll skip this for now. I want to go
over the code. I want to show you how we
can do this inside of code as well, not
just from this kind of terminal
environment in case you want to run a
local model through something like lang
chain. All right, so I've just opened up
cursor. I've got some code here. I just
had claude code generate this. So it's
probably, you know, a more optimal way
to run it. But effectively, I just want
to show you that from code, you can just
directly connect to Lama because Lama
exposes a REST API server. And then from
there, if you want to use an
orchestration framework like Langchain,
you can really easily connect to an MCP
server. So you can see like I've defined
my Zapier MCP URL. I define it as a
client. I then just create a React agent
using Langchain here. I think there's a
newer version, but this one works just
fine. I get all the tools, pass it
there, and then I can just start
chatting with my agent directly inside
of code. So if you're building a
product, you want to run a local model,
you have high enough hardware, you can
do it really easily. I just have a few
packages, right? Langchain MCP adapters,
lang chain, a llama, langraph in Python.
You can do it in any language you want.
And you don't even have to use these
packages. They're just an abstraction on
top of the HTTP server or REST API that
is exposed. So you can just directly
call a llama, call your model, and then
connect it up to the MCP server like I'm
doing right here. So, I'm just running
it and I'm just going to go like, you
know, actually what calendar events do I
have in the next, I don't know, 5 days.
Okay. And let's see what it gives us.
Okay. So, same thing. It's a little bit
slow, but I got the response back here.
You can see there's a bunch of data. I'm
going to have to blur some of it because
there's some custom stuff. So, um,
editors, please just like blur any of
the important information, but
effectively it summarized all of the
meetings that I have coming up here, and
these are accurate based on what I know
about the calendar. There we go, guys.
With that said, that's going to wrap
this up. This is super powerful,
relatively easy to do. Yes, there's a
few commands and setup, and once you get
it, it's pretty much just plugandplay.
And I mean, this is a huge unlock if you
want to run something locally or you
want to build anything that relies on
local models. Again, play with them, try
out different models, see what kind of
speed, accuracy combo you can find. Let
me know what you think of the video, and
I look forward to seeing you in the next
one.