[00:06]
Can you actually run a real LLM
[00:09]
yourself? No OpenAI key, no Anthropic
[00:12]
key, no per token bill, just a model
[00:14]
running on a server you control. Yes,
[00:17]
and in the next few minutes we're going
[00:19]
to set one up, expose it as an API, and
[00:21]
build a small chat app that talks to it.
[00:24]
There are four real reasons people do
[00:25]
this: privacy, predictable cost, working
[00:28]
on private networks, and version
[00:30]
control. We'll also talk about when
[00:32]
self-hosting is the wrong call, because
[00:34]
there are plenty of those cases, and
[00:36]
pretending otherwise wastes your time.
[00:38]
Quick distinction before we touch
[00:39]
anything. These two things get mixed up
[00:41]
constantly. Ollama is the runtime. It's
[00:45]
the program that loads a model into
[00:46]
memory and handles requests. Think of it
[00:49]
like a JVM. By itself, it doesn't do
[00:51]
anything useful, but it knows how to run
[00:53]
things.
[00:54]
Gemma, Llama, Mistral, QN, those are the
[00:58]
models, the actual neural networks,
[01:00]
multi-gigabyte files of weights that
[01:02]
produce text. You can swap models
[01:05]
anytime. Pull a new one, point Ollama at
[01:07]
it done.
[01:08]
Could you do this on your laptop? Sure.
[01:11]
Should you? Probably not. Laptops sleep,
[01:14]
models eat RAM, and you don't want a 4
[01:16]
billion parameter model warming your CPU
[01:18]
mid-call. So, a server, always on,
[01:21]
reachable with enough RAM to hold a
[01:23]
model without choking.
[01:25]
So, for this tutorial, I'll be using a
[01:27]
Hostinger VPS, and I want to walk
[01:30]
through why, because the plan choice is
[01:32]
important here. This isn't a generic any
[01:34]
server works situation. Memory and CPU
[01:37]
matter here, more than for a regular web
[01:40]
app.
[01:40]
This is Hostinger's VPS hosting page,
[01:43]
their AI managed VPS line.
[01:46]
Before we continue, a few things are
[01:47]
worth calling out.
[01:49]
Full root access, because Ollama runs as
[01:51]
a system service, we need to install it,
[01:54]
expose ports, and adjust firewall rules.
[01:57]
Locked-down hosts won't let you do that.
[01:59]
NVMe SSD storage, that's relevant
[02:02]
because model files are big. Gemma 3 4B
[02:05]
is roughly 3.3 gigs on disk. A 7B model
[02:08]
is around 4 to 5. Loading from a slow
[02:11]
disk means slow cold starts. AMD EPYC
[02:14]
processors, KVM virtualization. KVM
[02:18]
gives you a real virtual machine with
[02:20]
allocated CPU and RAM. Stronger
[02:22]
isolation than the kind of shared
[02:24]
container hosting where someone else's
[02:26]
workload can steal your CPU. The
[02:28]
one-click LLM template, specifically an
[02:31]
Ubuntu image preloaded with Ollama, the
[02:33]
Llama 3 model as a default, and Open Web
[02:36]
UI as a browser chat interface. We'll
[02:38]
use the template, skip the default
[02:40]
model, and pull Gemma 3 ourselves. Still
[02:43]
saves us 5 minutes of setup. About the
[02:45]
plan. There are four KVM tiers. I'm
[02:48]
picking KVM 2, two vCPUs, 8 gigs of RAM,
[02:51]
100 GB NVMe. Gemma 3 4B in its 4-bit
[02:55]
version uses about 3 to 5 gigs of RAM in
[02:57]
practice. The model file is 3.3 gigs on
[03:00]
disk plus context as it grows. OS and
[03:03]
runtime take another 1 to 2. That leaves
[03:05]
a gig or two of headroom on eight.
[03:07]
Tight, but it's enough for a 4B model
[03:09]
and a small app on top. If you want to
[03:11]
step up to 7B models, that's KVM 4
[03:14]
territory. 7B in 4-bit lands in the 5 to
[03:17]
7 gig range, which doesn't fit cleanly
[03:20]
on eight. So, KVM 2 if you're staying
[03:22]
with 3B or 4B, which is where we're
[03:24]
going for this build. KVM 4 if you want
[03:27]
to experiment with 7B.
[03:29]
If you want to follow along, the coupon
[03:31]
LTS10 takes 10% off whichever plan you
[03:34]
pick. Link is hosting.your.com/lts10.
[03:38]
Also in the description.
[03:40]
After provisioning, you land in Hpanel.
[03:43]
From the VPS section, head into OS and
[03:46]
panel, click operating system, pick
[03:49]
Ubuntu, then select applications, and
[03:53]
pick Ollama, and the template will take
[03:55]
care of VPS setup. H panel gives you the
[03:58]
VPS overview, free weekly backups,
[04:02]
firewall settings, a browser-based
[04:04]
terminal, meaning no SSH key fiddling if
[04:07]
you don't feel like it. One click and
[04:09]
you're inside the box. That icon is
[04:12]
Cody, their AI assistant. You can hand
[04:15]
it a task in plain English and it'll
[04:17]
actually run the VPS commands itself. No
[04:20]
copy-paste needed. Worth knowing about,
[04:23]
even if we won't use it today.
[04:26]
Because the template ships with Ollama,
[04:28]
we can just verify it's there. It prints
[04:30]
the version, so we're good. If you're on
[04:32]
a plain Ubuntu install instead, this
[04:34]
one-liner will do the job. Same outcome,
[04:37]
60 seconds longer.
[04:44]
First, let's see what's already there.
[04:47]
The template came with Llama 3 already
[04:49]
pulled.
[04:50]
Fine model, but we're going with Gemma 3
[04:53]
4B for this build. Google's open-weight
[04:55]
model. Smaller, fast on CPU, fits on our
[04:59]
8-gig box. The first pull is the slow
[05:01]
part.
[05:02]
About 3.3 gigs of the wire, I've sped
[05:05]
this up. After it's local, it's local.
[05:08]
You won't pull it again.
[05:10]
Ollama list confirms what's installed.
[05:12]
Both models sitting there ready to go.
[05:15]
To actually talk to it,
[05:17]
the first request takes a few seconds,
[05:19]
the model has to load into RAM. After
[05:21]
that, the next prompts are fast.
[05:23]
There it is. The model we just pulled
[05:25]
replying from our own server. Nothing
[05:28]
external touched.
[05:29]
Now, the cost. What's this taking in
[05:32]
RAM?
[05:33]
With the model loaded, we're using about
[05:35]
4 to 5 gigs of RAM, more than half our
[05:37]
8-gig box. Self-hosting isn't free.
[05:40]
Instead of paying per token, you're
[05:42]
paying a fixed monthly bill for the
[05:44]
server. If your traffic is bursty small,
[05:47]
hosted APIs are usually cheaper. If it's
[05:51]
steady or you've got privacy
[05:52]
constraints, self-hosting wins.
[05:55]
Now, we can call it from code. Ollama's
[05:58]
been serving a REST API on port 11343
[06:01]
the whole time. Nothing else to start,
[06:03]
nothing else to configure.
[06:05]
We hit the generate endpoint with a
[06:07]
simple prompt. Standard JSON, model
[06:09]
name, generated text, token counts,
[06:12]
timings, the usual fields you'd want.
[06:15]
Ollama also exposes an OpenAI compatible
[06:17]
endpoint at v1/chat/completions.
[06:21]
If you have code using the OpenAI SDK,
[06:24]
just point its base URL at your server.
[06:27]
The basic chat completions endpoint
[06:29]
works. One-line change and you've moved
[06:31]
off OpenAI. Not every OpenAI endpoint is
[06:34]
mirrored, but the everyday ones are
[06:36]
there.
[06:37]
If you ever open port 11343 to the
[06:40]
public internet, add authentication.
[06:42]
Ollama doesn't ship with off by default.
[06:45]
The HPanel firewall section is where
[06:47]
you'd lock things down. Whitelist your
[06:49]
app's IP or put a reverse proxy in front
[06:52]
with an API key.
[06:53]
For this video, we're keeping everything
[06:55]
on the same server. The app and Ollama
[06:57]
will talk over localhost, so we don't
[06:59]
need to expose the port at all.
[07:03]
Time to build something.
[07:05]
Keeping it deliberately simple, a chat
[07:07]
UI that streams responses from our
[07:08]
model. Node and Express on the back,
[07:11]
plain HTML on the front, around 80 lines
[07:13]
total, and it'll teach you the pattern
[07:15]
you can use in anything bigger.
[07:17]
Make the folder, init a package, install
[07:20]
Express. Done.
[07:22]
Now the server. Standard Express setup.
[07:25]
We serve a static front end from a
[07:27]
public folder and accept JSON bodies.
[07:30]
Nothing fancy.
[07:31]
One endpoint, post chat. It takes a
[07:34]
prompt from the request body, forwards
[07:36]
it to Ollama with stream true, and pipes
[07:39]
the stream of tokens back to the browser
[07:42]
as they arrive.
[07:43]
Ollama uses newline delimited JSON for
[07:46]
streaming. Each line is one chunk of the
[07:49]
response.
[07:50]
We buffer the chunks, split on new
[07:53]
lines, parse each line, and write the
[07:55]
response field back to the browser. The
[07:58]
browser sees tokens appear in real time,
[08:00]
same feel as ChatGPT.
[08:02]
The buffer is required as TCP chunks
[08:05]
don't always end on a JSON line. One
[08:08]
line can get split across two chunks.
[08:11]
Without it, you'd hit JSON.parse errors
[08:13]
on partial lines, maybe one request in
[08:16]
15.
[08:17]
Failures that look random until you spot
[08:19]
the cause.
[08:22]
The front end is very simple. Text area
[08:24]
for the prompt, a div for the output, a
[08:27]
button.
[08:28]
The script does one thing. Fetch to our
[08:30]
chat endpoint, read the response body as
[08:32]
a stream, and append each piece to the
[08:35]
output div as it arrives. That's the
[08:37]
full client.
[08:39]
Start the server, open the VPS IP on
[08:42]
port 3000. I'll ask it for a short poem
[08:44]
about a tired developer at 2:00 a.m. Hit
[08:46]
send and tokens stream back in real
[08:48]
time.
[08:49]
The full loop. Your code, your server,
[08:52]
your model. No external API in the
[08:55]
picture, no token meter anymore.
[08:58]
From here, you'd add chat history,
[09:01]
system prompts, maybe RAG over your own
[09:03]
documents. Same loop underneath, just
[09:06]
more layers on top.
[09:09]
Before anyone uninstalls the OpenAI SDK,
[09:12]
let's talk about the tradeoffs. Pick the
[09:14]
wrong tool for the job and you'll regret
[09:16]
it.
[09:17]
Self-hosting wins for learning, for
[09:19]
privacy-sensitive work, for side
[09:21]
projects and internal tools, for
[09:23]
predictable costs at low to medium
[09:24]
volume, for offline use, and for pinning
[09:27]
a model version so the vendor can't
[09:29]
change it on you mid-project.
[09:31]
Hosted APIs still win for frontier
[09:33]
capability, for high-throughput
[09:35]
production, for the zero ops overhead
[09:38]
and for the largest models you wouldn't
[09:40]
realistically run yourself.
[09:42]
A 4B open model handles plenty of jobs.
[09:45]
The largest closed models still pull
[09:47]
ahead on the hardest ones. Pick the
[09:49]
right tool for the job you're trying to
[09:50]
do.
[09:51]
Different tools for different jobs. Most
[09:54]
real systems end up using both. Hosted
[09:56]
APIs for the hard reasoning, self-hosted
[09:59]
for high volume traffic or anything
[10:01]
privacy sensitive.
[10:03]
That's it.
[10:05]
Pull a model, run it, expose it on a
[10:07]
port, point an app at it. Once you've
[10:10]
done it once, you can do it for any open
[10:12]
weight model. Llama, Mistral, Qwen,
[10:15]
DeepSeek, whatever drops next week.
[10:18]
If you want, spin up your own. The
[10:19]
coupon LTS10 takes 10% off at
[10:22]
hostingor.com. Links in the description.
[10:25]
Thanks for watching. See you in the next
[10:26]
one.