---
title: 'Launching Oobabooga via Cloud GPU with Vast.AI'
source: 'https://youtube.com/watch?v=FfTv37WQt8M'
video_id: 'FfTv37WQt8M'
date: 2026-06-14
duration_sec: 556
---

# Launching Oobabooga via Cloud GPU with Vast.AI

> Source: [Launching Oobabooga via Cloud GPU with Vast.AI](https://youtube.com/watch?v=FfTv37WQt8M)

## Summary

This video demonstrates how to run large language models from Hugging Face on powerful GPUs using Vast.ai and the Oobabooga web UI. The presenter walks through selecting a template, allocating sufficient disk space, choosing an appropriate GPU, and downloading models.

### Key Points

- **Introduction to Vast.ai and Oobabooga** [0:00] — The video shows how to run large language models from Hugging Face on powerful GPUs using Oobabooga as the web UI.
- **Selecting the Oobabooga Template** [0:50] — Select the recommended template for Oobabooga, which sets up the environment and opens port 7860 for the Gradio web interface.
- **Allocating Disk Space** [1:34] — Allocate at least 80 GB of disk space upfront because many models are 60-70 GB and disk space cannot be added later.
- **Matching GPU to Model Requirements** [2:22] — Check the model's GPU RAM requirements (e.g., Falcon 40B needs ~60 GB) and select a GPU with sufficient RAM, such as an A100 with 80 GB.
- **Selecting a GPU Instance** [3:42] — Choose a GPU with enough RAM, like a 1x A100 (80 GB) or multi-GPU options. Cheaper alternatives include 4x A5000 (96 GB) or A6000 (48 GB).
- **Instance Creation and Opening Interface** [5:40] — The instance takes about 3-5 minutes to load. Once ready, click the open button to access the Oobabooga web UI on port 7860.
- **Downloading a Model** [7:22] — In the Models tab, paste the Hugging Face username/model name (e.g., from the LLM leaderboard) and click download. After download, load the model into GPU RAM.
- **Billing and Monitoring** [8:48] — Check the billing tab to estimate credits needed for long runs and set auto-billing threshold to avoid instance stoppage.

### Conclusion

By following these steps, you can easily run large language models on cloud GPUs via Vast.ai using Oobabooga, ensuring proper disk space and GPU RAM allocation.

## Transcript

hello uh welcome to vast in this video I
want to show you how you can run some of
the best large language models that
exist from hugging face or other places
on a very powerful gpus and so
let me get started today we'll be using
uba Booga to as the web UI which is a
great interface for prompting the models
and also kind of loading and managing
them that's some open source software
that will will load up in an instance on
vast
so I'll first kind of Click into the
console and make sure that you are
logged into your account and that you
have credits if you've never done this
before with fast we have a different
video that can go over a lot of some of
the basics
but for uba booga
you're going to come in here and select
our recommended template for that
and uh that's gonna have the description
here it kind of shows you which language
models that you can run with it and uh
it's going to have some specific options
and an on-site script that you don't
want to mess with that's going to set up
your environment correctly so this will
work it's also going to open a port for
the open button and that will launch the
gradio web interface so that it all just
works so really all you need to do is
Select that uh template
now one of the most important things
um
is to make sure you allocate enough
Discord storage I just reset my filters
because the default is only 16 gigabytes
which is not going to be enough
a lot of these large language models are
60 70 gigabytes to download and your
instance will start to throw errors if
it runs out of disk space you also need
to allocate all the disk space that you
want to use up front for this instance
you cannot add it later
so with that in mind you're probably
going to want if you're just going to
try one language model at least about 80
gigabytes so I'm just going to move the
slider to 81 and get that all set up the
other important thing to understand when
you're running these large language
models is to match the GPU with the
model that you want to run for example
if you're looking at hugging face
hugging faces has a actual llm
leaderboard and so you can see some of
the most popular models here and
um
how you can run them
and basically what you will need to do
is to load these into uba Booga once we
have that running so we'll come back to
this
but know that the model that you each
one of these models that you're trying
to run for example if you want to run
Falcon 40 billion you need to read
through and understand how much GPU Ram
this is going to require because if this
requires say
60 gigabytes of GPU RAM and you select a
GPU that only has 10 gigabytes of GPU
Ram it is not going to work so you need
to make sure that the the large language
model that you want to run it's going to
have
um uh you need to figure out what exact
specifications it needs
and then select an appropriate GPU
what I like to do is to just actually
select the GPU that has the most GPU Ram
which is one of our a100 uh
smx4s or pcies so I'm going to go ahead
and select a 1x smx4 these have 80
gigabytes of GPU Ram so they have uh one
of the more powerful cards that are out
right now from Nvidia and 80 gigabytes
is is enough for most large language
models you can also select a multi-gpu
instance so if I needed even more space
I could have a 2X a100 that would be I
have 160 gigabytes of GPU RAM available
for the large language models or I could
select sort of a cheaper option like an
a5000 and this 4X a 5
000.
has actually 96 gigabytes of GPU RAM and
it is a little bit cheaper than a single
a100 you can also look at an a6000 they
have 48 gigabytes of GPU Ram
and an A40
has 45 gigabytes of GPU Ram
the consumer graphics cards like the
4090 and 3090 are only going to have 24
gigabytes of GPU Ram each so again this
is just something that you want to be
really aware of and make sure that
you're selecting a GPU that's going to
have enough space so I'm going to go
ahead and select a 1X a100 and now this
is going to load I have 80 gigabytes
allocated on this instance and I have
selected a Ooba Booga web UI which is
our recommended template and so if I
jump into my instances here I can see
that this is being created and set up
for me
it's going to take a three four or five
minutes to load maybe a little bit
longer it's really going to depend on
the internet connection speed of the
machine and
um
uh the size of the image this one loaded
about three and a half minutes for me
and now the open button is going to open
port 7860 which was put in the
environment variables when we set this
up
um and uh
there's a few things that were installed
and set up on the onstart script but
anyways this is all just stuff that's in
the template that we have set up for
Ooba booga
and I'm going to go ahead and open that
interface up and here it is
so uh here's where you can actually
query the model that you set up the most
important thing is that going to be
downloading and setting up the model so
um this software is not
developed or maintained by vast this is
open source software so to understand
how to use this software you're going to
want to find the open source project for
this
and and load that
so here's the
GitHub that's going to have a readme
um
the of course the installation steps you
don't have to worry about because
um you're using a Docker image and
everything is is
pre-loaded
um
so you can place the models into the
model folder or when you're using the
web UI you can just simply go to the
models tab where it was before and
here's where you can download the custom
model so for hugging face you just use
the username and model so for example if
I wanted to try to to use just looking
at the leaderboard if I wanted to use
this model I would simply select the
username and the name of the model like
that and copy and paste it into the web
UI
and hit download and now it is going to
start downloading
this model once that model is downloaded
I will be able to load the model in here
into the GPU
Ram in the instance
sort of memory so then I can query it I
can go back to text generation and
actually start using it there's probably
some other things that you can do and
become familiar with with this interface
this is a very nice way to run llms so
that you don't have to use a command
line
um so there's quite a bit here and but
again you're going to want to read about
this and the ooga booga
GitHub
so if I go back and look at my instance
um
you can see that it's running
you can also click on the billing tab if
you just want to see you know if you're
going to run something for multiple days
you can get an idea of how many credits
you're going to need you can set up your
auto billing threshold so that your
instance is not stopped when your
balance gets low
and that's the basics of running
ubeoka on vast thanks for your time