[0:06] hello hey this is Travis from Bast I
[0:08] wanted to make a quick video to show you
[0:10] how to spin up deep seeks R1 uh that's
[0:14] distilled down to the 70 billion uh size
[0:18] it fits very well on a 2X 490 machine so
[0:21] I just wanted to show you really quick
[0:22] how to spin that up on vast so I'll go
[0:24] ahead and jump into it so we have uh our
[0:28] console interface here and if you're new
[0:31] to vast and just getting started we have
[0:34] a quick start guide that goes over some
[0:36] of the basics of how to create an
[0:38] account add credit um set up your
[0:42] template and select a template filter
[0:44] machines and then R to
[0:46] GPU we'll go ahead and jump into uh the
[0:49] guide that we have for this use
[0:52] case
[0:54] and to run this model uh the Deep seek
[0:58] r170 billion we're going to use AMA and
[1:01] web UI AMA is the software stack that's
[1:04] going to run it and web UI actually set
[1:07] up the UI that you can access um to
[1:11] prompt the model very similar to what
[1:13] you do with uh open AI so let's go ahead
[1:17] and get started I have credit and my
[1:19] account is set up so I'm going to go
[1:21] ahead and pick the correct
[1:25] template which is this uhan web UI
[1:30] I'm going to allocate enough storage uh
[1:32] for this model I only need about 40 gab
[1:35] but I'll just kind of leave it around 60
[1:38] and then I'm going to want a
[1:40] 2X
[1:43] 4090 so I'll go ahead and put that in
[1:46] here now I'm also going to select um I
[1:49] don't know how long uh you plan on using
[1:51] the model but I'm only going to use this
[1:52] for a couple days so I'll just kind of
[1:54] move the max instance duration down not
[1:57] going to rent this for months um so
[2:00] here's all the uh availability of
[2:02] machines uh the ones that are that have
[2:04] the blue label are in a trusted Data
[2:06] Center and um I'll go ahead and pick uh
[2:09] this 2x 490 in Iceland now I'm going to
[2:12] click Rent Now what's going to happen is
[2:15] the instance going to spin up and it's
[2:17] going to start loading this
[2:21] template all right looks like a minute
[2:23] has passed and now the instance is uh
[2:26] loaded so I'm going to go ahead and
[2:28] click this open Button
[2:31] and this is going to open our instance
[2:33] portal uh which basically is uh shows
[2:37] all the process that that are running on
[2:39] this instance right now and you can see
[2:41] uh the open web UI is running on this
[2:44] port and the API is running on this
[2:47] boort so I'll go ahead and click
[2:51] this and this will start up and launch
[2:54] the web
[2:55] UI this is a open source software to run
[2:59] and and chat with
[3:01] llms so when you first boot this up you
[3:04] have to make a
[3:06] name uh
[3:09] email and a password for the
[3:12] admin account and then it's going to
[3:16] show you kind of what's new in open web
[3:19] UI and uh it's going to load you into a
[3:22] screen that has no model selected so
[3:24] you're going to have to download a
[3:27] model I think there's a few ways to add
[3:30] a model um in this interface I kind of
[3:32] just explore but if you go into the
[3:34] admin panel and then click
[3:40] settings
[3:43] models and then you
[3:46] can add a
[3:48] model and uh this model that I'm running
[3:51] is from ama.com so you just need to put
[3:54] in the
[3:55] tag and the tag is
[4:02] is just the name of the model which is
[4:03] right
[4:04] here so I'm going to grab the tag I'm
[4:07] going to drop it in
[4:12] here hit the download button and now
[4:16] it's going to start downloading the
[4:19] model it'll take a few minutes to
[4:21] download to go get a cup of
[4:24] coffee but once it gets to 100% now the
[4:28] Deep seek model is um
[4:30] load it onto this
[4:33] instance and you can simply start a new
[4:37] chat and select the model after it
[4:40] downloads it also could take a little
[4:43] bit of additional time to load
[4:45] it and there it is this is the Deep seek
[4:49] r170 billion model running on a 2X
[4:56] 490
[4:57] and there we can see
[5:00] it's a reasoning model so it has a
[5:02] thinking step uh that it's going to give
[5:04] the
[5:06] answer can see on vast um I'm paying
[5:10] about 80 cents an hour right now for
[5:12] this if I wanted to keep this machine
[5:15] longer there's a reserved instance
[5:17] discount um but I'm uh just going to
[5:21] keep it for uh a few hours to to play
[5:24] around with this a bit and if I wanted
[5:28] to run the API the API uh is available
[5:32] on uh this
[5:33] port and uh Jupiter is also available
[5:37] along with SSH so there's a lot of
[5:39] different ways you can connect to this
[5:41] machine um but it's always very simple
[5:44] to just use this sort of chat interface
[5:46] that loads onto it well that's deep seek
[5:49] r170 billion on vast AI thanks