[00:03] Folks, what I have in my hand is an AI supercomputer. It is NVIDIA DGX Spark. It has some serious hardware in it. I'm going to go through the setup of this machine in this video. We will use Ollama to download Gemma models on this
[00:17] computer and will run the inference. We'll also write code using this local models. All right, let's get started. Let's open this box.
[00:32] It's a mini supercomputer. There are ports. Uh, we'll use this to connect the devices with this computer. And look at devices with this computer. And look at the size of it. It can fit in my hand.
[00:46] the size of it. It can fit in my hand. Okay, so this is NVIDIA DGX Spark. And there is a power cable. Okay, so that's pretty straightforward and there is this [music] manual here. As you can see, I have plugged in the power cord in
[01:01] the very first USB port next to a [music] power button. So there is this little power button that you see. When you press the button and start the computer, it is very, very quiet. You will not even realize that it
[01:15] quiet. You will not even realize that it is starting. Okay, so now that power cable is attached, let's attach uh, rest of the devices. I have two monitors, keyboard, mouse. I'm going to attach all of that to
[01:28] >> DGX Spark. And the way I will do it is using the USB-C docking station. So I have this docking station where my monitors are connected via HDMI cables.
[01:41] connected, you know, you see this Bluetooth device. Bluetooth device. And then this is my uh, USB-C plug. So I'm going to insert that into the one of these three
[01:56] ports. Okay, So, this is how the connection looks. Now, the moment I do connection looks. Now, the moment I do that and I of course press this button, it will be again very, very quiet. So, I can see this.
[02:10] See, welcome to DGX Park. So, we'll go through the setup now. I pressed on get started. I see this screen now. So, the language is US English. [music] language is US English. [music] Time zone is EST. So, now
[02:24] select your keyboard layout. This looks good. Enter. Accept. Username, I will good. Enter. Accept. Username, I will just say DP Code Basics. And let's create a password. Hit enter. Continue.
[02:39] And it is updating now. Updates are done. Let's enter. This is running some done. Let's enter. This is running some Ubuntu based operating system. Okay? So, this is my browser. It's Firefox. It's [music] some customization of
[02:55] Ubuntu uh [music] operating system. And then there is display books that you can look at to start using your uh mini computer. Let's
[03:09] look at the specifications of this system. Here, you can click on specification and this page will appear. It is Nvidia's Grace Blackwell architecture. That is the architecture of your GPU.
[03:24] And it's a unified chip, which means it's a combination of CPU and GPU in one single chip. It has 20 core arms, okay? And 10 Cortex plus 10 additional Cortex with this specification.
[03:40] Then, the tensor performance is one operation. One petaflop is I think 10 raised to 15. Those many floating point operations it can do in parallel. This is humongous.
[03:57] When you're doing deep learning etc. you want to do a lot of matrix multiplications and matrix multiplications can be done in parallel and that's why [clears throat] GPUs are shining. You know, just imagine the
[04:10] scale 10 raised to 15 uh floating point operations it can do in parallel. It has a bandwidth of 273 It has a bandwidth of 273 gigabytes per second which means
[04:24] that much data per second can be sent to GPU for the processing. 4 terabyte is the hard disk and then these are rest of the specifications. As a next step you can click on show apps and just click on
[04:40] this DGX dashboard. The dashboard runs locally as you can see here and it shows you the utilization of your system. So out of 128 gigabyte memory I'm
[04:52] running 7 gigabyte and my GPU utilization is close to 12%. The next step is to install Ollama. Ollama lets you download those models locally and it will let you run those models. So it is an Linux operating system so I'm going
[05:08] to copy this command into my command bar, okay, and install it. After the installation is done you want to download some models locally. If you click on models you'll see all the open models.
[05:23] I'm particularly interested in Gemma which is Google's open source model. which is Google's open source model. Right now the model family is Gemma 4. It has all these different variants. I will download the latest so I'll just
[05:38] copy this. And then I can say Ollama pull Gemma 4. It will take some time because it's 9.6 gigabyte based on your internet speed, the time may vary. Also, if you directly
[05:52] run Ollama run Gemma 4, then what it will do is it will download the model and it will run it. So, now let's run Gemma 4. So, Gemma 4 is available
[06:06] locally. And now Ollama is running the model, And now Ollama is running the model, okay? So, here you can ask any question. You can say who was Patanjali. Patanjali was a founder of yoga. So, this is like
[06:19] as if you are asking it to ChatGPT. And look at this. first time when you run it, it may take some time, but otherwise, you see it's this kind of really fast, okay? So, here it is asking, "In what context you are
[06:35] referring?" I can say uh I want to know about Patanjali the father of yoga.
[06:48] father of yoga. Answer it in three lines. And see it is doing some internal thinking, so these are like the thinking tokens. And then it will generate the actual answer. So, this is the actual answer
[07:00] that it generated. And when you look at GPU utilization, see right now it is question. For example, uh write code to read
[07:13] uh write code to read a file line by Now, I am asking it to write code. And while it is writing the code, you see the GPU utilization is going up because it is using Gemma 4 model for doing the
[07:26] it is using Gemma 4 model for doing the local inference. And once the answer is generated, this utilization will go back to normal. This is one thing I don't like about AI. It it generates so many different variants of the same code. But
[07:40] anyway, see now code is generated and utilization is back to normal. The next step is to run local Gemma model with Claude code. Here, we can run this command Ollama launch Claude, and it will launch Claude
[07:55] code with Gemma 4 as a model. But, before we do that, we have to install Claude code. So, you type this command, take it to your command prompt, okay? And run it. So, that's going to install
[08:12] Claude on my DGX Spark machine. After that, I can just say Ollama launch Claude, okay? And I have created this folder called website, so you can see I have this website folder. It's a It's an empty folder.
[08:27] So, let me show you. So, when I was here, I just ran here, I just ran mkdir website, then I cd'd into it, and now I'm going to run this command Ollama launch Claude.
[08:41] Uh if you're running it for the first time, it will ask you for some setup, but we are done with that setup. Now, let's create a static simple website yoga
[08:56] accessories. Only HTML code, no back end, just a Only HTML code, no back end, just a simple catalog of items. So, I'm simple catalog of items. So, I'm creating a static website because
[09:10] going to take a long time. I just want to demonstrate how Claude can work with local Gemma model, and it can write code for you. This will save you the Claude
[09:24] subscription bill, basically, okay? Now, I'm running Gemma 4 model. You can run I'm running Gemma 4 model. You can run even a higher parameter model, okay? So, in Gemma 4, by default, I have this one, but let's say if you want to run 20 GB
[09:40] model with 31 billion parameter, you can specify this entire model string when you run that Ollama launch this command. All right, our static website is ready. I can go to this folder, go to website.
[09:57] I can go to this folder, go to website. And here I see index.html. My local Gemma model created this website. Our instructions were simple. You can of course give detailed specification on colors, images, etc.
[10:14] have the catalogs. This is what we asked for. for. Uh there is about us section, contact. I think catalog and home page is is what it has implemented. So this was a quick
[10:26] demonstration of how Cloud Code can be used with the local models. This way you're not paying for API bills. If you have a a higher parameter model which is very accurate, then you can generate infinite amount of
[10:41] then you can generate infinite amount of code. All you have to spend on is the electricity bill on this DGX computer. I want to show you some other specifications. For example, it has 6,144
[10:53] 6,144 CUDA cores. When you have a regular uh laptop, the CPU cores will be two or four, correct? Here it is 6,000. It's four, correct? Here it is 6,000. It's like it has 6,000 brains. It all works
[11:06] in parallel. And then it has CPU cores 20. So it's a unified architecture CPU and GPU is like in one chip. And then it can run AI models up to 200 billion parameters, folks. It is so
[11:21] powerful. That's it, folks. Uh in the future videos, we are going to uh make some more experiments. We will go through some of these use cases that they have mentioned. You can do fine-tuning of your models. You can do
[11:36] fast inference. You can run Hermes agents with local models. You can have CLI coding agents, all of that. See, you have so many amazing use cases which you
[11:49] can run on this local computer. All If you have any questions, please post in the comment box below.