[00:00]
I'm running something called private
[00:03]
except it's not. Everything about it
[00:07]
Am I even connected to the internet?
[00:08]
This is private contained and my data
[00:12]
company. So in this video I
[00:15]
I want to show you how to set this up.
[00:16]
It is ridiculously easy and fast to run
[00:21]
whatever. It's this is free, it's amazing.
[00:23]
It'll take you about five minutes and
[00:26]
I want to show you something even
[00:28]
I'll show you how you can connect
[00:31]
your documents,
[00:32]
your journal entries to your own
[00:37]
about your stuff. And then second,
[00:38]
I want to talk about how private AI is
[00:42]
Our jobs, you may not know this,
[00:44]
but not everyone can use chat GBT
[00:47]
Their companies won't let them mainly
[00:51]
but if they could run their own
[00:54]
That's a whole different ballgame and
[00:58]
They're the sponsor of this video and
[01:01]
companies can do on-Prem in their
[01:05]
And it's not just the cloud man,
[01:07]
The stuff they're doing is crazy. We're
[01:10]
but tell you what, go ahead and do
[01:13]
Just go ahead and open it and take a
[01:16]
We're going to dive deeper,
[01:16]
so just go ahead and have it open right
[01:20]
on the side or minimize. I
[01:22]
I dunno how many monitors you
[01:25]
I can see before we get started,
[01:27]
You can run your own private ai. That's
[01:34]
So yeah, please don't do
[01:37]
make sure you're paying attention
[01:39]
I'm doing a quiz and if you're one of
[01:42]
percent on this quiz, you're getting
[01:46]
So take some notes,
[01:51]
now real quick, before we install a
[01:55]
what does it even mean? What's
[01:58]
an AI model is simply an artificial
[02:02]
provided. One you may have
[02:05]
but it's not the only one out
[02:08]
We're going to go to a website
[02:11]
Just an incredible brand
[02:14]
This is an entire community dedicated
[02:18]
there are a ton. You're about
[02:21]
I'm going to click on models up here. Do
[02:26]
Many of these are open and free
[02:30]
which is kind of a crazy
[02:32]
We're going to search for
[02:35]
one of the most popular models out
[02:39]
I love the branding.
[02:40]
LAMA two is an AI model known as
[02:45]
open AI's Chat. GPT is
[02:48]
this pre-trained AI
[02:51]
AKA Facebook and what
[02:54]
This model is kind of insane and the fact
[02:58]
use it even crazier, check this out
[03:01]
here we go. Training data.
[03:03]
It was trained by over 2 trillion
[03:07]
sources. Instruction data sets over
[03:11]
data freshness. We're talking
[03:15]
Data freshness and getting
[03:18]
Step two is insane because this
[03:21]
Mata to train this model put together
[03:25]
It already sounds cool, right?
[03:29]
It took 1.7 million GPU hours to
[03:34]
costs around $20 million to train
[03:39]
here you go kid. Download this
[03:43]
I don't want to call it a being
[03:46]
but this intelligent source of information
[03:50]
laptop and ask it questions,
[03:51]
no internet required and this is just
[03:55]
They have special models like
[03:58]
They even have uncensored ones. They have
[04:02]
This guy George Sung,
[04:04]
took this model and fine tuned
[04:08]
took him 19 hours and made it to where
[04:11]
Anything you wanted, whatever
[04:14]
it's not going to hold back. Okay,
[04:16]
so how did we get this fine tuned
[04:19]
actually I should warn you, this
[04:22]
more than you would expect. Our
[04:26]
Let's go ahead and take a field
[04:28]
We'll go to O lama.ai. All we'll have
[04:32]
Alama,
[04:32]
and then we can run a ton of different
[04:37]
of llamas and there's others that are
[04:41]
Llamas. Tdrl. I'll show you in a second.
[04:46]
We can see right down here that we
[04:49]
but oh bummer, windows coming soon.
[04:52]
It's okay because we've got WSL,
[04:56]
which is now really easy to set up.
[04:58]
So we'll go ahead and click on
[05:01]
You'll just simply download this
[05:04]
applications for Linux.
[05:07]
We got to fun curl command that will
[05:09]
install WSL on Windows. This will
[05:15]
go ahead and just run that installer.
[05:19]
Now, if you're on Windows,
[05:20]
all you have to do now to get WSL
[05:23]
Just go to your search bar and search
[05:27]
just happen. It used to be so much
[05:32]
It'll go through a few steps.
[05:35]
I'll go ahead and let that do
[05:39]
I've got Ubuntu 22 0 4 3 lts installed
[05:44]
now. So now at this point, Linux
[05:47]
We're on the same path.
[05:49]
I'm going to copy that curl
[05:52]
jump back into my terminal, paste
[05:55]
Fingers crossed, everything should be
[05:59]
it'll ask for my pseudo password and
[06:04]
Now this will directly apply to
[06:07]
See right here where it says Nvidia
[06:10]
you're going to have a better time
[06:13]
I'll show you here in a second.
[06:15]
We'll keep going. Now let's run an
[06:18]
So we'll simply type in, oh Lama run,
[06:22]
and then we'll pick one llama
[06:26]
set go. It's going to pull the manifest.
[06:28]
It'll then start pulling down
[06:31]
And I want you to just realize this,
[06:34]
we talked about all the money and
[06:38]
This is the 7 billion
[06:42]
It's pretty powerful and we're about to
[06:45]
hands in like 3, 2, 1. Oh,
[06:49]
it's almost done. And boom, it's done.
[06:52]
We've got a nice success message
[06:56]
We can ask you anything.
[06:59]
Now the reason this is going
[07:01]
is that I'm running A GPU
[07:05]
So lemme just show you real quick.
[07:06]
I did install alama on a Linux
[07:10]
performance for you real quick. By the
[07:13]
M two or M three processor, it actually
[07:17]
I got to install it real quick and
[07:19]
What is a pug? It's going to
[07:22]
but it's going to be slower on CPUs and
[07:25]
but notice it is a bit slower.
[07:27]
Now if you're running WSL and you know
[07:31]
I'll show you in a minute how you can
[07:34]
just sit back for a minute,
[07:35]
sip your coffee and think
[07:38]
The tinfoil hat version of me
[07:43]
the zombie apocalypse happens, right?
[07:47]
but as long as I have my
[07:51]
I still have AI and it can help
[07:55]
Let's actually see how that would
[07:58]
I could have it help me with the water
[08:01]
right? It's amazing. But can
[08:04]
You may have caught this
[08:09]
What? Dude, I've always
[08:14]
That is so fun, but seriously,
[08:17]
It didn't have the correct information.
[08:19]
It's so funny how it mixed the
[08:23]
I love that so much. Let's try
[08:27]
I'll try a really fun one
[08:30]
if you want to know which ones you
[08:33]
they get a page for their models right
[08:36]
including llama two,
[08:39]
I might give that to my kids
[08:41]
Now who is Network Chuck?
[08:45]
Now my name is not Chuck Davis and my
[08:50]
Chuck on Tech.
[08:50]
So clearly the data this thing was trained
[08:54]
plain wrong. So now the question is cool,
[08:57]
we've got this local private ai,
[09:02]
but how do we teach it the
[09:05]
How can I teach it to know
[09:08]
and my channel is called Network Chuck.
[09:09]
Or maybe I'm a business and I want it
[09:13]
available because sure, right
[09:16]
you could probably use it in your job,
[09:17]
but you can only go so far without it
[09:22]
maybe you're on a help desk.
[09:23]
Imagine if you could take your help
[09:27]
your documentation. Not only that,
[09:29]
but maybe you have a database
[09:31]
If you could take all that data and
[09:35]
questions about all of
[09:38]
Or maybe you wanted to help troubleshoot
[09:41]
You could even make this LM
[09:44]
You feed information about your product
[09:47]
that chat bot you make.
[09:49]
Maybe this is all possible with a process
[09:53]
this AI on our own proprietary
[09:58]
company or maybe our lives or
[10:00]
whatever use case is,
[10:01]
and this is fantastic because maybe before
[10:05]
you weren't allowed to share your
[10:08]
whether it's compliance reasons or you
[10:10]
data because it's secret.
[10:12]
it's possible now because
[10:15]
it's local and whatever
[10:18]
it's going to stay right there in a
[10:20]
That idea just makes me so excited
[10:24]
how companies and individuals
[10:28]
Back to our question though,
[10:31]
Training and AI on your own
[10:34]
Because as we saw before with
[10:38]
it took them 6,000 GPUs
[10:42]
Do we have to have this massive
[10:46]
Check this out, and this is such a fun
[10:50]
what's the latest version
[10:52]
Now the latest chat GPT
[10:55]
but that wasn't helpful to VMware because
[10:58]
on chat hadn't been released yet.
[10:59]
So it wasn't public knowledge
[11:02]
And they wanted information like this
[11:06]
the public.
[11:07]
They wanted this to be available to
[11:10]
something like chat GBT, Hey, what's
[11:14]
And they could answer correctly.
[11:15]
So to do what VMware is trying to do
[11:19]
data, it does require a lot. First of all,
[11:22]
you would need some
[11:24]
Then you would also need a bunch of
[11:29]
and TensorFlow, pandas, MPI side
[11:33]
The list goes on.
[11:34]
You need lots of tools and resources
[11:37]
That's why I'm a massive fan of
[11:40]
They have something called the
[11:44]
the gajillion things I just listed
[11:49]
one combo meal, a recipe of
[11:53]
So as a company it becomes a bit easier
[11:57]
For the system engineer you have on
[12:00]
they could do this stuff,
[12:01]
they could implement this and the data
[12:04]
actually do some of the fine tuning,
[12:07]
So here's what it looks like to fine tune
[12:10]
the curtain at what a data
[12:12]
So first we have the infrastructure
[12:17]
Now if you don't know what vSphere
[12:20]
you got one big physical server. The
[12:23]
touch and smell. You haven't smelled
[12:26]
And instead of installing one operating
[12:29]
you install VMware's, EA XI,
[12:31]
which will then allow you to virtualize
[12:35]
computers. So instead of one computer,
[12:37]
you've got a bunch of computers all
[12:40]
And that's what we have right here.
[12:43]
a virtual machine.
[12:44]
This by the way is one of their special
[12:49]
I mentioned and many, many more
[12:53]
Everything a data scientist could love.
[12:55]
It's kind of like a surgeon walking in
[12:59]
assistants or whatever have
[13:01]
It's all in the tray laid out
[13:04]
All he has to do is walk
[13:08]
That's what we're doing
[13:10]
Now talking more about hardware,
[13:11]
this guy has a couple Nvidia GPUs assigned
[13:16]
a technology called PCIE Passthrough.
[13:20]
I notice they are V GPU for virtual GPU
[13:25]
cutting up the PU and assigning some
[13:29]
machine. So here we are in data scientists
[13:33]
a common tool used by a data scientist,
[13:35]
and what you're going to see here is a
[13:37]
the data,
[13:38]
specifically the data that they're
[13:42]
model on. Now we're not
[13:44]
but I do want you to see
[13:45]
A lot of this code is all about getting
[13:48]
it might be a bunch of the knowledge
[13:51]
getting it ready to be fed to the LLM.
[13:55]
Here's the dataset that we're training
[13:59]
We only have 9,800 examples that we're
[14:04]
pieces of data. And that
[14:06]
like a simple question or a prompt and
[14:11]
that's how we essentially
[14:14]
we're only giving it 9,800 examples,
[14:16]
which is not a lot at all and is
[14:20]
model was originally trained.
[14:22]
And I point that out to say that we're
[14:25]
ton of resources to fine tune this model.
[14:28]
We won't need the 6,000 GPUs we needed
[14:32]
We're just adding to it,
[14:33]
changing some things or fine tuning it
[14:37]
what actually will be changed
[14:41]
we're only changing 65 million parameters,
[14:46]
But not in the grand scheme of things
[14:49]
We're only changing 0.93% of the model.
[14:52]
And then we can actually
[14:54]
which this is a specific technique in
[14:58]
simply feed up additional prompts with
[15:02]
people asking you questions.
[15:03]
This process will take three to four
[15:06]
we're not changing a lot and that is
[15:10]
is leading the charge with private ai.
[15:12]
VMware and Nvidia take all the guesswork
[15:17]
tune an LLM. They've
[15:19]
which are insane VMs that
[15:23]
everything a data scientist
[15:26]
Then Nvidia has an entire suite
[15:29]
taking advantage of some really exciting
[15:33]
Now there's one thing I didn't talk about
[15:36]
For right now it's this right
[15:39]
post gray SQL box here.
[15:42]
This is something called rag and it's
[15:46]
personal GPT here in a bit. Retrieval,
[15:51]
let's say you have a database of
[15:54]
whatever it is, and you haven't fine
[15:58]
So it doesn't know about it. You
[16:01]
You can connect your LLM to
[16:05]
this knowledge base and
[16:08]
Say whenever I ask you a question about
[16:11]
before you answer, consult the database,
[16:13]
go look at it and make sure
[16:16]
We're not retraining the LLM, we're
[16:20]
go check real quick in this database to
[16:23]
got your stuff right.
[16:25]
fine tuning is cool and training
[16:29]
but in between those
[16:31]
you can have rag set up where
[16:34]
your internal documentation and give
[16:38]
that database. That is so stinking cool.
[16:40]
So with VMware private AI
[16:43]
they have those tools baked right in
[16:47]
would otherwise be a very complex setup.
[16:51]
like I said earlier,
[16:53]
I actually connected a lot of my notes
[16:58]
using RAG and I was able to talk
[17:03]
journal entries and answering questions
[17:07]
before we move on,
[17:08]
I just want to highlight the fact that
[17:12]
gives you some amazing and fantastic
[17:17]
then fine tune and customize and deploy
[17:21]
So VMware Cloud Foundation,
[17:22]
they provide the robust infrastructure
[17:26]
tools you need to develop
[17:29]
Now it's not just Nvidia, they're
[17:31]
So VMware is covering all the
[17:34]
And then for the data
[17:36]
Intel's got your back data analytics,
[17:38]
generative AI and deep learning tools
[17:42]
And they're also working with IBM, all
[17:46]
VMware has the admin's back. But
[17:49]
one of the first AI things I ever
[17:52]
and I love this because what VMware
[17:55]
If you want to run your own
[17:58]
You're not just stuck with one of the
[18:00]
run it with Nvidia and VMware,
[18:04]
You got options. So there's
[18:06]
It's not for some of the bonus section
[18:09]
own private GPT with your own
[18:14]
it is a bit more advanced,
[18:16]
you should be able to get this up and
[18:20]
Let's get this going. Now, first of
[18:23]
This will be a separate project
[18:26]
this is kind of hard to do.
[18:29]
which they do it all for you,
[18:30]
it's a complete solution for companies
[18:34]
What I'm about to show you is not that
[18:37]
It's a free side project.
[18:39]
You can try just to get a little taste
[18:44]
rag tastes like. Did I do
[18:47]
Now L Martinez has a great doc on
[18:51]
but you can do it. And if
[18:53]
he does have a few lines of code for
[18:57]
this is CPU only. You can't really
[19:00]
which is what I wanted to do. So
[19:03]
I've got a Windows PC with an
[19:06]
Linux-based project. WSL, and I'm so
[19:11]
He put an entire guide
[19:14]
I'm not going to walk you through every
[19:17]
below, but I seriously need to buy
[19:20]
I don't know, Emil, if you're
[19:22]
I'll send you some coffee. So anyways,
[19:24]
I went through every step from installing
[19:27]
drivers and using poetry to handle
[19:31]
I landed here.
[19:32]
I've got a private local working private
[19:36]
browser and it's using my GPU,
[19:38]
first I try a simple document upload,
[19:40]
got this VMware article that details
[19:43]
video. I upload it and I start asking
[19:46]
I tried something specific like show me
[19:50]
Bam, it figured it out,
[19:52]
what's the coolest thing
[19:55]
It told me I'm sitting here chatting
[19:58]
let's try something bigger. I
[20:00]
I've got a ton of journals on markdown
[20:03]
about me. Now this specific step
[20:06]
So here's how you do it. First,
[20:07]
you'll want to grab your
[20:10]
about and throw it onto your machine.
[20:12]
So I copied over to my WSL machine and
[20:16]
complete and I ran private GPT. Again,
[20:18]
here's all my documents and
[20:21]
So let's test this out. I'm going
[20:26]
So I went to Japan in November of 2023.
[20:31]
figure out when that was and what I did.
[20:36]
That's awesome. Oh my goodness.
[20:41]
Let's see, what did I eat in Tokyo?
[20:45]
How cool is that? Oh my gosh,
[20:49]
but I can see the potential here.
[20:53]
Private AI is the future and that's why
[20:57]
this to companies to run their own
[21:01]
easy. If you actually did that private
[21:04]
there's a lot to it. Lots of tools you
[21:07]
But with VMware,
[21:08]
they kind of cover everything like that
[21:11]
their solution. It's got all the
[21:15]
you're like a surgeon just
[21:17]
You got all this stuff right there. So
[21:20]
check out VMware private AI link below
[21:24]
sponsoring this video. You made it to
[21:28]
This quiz will test the knowledge you've
[21:32]
people to get a hundred percent on this
[21:36]
Chuck Coffee. So here's how
[21:38]
Check the description in your
[21:41]
If you're not currently signed into the
[21:43]
If you're not a member, go ahead
[21:47]
Once you're signed in,
[21:48]
it will take you to your dashboard showing
[21:51]
with your free academy account.
[21:54]
go back to the YouTube video,
[21:55]
click on that link once more and
[21:58]
Go ahead and click on start now and
[22:03]
That's it. The first five to get
[22:06]
If you're one of the five,
[22:06]
you'll know because you'll
[22:09]
You got to be quick, you got to be smart.