How to Finetune Llama3-8b in Google Colab for Free

[00:00] everybody training new large language

[00:02] models are training them out the box for

[00:04] chat a chat trained llm is like an

[00:07] intelligent student who finished general

[00:09] education in 50 languages now think of

[00:12] fine-tuning these llms like kicking the

[00:14] broke General ed student out of the

[00:16] house and choosing exactly what

[00:18] specialized degree they will get in this

[00:20] video I'm going to demystify the concept

[00:22] of fine-tuning a language model no

[00:25] programming experience will be needed to

[00:27] follow along this tutorial I'll be

[00:29] showing how to fine tune meta's latest

[00:31] llama 38 billion parameter model on free

[00:35] gpus in Google collab let's discuss a

[00:38] huge problem with implementation of

[00:40] every team of AI agent project we have

[00:43] seen to date it started with auto GPT

[00:46] and now the current leader in hype crew

[00:49] AI the reason most teams of AI agents

[00:52] that people are creating to attempt to

[00:54] accomplish complex tasks sucks is

[00:57] because they are using raw large

[00:59] language models trained to respond as

[01:02] intelligent chat Bots and not as action

[01:04] models out of the box these models are

[01:07] trained to have decent general

[01:09] intelligence and serve a general

[01:11] audience like chat gbt as an AI software

[01:14] engineer the first step I would take to

[01:16] develop a system to outperform any of

[01:19] these current existing AI agent swarms

[01:22] is to train a decision-making model

[01:24] based on first principal reasoning I

[01:26] think it's also important to reason from

[01:28] first principles rather than by analogy

[01:31] the normal way that we conduct Our Lives

[01:32] is we Reason by analogy we're doing this

[01:34] because it's like something else that

[01:36] was done hold up wait a minute and what

[01:39] that really means is you kind of boil

[01:41] things down to the most fundamental

[01:43] truths say okay what are we sure as

[01:44] possible is true and then reason up from

[01:47] there that takes a lot more mental

[01:48] energy think of each response from an

[01:50] llm as a thought we want this

[01:53] intelligent AI to generate in 10 seconds

[01:56] or less instead of expecting our AI to

[01:58] use first principle reasoning break the

[02:01] prompt into multiple needles in a

[02:03] haystack of facts the model needs to

[02:05] understand or tasks the AI needs to

[02:08] complete all in a single response in

[02:10] this video I will fine-tune a llama

[02:12] model to power the first AI agent of

[02:14] contact in a team of AI agents I want

[02:17] the AI agent to be able to use first

[02:19] principal reasoning to Output highly

[02:22] logical order of steps that need to be

[02:24] completed to actually provide a factual

[02:27] response and better yet automate a

[02:29] complex task before we get into

[02:31] fine-tuning a model let's first demo raw

[02:34] llama 3 8bs ability to accomplish just

[02:37] the task of generating first principal

[02:39] reasoning outputs if I go try to get

[02:42] llama 38b to act as a decision model the

[02:45] reasoning is as though it came from a

[02:47] mind who wasn't trained to make actions

[02:49] in the real world tell llama to respond

[02:52] with just a python list and it

[02:54] constantly adds notes before or after

[02:56] the list making the responses unreliable

[02:59] as commands for a Python program or even

[03:01] other AI agents to process we can even

[03:04] try these same tasks on llama 370b the

[03:07] chat tune model still cannot manage to

[03:10] Output the correct response format

[03:12] reliably now let's look at the responses

[03:14] from llama 3 8B fine-tuned on a tiny but

[03:18] highquality data set that I created to

[03:20] show the model exactly how I want it to

[03:23] respond to agent prompts fine tuning on

[03:26] just 40 parameters in this case allowed

[03:28] the model to break break out of thinking

[03:30] it is just a chatbot limited to

[03:33] generating text now llama 3 thinks

[03:35] freely about what tasks would need to be

[03:38] accomplished by an AI to actually

[03:41] accomplish my instructions despite half

[03:43] of them claiming to in the title all of

[03:46] the video fine-tuning tutorials I have

[03:48] found do not show how to fine-tune on

[03:50] your own data set in this video I will

[03:53] show you how to create your own data set

[03:55] to fine tune on instead of using some

[03:58] pre-existing data set the data set you

[04:00] use for fine-tuning is about quality and

[04:03] not quantity since we are training a

[04:06] specialized model we want to take full

[04:08] control of maximizing our data set's

[04:11] response examples quality on exactly the

[04:14] task we need it to work at as my data

[04:16] set I have a Json file called Data set.

[04:19] Json inside this file I have one long

[04:22] list of dictionaries each dictionary

[04:25] consist of the same system prompt I'm

[04:27] trying to get the model to properly

[04:29] respond to as well as the prompt for the

[04:32] input value and the response as the

[04:34] output value to create my data set for

[04:37] each response I used mixl 8X 22b to

[04:40] generate rough draft responses before

[04:43] adding any of these responses to my data

[04:46] set I'll go through and manually edit

[04:48] each to improve upon the quality and

[04:50] ensure perfect formatting as a python

[04:52] list your data set for fine tuning could

[04:55] be 20 examples or thousands of examples

[04:58] while larger fine tuning data sets can

[05:01] improve upon your model's performance I

[05:03] can't stress it enough the importance of

[05:05] adding only highquality examples to your

[05:08] data set each example in your data set

[05:10] is an example showing llama 3 what you

[05:13] expect a perfect response from that

[05:16] input should be so if you are

[05:17] fine-tuning on Mid data expect the

[05:20] quality from your fine-tune model to be

[05:23] mid if you want a copy of my data set to

[05:25] skip making your own for this tutorial

[05:27] or just have a copy of my data to ask on

[05:29] to for your own fine tuning it's

[05:31] available in the Pro learning docs

[05:33] channel of my Discord for anyone with an

[05:36] AI Austin Pro membership with my data

[05:38] set of 40 examples complete I now am

[05:41] ready to start loading them into my

[05:42] collab notebook and start fine-tuning

[05:45] llama 3 check the comment section for my

[05:48] pinned comment with the link to the

[05:49] Google collab notebook that I will be

[05:51] going through in this video Once the

[05:53] notebook loads I can drag my data set.

[05:56] Json file into the main content folder

[05:59] then I will select my runtime type to

[06:02] use a free T4 GPU and save it to start

[06:05] the runtime once it is up and running

[06:07] click the play button on the first code

[06:09] Block in step one to install the needed

[06:12] python libraries for a T4 GPU I'll run

[06:15] step two to import the libraries into my

[06:18] runtime once the installations and

[06:20] imports complete we'll run this next

[06:22] oneline block to log into our hugging

[06:25] face account with a right access token

[06:27] if you don't have a hugging face access

[06:30] token yet you can get one for free by

[06:32] logging into your account going to

[06:34] settings clicking access tokens and

[06:36] create a token with right access granted

[06:39] copy that and paste that into the field

[06:41] to log into your hugging face in the

[06:43] next block we have some python code that

[06:45] loads our data set. Json file and

[06:48] converts our examples into llama 3's

[06:50] correct template format you'll see the

[06:52] hugging phore userv value is set as my

[06:56] username make sure you change this to

[06:58] your actual hugging face username our

[07:01] next code block will set up our

[07:03] configuration settings for the

[07:04] fine-tuning the fine-tuned model

[07:07] variable sets the name you want to save

[07:09] the model as in your hugging face

[07:11] repository so feel free to change this

[07:13] too we can run the configuration

[07:15] settings block now and the next block to

[07:18] load the Llama 38b Cur and trainer model

[07:22] now we'll run the trainer to start the

[07:24] fine-tuning process on our data set

[07:27] you'll see the trainer going through

[07:28] multiple training steps before

[07:30] completing each training step is a batch

[07:33] of our training data being ran each

[07:35] batch in the training steps you will see

[07:37] this training loss number start to drop

[07:40] when our model is training it is going

[07:42] through each of our prompts from the

[07:44] data set and blind generating what it

[07:46] expects our example response in the data

[07:48] set is training loss is a value to

[07:51] represent the difference between the

[07:53] model in training's predicted response

[07:55] to the example response in our data set

[07:58] a lower training loss value means that

[08:00] the predicted outputs during fine-tuning

[08:03] are getting closer to the responses in

[08:05] our data set this code with its current

[08:07] configuration settings runs one Epoch

[08:10] one Epoch equals one pass through of our

[08:13] entire training data set running more

[08:15] Epoch up to a certain threshold will

[08:18] absolutely allow your model to achieve a

[08:21] lower training loss during fine-tuning

[08:23] going back up to your Laura

[08:25] configurations you can change the num

[08:27] train epox variable to the number of

[08:30] passes through your data you want it to

[08:31] run now there is a few things to note

[08:34] before changing this increasing this

[08:36] number will increase memory usage

[08:38] meaning you can only raise it so much on

[08:40] the free collab gpus before the runtime

[08:43] will fail from exceeding memory another

[08:46] consideration is that the benefits of

[08:47] raising the epoch is diminishing meaning

[08:50] at some point running more Epoch will

[08:52] not decrease the training loss value the

[08:55] ideal number for my training data set

[08:57] was about 15 to 20 EP talks before the

[09:00] training loss was practically staying

[09:02] the same step eight will save the

[09:04] trainer stats. Json file to your collabs

[09:07] content folder step nine will quantize

[09:10] your fine-tune model and save it to your

[09:12] hugging face quantizing the model will

[09:14] allow it to perform much faster on your

[09:17] local machine this code block will take

[09:19] about 20 minutes to complete in the last

[09:21] step of the notebook you can test some

[09:23] of your prompts to your custom model a

[09:26] better option I can recommend for anyone

[09:28] with a computer with at least 8 GB of

[09:31] RAM and ideally 16 or more you can test

[09:34] the model locally with LM Studio LM

[09:37] studio is completely free to use and

[09:39] easy to install inside LM Studio I can

[09:42] go to the search Tab and type my hugging

[09:45] face username inside there I can click

[09:47] my Project's repository locate the file

[09:50] with Q4 korm at the end of the file and

[09:55] download that model file once downloaded

[09:57] I can go to the chat tab click new chat

[10:00] and load my custom fine-tuned model in

[10:02] the system prompt tab I will paste in

[10:05] the same exact system prompt that I used

[10:07] to fine-tune my model on while this is

[10:09] not going to be a tutorial on how to use

[10:12] LM studio just note that there's also a

[10:14] lot of settings for optimizing the speed

[10:16] of your model on your machine don't

[10:18] forget to hit the like button on this

[10:20] video If you learned anything new about

[10:22] fine-tuning this has been AI Austin I

[10:25] will see you in the next one