---
title: '"okay, but I want Llama 3 for my specific use case" - Here''s how'
source: 'https://youtube.com/watch?v=pK8u4QfdLx0'
video_id: 'pK8u4QfdLx0'
date: 2026-07-28
duration_sec: 1460
---

# "okay, but I want Llama 3 for my specific use case" - Here's how

> Source: ["okay, but I want Llama 3 for my specific use case" - Here's how](https://youtube.com/watch?v=pK8u4QfdLx0)

## Summary

David Andre explains how to fine-tune Llama 3 for free using Google Colab and the Unsloth framework. He covers the basics of fine-tuning, data preparation, and step-by-step implementation to adapt the model for specific tasks.

### Key Points

- **What is Fine-Tuning?** [00:15] — Fine-tuning adapts a pre-trained LLM like Llama 3 to a specific task by adjusting a small portion of its parameters on a focused dataset.
- **Benefits of Fine-Tuning** [00:52] — Cost-effective (uses a GPU for hours instead of millions), improved performance on specific tasks, and data-efficient (works with as few as 300-500 entries).
- **How Fine-Tuning Works** [01:49] — Steps: prepare a tailored dataset, update pre-trained weights using optimization algorithms (only possible with open-weight models), then monitor and refine to prevent overfitting.
- **Real-World Use Cases** [02:48] — Customer service chatbots using proprietary transcripts, content generation in a specific writing style, and domain-specific analysis (e.g., legal or medical texts).
- **Implementation with Llama 3** [04:11] — Uses a Google Colab notebook (created with Unsloth) to fine-tune Llama 3 8B for free on a T4 GPU. Steps include checking GPU, installing dependencies, loading the model, and configuring LoRA.
- **Data Preparation** [08:43] — Uses the Alpaca dataset (50,000 rows) with instruction-input-output format. Custom datasets must follow the same structure. Suggests using LLMs to generate larger datasets from a few hand-crafted examples.
- **Training Configuration** [11:39] — Trains for 60 steps (not a full epoch) for demonstration. For production, use multiple epochs and set max_steps to None. Training loss dropped from ~1.9 to ~0.8 in 8 minutes.
- **Testing the Fine-Tuned Model** [16:22] — The model correctly answered prompts like listing prime numbers (1-50) and converting binary to decimal. Uses text streamer for token-by-token generation.
- **Saving the Model** [19:28] — Save LoRA adapters locally or push to Hugging Face Hub. For inference, load adapters by setting a flag to true. Recommends using Unsloth for faster inference.
- **Quantization and Deployment** [23:03] — Quantize the model (e.g., Q4) for easier deployment on weaker hardware. Can be used with UIs like GPT4All or Oobabooga for easy chatting.

### Conclusion

Fine-tuning Llama 3 is accessible and cost-effective, enabling anyone to adapt a powerful LLM to their specific needs using free tools like Google Colab and Unsloth.

## Transcript

my name is David Andre and in this video
I'll teach you how to fine tune llama
free so that it performs 10 times better
for your specific use case let's start
with what even is fine tuning and I made
this explanation in plain English so
that anybody can understand fine-tuning
is adapting a pre-trained llm like gbd4
or in this case Lama 3 to a specific
task or domain it involves adjusting a
small portion of the parameters on a
more focused data set so you know when a
new model releases what everybody needs
to know is how many parameters it has we
have llama 3 8B and always that number
like 8B or 70b that's the number of
parameters so we're adjusting just a
small number of them to make it more
focused on a specific thing fine tuning
customizes the outputs to be more
relevant and accurate for your use case
here's the power of fine tuning cost
Effectiveness it leverages the power of
pre-trained llms which cost tens of
millions of dollar if not hundreds of
millions to train and we can just you
know run a GPU for a few hours and fine
tune something for I don't
know like cents a few cents or few
dollars at most which is just amazing it
gives you improved performance because
you can enhance the llm on your data set
and improve accuracy for specific tasks
and it it also is more data efficient
you can achieve excellent results even
with smaller data sets so you know maybe
maybe even like 300 500 entries while
you know llama 3 was trained on 15
trillion tokens I don't know about you
but I'm not have I don't have nearly as
much data as Zak so that's why fine
tuning is great for people like you and
me so how does llm fine tuning actually
work first you need to prepare your data
set and this you know depending on how
hardcore you want to go this can take
anywhere from 20 minutes to a few hours
to week
potentially depends how far you want to
take it so you create a smaller high
quality data set tailored to your
specific use case and label it
appropriately which I'll teach you in a
bit the pre-rain llms weights are
updated incrementally using the
optimization algorithms like grade in
descent based on the new dat set so we
can only fine-tune uh llms that we have
access to the weights meaning open
source open weights llms you cannot find
you gbt 4 if you are not open AI open
can do it obviously but me and you we
probably don't have gp4 just laying on
our
computer then you Monitor and refine you
evaluate the model's performance on a
validation set preventing overfitting
and guide adjustments now here are some
real world use cases for fine tuning
fine tuning and llm or customer service
transcripts can create a chat bot like
this one that can address issue in a way
specific to the company so let's say you
know you have a specific product very
Niche that is not there is not much data
about it on the internet and if somebody
messages your customer support email you
want your you know chatbot to respond in
a specific way based on the information
of your product and that data is
proprietary it's private only you have
it and you can find you an llm to
respond based on that data so like
technically if you have enough script
you can find you an llm to respond like
you and you know if you try sh GPT if
you even give like sh GPT some writing
and tell it continue in this writing
style it's terrible so this is where
fine tuning could be better tailored
content generation so you can fine tune
in llm on your posts and descriptions to
create engaging summar or marketing copy
again in your writing style tailored to
your
audience domain specific analysis so
fine tuning llm on legal or medical text
can make it much better for those
specific Benchmark so and you might have
a model that let's say it reaches 50 on
some arbitrary Benchmark with fine
tuning it can reach 70 or 80 now let's
dive into how to actually implement this
on Lama free so I created this Google
collab well actually most of it was
created by ansoff team A huge shout out
to ansoff because they did all the heavy
lifting so I'm going to also link their
GitHub below now first off I added a
component that's only available in April
to the community so if you join during
April you will get a personalized AI
strategy to Future proof yourself and
your business so if you want to be among
people who are building the future if
you want access to all the different
courses modules and everything else in
the community and to two we Rec calls
then consider joining and especially if
you want me to give you a personalized
AI strategy to Future PR yourself so if
that's interesting to you make sure to
join the community it's the first link
in the description now let's find youe
Lama free shall we so first thing we
check the GPU version available in the
environment and install specific
dependencies that are compatible with
the detected GPU to prevent conflict so
this is uh this cell by the way if you
don't know how uh Google collab Works
which is you know the software I'm using
right now it's super simple it's
basically um splitting the code into
cells it's called The jupyter Notebook
but it's like much more easier to see
you can add text you can add graphics
and it's great for like tutorials and
explaining right so if you never use
this it's great because it's free and
Google actually gives you a GPU so you
can use this T4 GPU to train this model
for free and if you want faster you can
obviously upgrade it right so I'm going
to link this collab below the video as
well so we run this cell which does what
I just explained the next cell we need
to prepare to load a range of quantied
language models including the new 15
trillion lvfree model so trained on 15
trillion tokens and it's optimized for
efficiency with forbit quantization I
mean I'm not going to even pretend I
know everything about fire tuning
because I don't so if you know if um it
seems like I have gaps in my knowledge
it because it is I do have those gaps in
my knowledge so I try to make it as
simple as possible but if this proves
something it proves that you don't have
to be a machine learning expert to find
your models so you know just follow
along so here this is the max sequence
length uh obviously 3 is up to 8,000 so
I mean 2,000 is plenty for this
demonstration but you can do anything
you can do 4,000 or
8,000 here use 4bit quantization to
reduce memory usage but it can be false
as well so here are the models we can
see like we have mro 7B llama 2 which is
the old one Gemma from Google but
obviously we're interested in llama 3 8B
and by the way we can also use llama
370p if you want which obviously will
take longer because uh it's a much
bigger model so in that case you might
uh want to buy the premium version of of
collab or just wait for a while but yeah
I mean uh everything is the same just
here you would change the model to Lama
fre 70p and if you want to use like a
gated models from hugging face which
gated means that you have to usually
agree to some you know license or
whatever then here just remove the
comand and then put your hugging face
token here super simple now by the way
you always have to run this so what you
do when you go to Google collab you
click on run time and click run all that
way all of the cells run but you can
also do it one by one by clicking this
button right here next to each cell and
it needs to have this little tick green
tick that way it was uh executed here
it's not because I you know removed the
I changed this so anytime you make any
change it disappears but that doesn't
matter it was still executed so it's
stored in the run time next next up we
integrate Laura again you don't have to
understand what this is but it's
basically um way of fine-tuning into our
model which allows us to efficiently
update just a fraction of the parameters
enhancing training speed and reducing
computation load so again we are not
training the model from scratch we're
just fine-tuning a few parameters for
our specific use case and here you can
change the r to Any number greater than
zero 8 16 32 64 up to
you and your goals would want to do with
it by the way on SLO the reason I'm
using it is because it's uh makes fine
tuning much faster and consuming less
memory so it's actually a great uh great
framework for this data prep we now use
the alpaka data set from yma which is
this one which has 50,000 rows and I
have it loaded in vs code here just that
way you see how it looks like in Json
formatting so you know it's a lot of
lines because for everyone it's
basically times five yeah so like 200
50,000 uh lines and it's like every one
every one of them has an
instruction should probably Zoom it up
Zoom it
in so yeah every one entry has a
instruction give fre tips for staying
healie input this is not mandatory
because instruction is already enough
context and then output this is what the
llm should say and you do this enough
times and the llm you know learns it
basically learns right so we you can see
it probably better here uh and if you
want to use your own data set you have
to format it the same way so you know
just having output input and
instructions these three um parameters
but yeah just look at this not all of
them have the input which is fine I mean
probably like 20% or 15% have the input
and that's just extra context so yeah uh
I'm also going to link this data set
below but if you want your own data set
which you know if you want your own use
case just make sure to format it the
same way so you know instruction some
text input some extra context or empty
and output how the model should respond
and you know if you if you're getting
creative you can definitely use llms to
generate these large data sets much
faster I mean maybe you create really
like 20 high quality examples by hand
and then you run a team of Agents um for
creating that data set that can just you
know use those 20 examples to create
50,000 like in this data set but yeah
that's a topic for a whole another video
so if you want me to make a video on how
to make data sets for fine tuning then
let me know but let's go back to our
collab so then we Define a system prompt
which is you know custom instruction
system prompt which you already know
hopefully that formats tasks into
instruction inputs and responses so this
has to fit with our data set and we
apply it to our data set for the model
and we add the EOS token to Signal
completion so this token right here here
we Define it and here here we add it
because without this the token
generation continues forever so we don't
want that obviously so let's look at the
system prompt it's very simple it says
below is a instruction that describes a
task paired with an input that provides
further context WR the response
that appropriately completes the request
and that's our system prompt and then we
feed it the instruction the input and
response and obviously you can change
the system prompt if you
want now train the model we do a 60 step
uh we do only 60 steps here to speed
things up um you can like this is
obviously very small because it's not
even one Epoch training Epoch so uh if
you want to like actually use something
for production or your business you
probably want to train it for longer
than 60 steps and I'm going to show you
how how in this
bit so if you if you do multiple EO you
have to turn Max steps none so here okay
number number of trained eox is not
included in here so what you would do is
you would copy this and you would go in
here and look at the steps right so we
have the steps here you would add this
maybe you would do four or
whatever however many you want the more
the better but at a certain point it
starts to not yield better result so max
steps you have to change it to none
right so this is 60 60 right now so you
do none and this is where you would do
like proper fine tuning but um you know
I just add it that 604 demonstration
that way it's faster and it still took
like 8 minutes so I'm not going to
replicate it I'm just going to show it
everything but yeah basically um you
know this is what you do you decide how
many EO you want and then at this stage
we confir configuring our models
training setup where we Define things
like badge
size and learning rate to teach our
model effectively with the data we've
prep prepared so obviously you can like
mess with stuff here um again I'm not
going to PR pretend I understand
everything but the main things are you
know backing like this can make it five
times faster for short sequences
obviously the steps and the epox but um
yeah I mean if you're confused something
just take a screenshot boom like this
and ask sh
GPD now this is the current memory stats
right so we're using the Tesla T4 GPU
provided from Google for free and the
max memory is 14
GB and this is where the training begins
this is the magical part right so here
we do this line of code trainer stats uh
trainer. train and this will give us the
statistics as the model trains so again
this is only 60
steps which is um like zero EPO but yeah
um you can see the training loss going
down so like basically smaller number is
better here so you can see like at the
start we have 1.8 2 like 1.9 and then it
quickly starts dropping to like 0.9 you
know around 1 0.8 so it fluctuates a bit
but it consistently go down 0.7 but you
can see it's reaching like a as symt
right obviously it's only 60 steps so
really doesn't mean anything um but yeah
like we ended up like 0.8 from like two
so it shows you like if the model is
actually
improving and this took like 8 minutes
you can see the stats here right so 476
seconds almost exactly 8 minutes Peak
Reserve memory was 8.9 GB and for
training was 3.3 GB so not like this is
the power of unso it's like really
optimized for for this to use uh to run
faster and to use less memory so that
way we can find tune gpus for cheaper I
mean you know I'm using a free T4 GPU
from Google so it's free but it's faster
like if you didn't use unso it would be
a lot
slower so okay so 60% of we used 60% of
max memory so that's good because we
didn't like hit the limit so we still
have like 40% reserved and for training
uh it was only 22% which is even better
inference which is which means here we
actually run our new model that we
fine-tuned and okay so this data set is
for like instructions and this is
basically when you see a model that is
like instruct at the end of it this is
what they mean it's just trained on a
large data set of instructions because
usually the models are more for like
chatting for text generation you know
you give it some input and it's like
gives you some output it's you know for
more conversational here for
instructions for instruct models is to
follow instructions you give it a task
and it completes it so like we can see
it probably here in vs code like rewrite
the sentence to change its meaning and
then output the Fe
escaped compar to dat sub so this is
like all tasks it's all in instructions
and then it shows how the model should
do it
so let's look at it right so now we've
trained the model this took like 8
minutes to do so all of you can do this
the beauty of using a Google cloud is
that obviously ly it doesn't matter what
machine you have even if you have a
terrible computer this will take the
exact same time because you're using the
GPU and
Cloud so obviously here you can change
your prompt I mean this is you know I
changed the prompts here so this is my
prompt uh but always make sure to leave
the output blank so here the first one
is the instruction then this is the
input like the extra added context and
the output leave it blank because the
model will generate it right so list the
prime numbers contained within this
range and then the range is here in the
input 1 to 50 and then the model our new
findun the Lama 3 generates the output
so let's look at this 2 3 5 7 11 13 17
19 23 29 and just by looking at it uh
you can see it's correct I mean none of
these numbers are divisible so yeah this
is correct all of them are prime
numbers and also this is this is even
better like I think this is much more
visible using text streamer for
continuous inference and I'm I'm just
going to show it again by the way this
is how it looks right so you have the
instructions it's separated but that's
not the main thing not only is it
formatted better it's uh continuous
inference so you can see the token
generation token by token instead of
waiting for the whole time so if I run
this as you can see it waits and it
generates it all at once right so boom
it like appeared all at once so if you
want to see it token by token this is
much better right look at how fast it is
this is the power of llama 3 8 billion a
small model but a very capable model so
um yeah Tech streamer is great for this
and you can see it how it's generating
the
answer so yeah this is um the next
prompt I Ed myself convert these binary
numbers to decimal and then here and by
the way again you can use these proms
like example create like 20 30 by hand
maybe and then you know feed this into
CH GPD or your team of Agents something
automated ideally or there is I think
there's a service for like like
reflection AI or something like that but
yeah either way you can creating large
data sets need to needs to be automated
right so you cannot do that by hand but
either way like you create something
like this so uh these examples and then
you would feed that into your own data
set obviously relevant to your use case
to your business and you just go crazy
and create as many as as you possibly
can
so like really at least 1,000 like this
is 50,000 and it's still probably could
be larger so yeah I mean you have to
again that's probably another video to
build a team of agents to um generate
data sets but yeah okay so here we give
it um three different binary numbers and
it tasks its tasks is to convert to
decimal and as you can see it does it
flawlessly I mean this is 10 13 15 that
is correct so we have the model we
tested it a bit with two prompts now
it's time to to save it and because you
know we spend all this time all this GPU
power training it we don't want to go
with to waste because if you restart the
run time in Google collab um your model
will disappear obviously you can run it
again but then you have to wait again
and you know maybe run out of the three
GPU hours so to save the final model as
Lura adapters we can either use hugging
face push to hub for an online save if
you wanted your model listed on hugging
phase so hugging phase lists data assets
and models it's probably the main two
things it's used for so if you want your
model shared then you would do that but
if you want it just Sav on your computer
do safe pre-train for a local
save by the way this only saves the
loraa adapters meaning um the like
basically the things that were changed
it doesn't save the entire model with
the change parameters just the changes
right so uh it's less memory and yeah
just faster to save so but if you want
to save the L adapters with the save
model uh you can change this if you want
to load the L adapters we saved for
inference you would change this false to
true so simply changing
this and yeah this is the model name so
obviously you can change this this is
your model used for training uh here is
just laa model but you can name it lava
free I don't know copyrighting or Lama
free uh medical diagnosis whatever your
um use case is obviously and then
um here the alpaka prompt so yeah this
is the variable we declared earlier so
this is the importance you can just go
into the collab and try to running this
cell you have to run the cells from
above otherwise this will not work so
whenever you're using a jupyter notebook
such as Google collab always run all
cells in order otherwise it will not
work so
yeah so this is the same uh format right
from earlier inst ction input output at
this point you should be familiar with
this and that's just for this particular
data set and for this style of prompting
so if you have a different one then
follow the different one so obviously
here U what is the famous St Tower in
Paris obviously it's Eiffel Tower blah
blah blah it gives some extra info about
it so you can also use hugging face Auto
model for perf casual LM but ANS slof
does not recommend this because it's a
lot slower than ANS slof
so yeah uh if possible use unslow for
Speed and as the name suggests of you
know unlove it's UNS slowing everything
it's making everything two to five times
faster so why not do that with 80% less
memory so
yeah okay and then we're preparing to
save our trained model in a more compact
format and then upload it into a cloud
platform which allows for Less storage
and comparation power so again like I'm
not going to even pretend I understand
everything because this is honestly
stepping outside of my comfort zone but
like just building this and doing this
fine tuning taught me a lot so if you
want more technical videos like this let
me know next we're ready to compress our
model using various quantization methods
which means just you know making it
easier to run or a machine so maybe you
cannot like if you have a bad computer
maybe you cannot run the full model but
you definitely can run a quantized
version of it it makes it leaner and
then uh we upload it to the cloud for
easy sh in this is what this piece of
code does
and so we use the model un. GF file or
the quantise version so the Q4 means
quanti in Lama CCP or if you want a UI
based system which probably you do which
is easier to use you can use like GPT
for or or um is the other one is
escaping me but yeah these are basically
these USB system that you can use
to or llm anything yeah I don't know if
this supports it but yeah basically
these uh these U Frameworks have a UI
that's easy to chat with and you can use
open source model there so if you do if
you find this you can upload this to GPD
for all and chat with your own model
very easily and yeah that's it you know
how to fine tune Lama 3 for your your
own specific use case again I'm going to
leave these resources below the video
and if you have any questions regarding
to anso join the their Discord so yeah
that's it if you find this useful then
please subscribe and again if you want
during April which is what like eight
days nine days left if you join the
community you will get a personalized AI
strategy to Future prooof yourself and
your business so if that sounds valuable
to you then make sure to join it's the
first link in the description thank you
for watching