[00:00]
so you want better mouth tracking I can
[00:02]
probably help with that hi what's up I'm
[00:04]
cozy Chu and if you don't know me
[00:06]
already I actually made one of these
[00:08]
guides before but on my main Channel and
[00:10]
now I'm just remaking it uh with a
[00:12]
little bit more depth I wanted to make
[00:15]
sure that there was a guide out there
[00:18]
specifically for mouth tracking and like
[00:20]
changing settings and stuff like that
[00:22]
for both iPhone and webcam keep in mind
[00:25]
that this is something specifically for
[00:27]
vtube studio and it's not for any other
[00:30]
tracking programs and it's not for 3D
[00:32]
and it's only for 2D avatars all right
[00:35]
okay so we're going to start with iPhone
[00:36]
tracking but we're going to start with
[00:37]
something really specific with iPhone
[00:39]
tracking that I don't think a lot of
[00:40]
people talk about and that is the actual
[00:43]
settings on your iPhone for your
[00:45]
tracking so you'll have to go to the app
[00:47]
you'll have to go to this little camera
[00:48]
button here and you'll have to scroll
[00:50]
down to specifically these settings the
[00:54]
blink sensitivity eye open sensitivity
[00:56]
mouth open sensitivity all of this is
[00:58]
controlled specifically by your iPhone
[01:01]
settings it's not controlled in app and
[01:04]
if you change any of these settings on
[01:06]
Steam then it's not actually going to do
[01:09]
anything so you'll have to change if you
[01:11]
want your blink sensitivity to be higher
[01:13]
if you want your eye open sensitivity to
[01:15]
be higher all of this stuff on here is
[01:18]
controlled by your iPhone so make sure
[01:21]
that you make these changes here first
[01:24]
um same thing with if you if you don't
[01:27]
want your eye blinking linked or if you
[01:29]
do want your eye blinking linked or
[01:31]
something like that you have to control
[01:32]
that through here not through the Steam
[01:35]
app there's also a blink fix thing if
[01:38]
you need to mess with that I'm going to
[01:40]
talk about mostly mouth tracking just
[01:43]
make sure that you mess around with your
[01:45]
settings in the iPhone app as well as
[01:50]
your actual vtube studio app sorry if my
[01:52]
tracking is a little wonky it's because
[01:53]
I'm waving my my hand in front of
[01:56]
my
[01:57]
iPhone a lot of those set are really
[02:00]
self-explanatory so I'm not really going
[02:02]
to go over them I have created a
[02:04]
different export of my model that is
[02:07]
specifically for iPhone tracking that
[02:09]
doesn't have any settings on it so I'm
[02:11]
going to start from a completely fresh
[02:13]
State as if I just have uploaded this
[02:15]
model for the first time usually I start
[02:17]
with the auto setup because that's a
[02:18]
good starting place the auto setup sets
[02:21]
things up kind of weirdly sometimes if
[02:23]
we zoom in here you can see there's
[02:26]
something weird going on with my cheek
[02:27]
puff I find this happens every single
[02:30]
time this might not be the case for you
[02:32]
um from a model that you got from a
[02:34]
different rigger just if it does happen
[02:37]
if this is something that's going on for
[02:39]
you make sure you go into your settings
[02:41]
into little person icon at the top
[02:43]
scroll down and try to find where your
[02:46]
cheek puff setting is mine
[02:50]
is down near the
[02:53]
bottom um here we go so it says Pam
[02:56]
cheek and right now it's linked to mouth
[02:58]
smile what you need the the input to be
[03:01]
is instead cheek puff this is an iPhone
[03:05]
only thing so if if you don't have an
[03:08]
iPhone then don't worry about it but
[03:11]
we'll get to that in the actual webcam
[03:14]
part okay so you can turn that on for me
[03:18]
I also need to do something like this to
[03:20]
make sure that it doesn't do a cheek
[03:21]
puff like at the start and
[03:25]
then that's what happens when I like
[03:27]
puff my cheeks out and stuff I'm
[03:28]
actually going to make this like point
[03:30]
or something I don't
[03:31]
know okay so the first problem that I
[03:33]
like to solve after my cheek puff is I
[03:35]
like to make sure that my mouth can
[03:38]
close usually I have a lot of issues
[03:39]
with that because I have really big lips
[03:41]
so I'll go to the mouth open parameter
[03:45]
and I'll go down to the bottom here and
[03:47]
I will make the bottom just something
[03:48]
like
[03:50]
0.03 something really
[03:52]
small so that my mouth closes and make
[03:56]
sure that it completely closes no matter
[03:58]
what so that's my first issue second um
[04:03]
if you watch this red dot on both the
[04:07]
mouth smile and mouth open parameters
[04:10]
you can get an idea of how much you're
[04:13]
getting out of your mouth tracking as
[04:15]
you can see on the mouth form it only
[04:18]
really stays in the middle and then goes
[04:21]
up and then on the mouth open it only
[04:24]
really stays on the bottom and hits the
[04:27]
middle so I'm not getting the most out
[04:29]
of my mouth tracking right now so the
[04:31]
best way to deal with this is
[04:34]
usually to go to this in part
[04:38]
here and if you want to make it so that
[04:40]
it goes lower if you want you know the
[04:43]
the bottom point to actually hit this
[04:45]
bottom here you want to make the bottom
[04:48]
number a little bit bigger so I'm going
[04:49]
to make mine. 3 so that starts
[04:54]
me at a more default expression it
[04:56]
starts me not quite frowning but I can
[04:59]
still like hit a frown point with my
[05:02]
Expressions um but I'm I'm not
[05:05]
constantly smiling and only able to
[05:08]
smile next thing I'll usually do to get
[05:11]
the most out of my my parameters is to
[05:14]
change the top number and I usually
[05:16]
lower this a tiny bit my default is to
[05:18]
go3 on the bottom and 08 on the top and
[05:21]
you can see that offers a pretty large
[05:24]
range of motion depending on whatever
[05:27]
Expressions I'm using and you could see
[05:29]
the Red Dot like really bumping around
[05:31]
all over the place that's what I
[05:34]
like then we can go back to the mouth
[05:36]
open parameter you can see that usually
[05:39]
it's only really close to the bottom and
[05:42]
it only really hits up to about the
[05:43]
middle portion sometimes that's fine
[05:46]
sometimes you don't want your mouth to
[05:47]
open that wide that's okay if you do
[05:49]
want your mouth to open wider then
[05:51]
you're going to want to lower this top
[05:53]
number again I'm going to go for
[05:55]
something just to show you I'm going to
[05:57]
go for something like 7 so so you can
[06:00]
see that it hits the top range of my
[06:03]
possible mouth open a lot quicker and a
[06:06]
lot easier I usually I think for this
[06:09]
model go for about 0.9 so it's a little
[06:11]
bit higher but it's not like super
[06:14]
overboard um and that is the easy way to
[06:19]
set up your mouth tracking parameters as
[06:22]
you could see I already have a much
[06:24]
better mouth tracking um setup there is
[06:28]
one other thing that you can mess with
[06:30]
to try and make this better you can turn
[06:32]
on the advanced lip sync which is using
[06:35]
my voice to try and tart each one of
[06:38]
these shapes um personally I found that
[06:44]
uh if I just calibrate each of these
[06:47]
like letters as I'm making a sound with
[06:50]
the right shaping for my mouth it
[06:52]
usually makes it a little bit more
[06:53]
accurate so for
[06:55]
instance for this is a a o uh are
[06:59]
different sounds that it's working with
[07:02]
so I'll hit
[07:03]
calibrate
[07:05]
ah and I'll try and make the right mouth
[07:08]
shape as I'm trying to do these
[07:10]
different
[07:14]
sounds No that's not quite what I
[07:19]
want I'm not quite hitting it but you
[07:22]
know what I'm kind of going for there
[07:23]
with the with the teeth closed and stuff
[07:25]
then
[07:30]
[Music]
[07:35]
[Laughter]
[07:37]
o
[07:41]
o and then you could do uh silence
[07:43]
calibration too you could just be
[07:48]
like so yeah that ends up calibrating a
[07:51]
little bit better to my voice I found so
[07:53]
it it listens to whatever sounds I'm
[07:55]
making and it tries to set them to the
[07:58]
right mouth parameters there is a way
[08:00]
for a live TD riger to set it up to have
[08:02]
each one of these different parameters
[08:05]
its own thing um and then you you
[08:09]
usually do this setup in model sort of
[08:11]
thing here I've never really done that
[08:14]
so I can't really explain to you how to
[08:16]
do that there may be another video out
[08:18]
there that does I personally have never
[08:20]
really needed it because I usually get
[08:22]
pretty good mouth tracking without that
[08:24]
so I'm not really going to worry about
[08:26]
it this time around uh maybe next time
[08:30]
but this is basically um set up for
[08:35]
iPhone that that's pretty much how I
[08:36]
would deal with my mouse tracking on
[08:38]
iPhone if we pull back up my other
[08:42]
model I
[08:44]
have this one is already set up with all
[08:47]
my tracking how I usually do it you can
[08:50]
kind of see that you can kind of see
[08:54]
that the Mal tracking is about the same
[08:56]
it's a little bit different on this one
[08:58]
but that's just because I I've tweaked
[08:59]
it a little bit more than that the the
[09:01]
settings are slightly different than
[09:03]
they are um in the you know default
[09:05]
setup I I mess with my settings a
[09:08]
lot but it's a lot better than it was
[09:12]
and you could play with the numbers and
[09:13]
you could try and make things you know
[09:15]
closer to however you want them to be
[09:17]
just in case you were curious what my
[09:19]
default settings are for this model
[09:21]
right now um and how I've have tweaked
[09:23]
things uh you can kind of
[09:25]
see it's not quite what I was saying
[09:30]
for the default one that I was making
[09:31]
but it's pretty close um I also
[09:34]
technically changed the uh maximum on my
[09:37]
mouth open um technically it'll say at
[09:40]
the top 1.2 Max that is the total like
[09:44]
Max points of the rigging um you'll
[09:48]
usually want that number to match with
[09:51]
what the actual Max is because that'll
[09:53]
usually help you get the most out of
[09:54]
your tracking um I think on the the
[09:57]
default one that I put in it was it
[09:59]
something like 2.1 so make sure that
[10:02]
that is about uh what the actual Max is
[10:05]
there or if not quite at least pretty
[10:08]
close in number okay and the last face
[10:11]
tracking thing that you'll have to deal
[10:13]
with with an iPhone specifically the
[10:16]
this is something that only iPhones can
[10:18]
do along with like the cheek puff thing
[10:21]
is mouth X you may or may not have this
[10:24]
rigging um just make sure that you know
[10:26]
if you if you want this sort of rigging
[10:28]
if you want the cheek puff if you want
[10:29]
the mouth X you're telling the person
[10:31]
who is doing your rigging um that you
[10:33]
want these features so that they can
[10:35]
make sure that they're in there but uh
[10:37]
mouth X is
[10:39]
basically the side to side motion of
[10:41]
your mouth so um you may need to set up
[10:45]
this parameter in general it might not
[10:47]
even be here um this is on my default
[10:49]
model so I've already set it up and this
[10:52]
is on the iPhone default model um where
[10:56]
it actually already automatically set it
[10:58]
up so that I can
[11:00]
move side to side and stuff but I like
[11:02]
to make it a little bit easier for me to
[11:05]
actually hit the the mouth um X because
[11:09]
I think it's a little bit nicer if I
[11:11]
don't have to like move my face as much
[11:14]
I I make pretty small facial motions
[11:16]
usually so it's a little bit easier if I
[11:19]
set it up to uh account for that so what
[11:22]
I'll usually do is I'll make the top
[11:25]
like3 and the bottom like netive .3 that
[11:28]
keeps it still in the middle but makes
[11:31]
it a lot easier to hit the mouth X
[11:34]
parameter without like straining your
[11:38]
face okay so now we've set up for iPhone
[11:42]
so I'm going to set up for webcam so I'm
[11:45]
going to do the auto setup
[11:50]
again oh my God I look so mad so my
[11:54]
webcam is set up above my uh normal like
[11:59]
PC setup my iPhone is usually set below
[12:02]
so I've got like a a monitor that you
[12:04]
know it it sits up a little bit oh my
[12:06]
God I look so crazy right
[12:08]
now um I usually find that webcam
[12:12]
tracking is better from above and for
[12:15]
some reason iPhone tracking is better
[12:16]
from below I don't know why that is
[12:19]
that's just what I've found uh let me
[12:22]
fix this freaking cheek puff I'm just
[12:24]
going to get rid of it because like I
[12:27]
said that does not work on webcam oh my
[12:30]
God look at how crazy look at how
[12:33]
insane look how actually insane I
[12:37]
look okay so um on set this to 1.2 Max
[12:43]
okay that already helped quite a bit but
[12:45]
um for webcam tracking my mouth actually
[12:48]
opens a little bit too far now keep in
[12:51]
mind when we started iPhone tracking we
[12:54]
uh changed these things here the
[12:56]
tracking settings all of
[12:59]
these have to be set up in vtube studio
[13:04]
when you're doing webcam tracking so for
[13:07]
instance um I think my mouth open
[13:09]
sensitivity might be too high so I'm
[13:11]
going to lower this and as you can see
[13:14]
that actually already helped quite a bit
[13:16]
same with the eye open sensitivity
[13:19]
Ah that's that does help it helps the
[13:23]
blink sensitivity will lower a little
[13:25]
bit too
[13:27]
um I I do look a little bit insane on
[13:31]
this one um and then the mouth smile
[13:33]
sensitivity we can probably lower too um
[13:36]
so I'm going to start with this and then
[13:38]
we will go back here and take a look at
[13:41]
my mouth open settings and stuff like
[13:43]
that this one instead of having a
[13:45]
problem with my mouth not um staying
[13:47]
closed I'm having a problem with my
[13:49]
mouth closing too much so what I'm
[13:52]
actually going to want to do is I think
[13:55]
lower uh this number
[14:00]
so that is an idea of how you can fix
[14:03]
that I'm going to actually make
[14:07]
it1 still not quite
[14:12]
right3 not quite
[14:22]
ne01 not
[14:25]
quite. you really got to play with
[14:27]
things um and and then I'm going to make
[14:29]
the top part let's
[14:33]
go8 or something like that seems a
[14:35]
little bit better same thing with the
[14:37]
mouth smile currently I'm stuck in mouth
[14:41]
smile hell I'm so far up so I'm actually
[14:45]
going to make this like0 five or
[14:47]
something like that instead oh my God
[14:51]
iph iPhone is what I'm used to At This
[14:54]
Point webcam tracking is insane man
[14:58]
there so many problem problems with this
[15:00]
I'm going to have to do a full webcam uh
[15:02]
tracking setup I'm just going to focus
[15:04]
on the mouth for now though I'm also
[15:06]
going to
[15:08]
reset this to default values um just
[15:13]
to actually let me go to simple lip sync
[15:16]
just for a
[15:17]
moment so okay
[15:21]
so my God I'm sorry I'm just I'm
[15:24]
laughing at myself I'm I look so insane
[15:26]
right now I look crazy oh oh my God
[15:30]
webcam is wild okay so just a little bit
[15:34]
of an update I changed a couple of
[15:35]
things um this is my normal model with
[15:39]
webcam tracking as you can see the
[15:40]
webcam is tracking my face and stuff
[15:43]
like that already um these are the
[15:45]
settings that I ended up at and actually
[15:48]
I swapped from automatic to manual which
[15:52]
seemed to help a lot with the parameter
[15:54]
setup that I'm doing on here and then I
[15:56]
am also still using the simple lip sync
[15:59]
right now so let me go ahead and go into
[16:01]
my face settings on here and show you
[16:03]
what I've got for the mouth smile this
[16:05]
is the same as I use my default mouth
[16:08]
open this is the same as I use my
[16:10]
default mouth X I haven't changed
[16:13]
anything in here from like my normal
[16:15]
default stuff um technically I've still
[16:19]
got my like mouth X and stuff but if I
[16:22]
try and wiggle my mouth it doesn't do
[16:24]
anything because I am in I'm completely
[16:26]
in webcam mode now if if I swap from
[16:30]
webcam to normal tracking you can get an
[16:33]
idea of the difference between the two
[16:35]
So currently camera is
[16:38]
on camera is off now I'm back to uh my
[16:41]
iPhone tracking again so that is a
[16:45]
difference it's quite a big difference
[16:48]
but that's a difference between webcam
[16:50]
and iPhone tracking iPhone tracking is
[16:52]
just always going to be better like I
[16:53]
mentioned
[16:56]
um but you can at least set it up to be
[16:59]
better and look a lot more smooth
[17:02]
overall so definitely mess with your
[17:04]
settings if you want to try and copy the
[17:07]
settings that I've got here and see if
[17:08]
that works for you that's fine but make
[17:10]
sure that everyone's face is different
[17:12]
right so you got to make sure you do
[17:14]
edits for yourself I had to do certain
[17:17]
things because things like I have a a
[17:20]
little bit of a lazy eye sometimes I've
[17:23]
got I've got problems with my lips being
[17:24]
too big Etc just make the changes that
[17:27]
you need and keep twe
[17:29]
until it you think it looks good so this
[17:31]
isn't set up for an entire model there
[17:33]
are other things that I like to make
[17:35]
changes on on model setup so I will do
[17:38]
probably a second video for that later
[17:40]
on especially things like uh tweaking
[17:43]
your eye settings so that you smile more
[17:46]
tweaking things like um adding in a Tail
[17:49]
Wag sort of thing stuff like that a lot
[17:52]
of this stuff will be done by your model
[17:55]
Creator before you get the model so you
[17:58]
only only really need to make changes if
[18:01]
this is like your first time ever
[18:03]
setting up a model and you rigged it
[18:04]
yourself or something or um if you are
[18:08]
noticing that you're having a lot of
[18:10]
issues with your face tracking just not
[18:12]
being what you want it to be so just
[18:15]
make sure that you are trying to keep in
[18:19]
contact with your rigger if you're
[18:20]
having any issues with tracking um talk
[18:23]
to them about it as well as trying to
[18:25]
make changes see if there's anything
[18:27]
that they can help you with ask
[18:30]
questions don't ask me questions I don't
[18:32]
know how your model's
[18:34]
rigged so yeah anyway I hope it helps um
[18:39]
see you later byebye