[0:00] so you want better mouth tracking I can
[0:02] probably help with that hi what's up I'm
[0:04] cozy Chu and if you don't know me
[0:06] already I actually made one of these
[0:08] guides before but on my main Channel and
[0:10] now I'm just remaking it uh with a
[0:12] little bit more depth I wanted to make
[0:15] sure that there was a guide out there
[0:18] specifically for mouth tracking and like
[0:20] changing settings and stuff like that
[0:22] for both iPhone and webcam keep in mind
[0:25] that this is something specifically for
[0:27] vtube studio and it's not for any other
[0:30] tracking programs and it's not for 3D
[0:32] and it's only for 2D avatars all right
[0:35] okay so we're going to start with iPhone
[0:36] tracking but we're going to start with
[0:37] something really specific with iPhone
[0:39] tracking that I don't think a lot of
[0:40] people talk about and that is the actual
[0:43] settings on your iPhone for your
[0:45] tracking so you'll have to go to the app
[0:47] you'll have to go to this little camera
[0:48] button here and you'll have to scroll
[0:50] down to specifically these settings the
[0:54] blink sensitivity eye open sensitivity
[0:56] mouth open sensitivity all of this is
[0:58] controlled specifically by your iPhone
[1:01] settings it's not controlled in app and
[1:04] if you change any of these settings on
[1:06] Steam then it's not actually going to do
[1:09] anything so you'll have to change if you
[1:11] want your blink sensitivity to be higher
[1:13] if you want your eye open sensitivity to
[1:15] be higher all of this stuff on here is
[1:18] controlled by your iPhone so make sure
[1:21] that you make these changes here first
[1:24] um same thing with if you if you don't
[1:27] want your eye blinking linked or if you
[1:29] do want your eye blinking linked or
[1:31] something like that you have to control
[1:32] that through here not through the Steam
[1:35] app there's also a blink fix thing if
[1:38] you need to mess with that I'm going to
[1:40] talk about mostly mouth tracking just
[1:43] make sure that you mess around with your
[1:45] settings in the iPhone app as well as
[1:50] your actual vtube studio app sorry if my
[1:52] tracking is a little wonky it's because
[1:53] I'm waving my my hand in front of
[1:56] my
[1:57] iPhone a lot of those set are really
[2:00] self-explanatory so I'm not really going
[2:02] to go over them I have created a
[2:04] different export of my model that is
[2:07] specifically for iPhone tracking that
[2:09] doesn't have any settings on it so I'm
[2:11] going to start from a completely fresh
[2:13] State as if I just have uploaded this
[2:15] model for the first time usually I start
[2:17] with the auto setup because that's a
[2:18] good starting place the auto setup sets
[2:21] things up kind of weirdly sometimes if
[2:23] we zoom in here you can see there's
[2:26] something weird going on with my cheek
[2:27] puff I find this happens every single
[2:30] time this might not be the case for you
[2:32] um from a model that you got from a
[2:34] different rigger just if it does happen
[2:37] if this is something that's going on for
[2:39] you make sure you go into your settings
[2:41] into little person icon at the top
[2:43] scroll down and try to find where your
[2:46] cheek puff setting is mine
[2:50] is down near the
[2:53] bottom um here we go so it says Pam
[2:56] cheek and right now it's linked to mouth
[2:58] smile what you need the the input to be
[3:01] is instead cheek puff this is an iPhone
[3:05] only thing so if if you don't have an
[3:08] iPhone then don't worry about it but
[3:11] we'll get to that in the actual webcam
[3:14] part okay so you can turn that on for me
[3:18] I also need to do something like this to
[3:20] make sure that it doesn't do a cheek
[3:21] puff like at the start and
[3:25] then that's what happens when I like
[3:27] puff my cheeks out and stuff I'm
[3:28] actually going to make this like point
[3:30] or something I don't
[3:31] know okay so the first problem that I
[3:33] like to solve after my cheek puff is I
[3:35] like to make sure that my mouth can
[3:38] close usually I have a lot of issues
[3:39] with that because I have really big lips
[3:41] so I'll go to the mouth open parameter
[3:45] and I'll go down to the bottom here and
[3:47] I will make the bottom just something
[3:48] like
[3:50] 0.03 something really
[3:52] small so that my mouth closes and make
[3:56] sure that it completely closes no matter
[3:58] what so that's my first issue second um
[4:03] if you watch this red dot on both the
[4:07] mouth smile and mouth open parameters
[4:10] you can get an idea of how much you're
[4:13] getting out of your mouth tracking as
[4:15] you can see on the mouth form it only
[4:18] really stays in the middle and then goes
[4:21] up and then on the mouth open it only
[4:24] really stays on the bottom and hits the
[4:27] middle so I'm not getting the most out
[4:29] of my mouth tracking right now so the
[4:31] best way to deal with this is
[4:34] usually to go to this in part
[4:38] here and if you want to make it so that
[4:40] it goes lower if you want you know the
[4:43] the bottom point to actually hit this
[4:45] bottom here you want to make the bottom
[4:48] number a little bit bigger so I'm going
[4:49] to make mine. 3 so that starts
[4:54] me at a more default expression it
[4:56] starts me not quite frowning but I can
[4:59] still like hit a frown point with my
[5:02] Expressions um but I'm I'm not
[5:05] constantly smiling and only able to
[5:08] smile next thing I'll usually do to get
[5:11] the most out of my my parameters is to
[5:14] change the top number and I usually
[5:16] lower this a tiny bit my default is to
[5:18] go3 on the bottom and 08 on the top and
[5:21] you can see that offers a pretty large
[5:24] range of motion depending on whatever
[5:27] Expressions I'm using and you could see
[5:29] the Red Dot like really bumping around
[5:31] all over the place that's what I
[5:34] like then we can go back to the mouth
[5:36] open parameter you can see that usually
[5:39] it's only really close to the bottom and
[5:42] it only really hits up to about the
[5:43] middle portion sometimes that's fine
[5:46] sometimes you don't want your mouth to
[5:47] open that wide that's okay if you do
[5:49] want your mouth to open wider then
[5:51] you're going to want to lower this top
[5:53] number again I'm going to go for
[5:55] something just to show you I'm going to
[5:57] go for something like 7 so so you can
[6:00] see that it hits the top range of my
[6:03] possible mouth open a lot quicker and a
[6:06] lot easier I usually I think for this
[6:09] model go for about 0.9 so it's a little
[6:11] bit higher but it's not like super
[6:14] overboard um and that is the easy way to
[6:19] set up your mouth tracking parameters as
[6:22] you could see I already have a much
[6:24] better mouth tracking um setup there is
[6:28] one other thing that you can mess with
[6:30] to try and make this better you can turn
[6:32] on the advanced lip sync which is using
[6:35] my voice to try and tart each one of
[6:38] these shapes um personally I found that
[6:44] uh if I just calibrate each of these
[6:47] like letters as I'm making a sound with
[6:50] the right shaping for my mouth it
[6:52] usually makes it a little bit more
[6:53] accurate so for
[6:55] instance for this is a a o uh are
[6:59] different sounds that it's working with
[7:02] so I'll hit
[7:03] calibrate
[7:05] ah and I'll try and make the right mouth
[7:08] shape as I'm trying to do these
[7:10] different
[7:14] sounds No that's not quite what I
[7:19] want I'm not quite hitting it but you
[7:22] know what I'm kind of going for there
[7:23] with the with the teeth closed and stuff
[7:25] then
[7:30] [Music]
[7:35] [Laughter]
[7:37] o
[7:41] o and then you could do uh silence
[7:43] calibration too you could just be
[7:48] like so yeah that ends up calibrating a
[7:51] little bit better to my voice I found so
[7:53] it it listens to whatever sounds I'm
[7:55] making and it tries to set them to the
[7:58] right mouth parameters there is a way
[8:00] for a live TD riger to set it up to have
[8:02] each one of these different parameters
[8:05] its own thing um and then you you
[8:09] usually do this setup in model sort of
[8:11] thing here I've never really done that
[8:14] so I can't really explain to you how to
[8:16] do that there may be another video out
[8:18] there that does I personally have never
[8:20] really needed it because I usually get
[8:22] pretty good mouth tracking without that
[8:24] so I'm not really going to worry about
[8:26] it this time around uh maybe next time
[8:30] but this is basically um set up for
[8:35] iPhone that that's pretty much how I
[8:36] would deal with my mouse tracking on
[8:38] iPhone if we pull back up my other
[8:42] model I
[8:44] have this one is already set up with all
[8:47] my tracking how I usually do it you can
[8:50] kind of see that you can kind of see
[8:54] that the Mal tracking is about the same
[8:56] it's a little bit different on this one
[8:58] but that's just because I I've tweaked
[8:59] it a little bit more than that the the
[9:01] settings are slightly different than
[9:03] they are um in the you know default
[9:05] setup I I mess with my settings a
[9:08] lot but it's a lot better than it was
[9:12] and you could play with the numbers and
[9:13] you could try and make things you know
[9:15] closer to however you want them to be
[9:17] just in case you were curious what my
[9:19] default settings are for this model
[9:21] right now um and how I've have tweaked
[9:23] things uh you can kind of
[9:25] see it's not quite what I was saying
[9:30] for the default one that I was making
[9:31] but it's pretty close um I also
[9:34] technically changed the uh maximum on my
[9:37] mouth open um technically it'll say at
[9:40] the top 1.2 Max that is the total like
[9:44] Max points of the rigging um you'll
[9:48] usually want that number to match with
[9:51] what the actual Max is because that'll
[9:53] usually help you get the most out of
[9:54] your tracking um I think on the the
[9:57] default one that I put in it was it
[9:59] something like 2.1 so make sure that
[10:02] that is about uh what the actual Max is
[10:05] there or if not quite at least pretty
[10:08] close in number okay and the last face
[10:11] tracking thing that you'll have to deal
[10:13] with with an iPhone specifically the
[10:16] this is something that only iPhones can
[10:18] do along with like the cheek puff thing
[10:21] is mouth X you may or may not have this
[10:24] rigging um just make sure that you know
[10:26] if you if you want this sort of rigging
[10:28] if you want the cheek puff if you want
[10:29] the mouth X you're telling the person
[10:31] who is doing your rigging um that you
[10:33] want these features so that they can
[10:35] make sure that they're in there but uh
[10:37] mouth X is
[10:39] basically the side to side motion of
[10:41] your mouth so um you may need to set up
[10:45] this parameter in general it might not
[10:47] even be here um this is on my default
[10:49] model so I've already set it up and this
[10:52] is on the iPhone default model um where
[10:56] it actually already automatically set it
[10:58] up so that I can
[11:00] move side to side and stuff but I like
[11:02] to make it a little bit easier for me to
[11:05] actually hit the the mouth um X because
[11:09] I think it's a little bit nicer if I
[11:11] don't have to like move my face as much
[11:14] I I make pretty small facial motions
[11:16] usually so it's a little bit easier if I
[11:19] set it up to uh account for that so what
[11:22] I'll usually do is I'll make the top
[11:25] like3 and the bottom like netive .3 that
[11:28] keeps it still in the middle but makes
[11:31] it a lot easier to hit the mouth X
[11:34] parameter without like straining your
[11:38] face okay so now we've set up for iPhone
[11:42] so I'm going to set up for webcam so I'm
[11:45] going to do the auto setup
[11:50] again oh my God I look so mad so my
[11:54] webcam is set up above my uh normal like
[11:59] PC setup my iPhone is usually set below
[12:02] so I've got like a a monitor that you
[12:04] know it it sits up a little bit oh my
[12:06] God I look so crazy right
[12:08] now um I usually find that webcam
[12:12] tracking is better from above and for
[12:15] some reason iPhone tracking is better
[12:16] from below I don't know why that is
[12:19] that's just what I've found uh let me
[12:22] fix this freaking cheek puff I'm just
[12:24] going to get rid of it because like I
[12:27] said that does not work on webcam oh my
[12:30] God look at how crazy look at how
[12:33] insane look how actually insane I
[12:37] look okay so um on set this to 1.2 Max
[12:43] okay that already helped quite a bit but
[12:45] um for webcam tracking my mouth actually
[12:48] opens a little bit too far now keep in
[12:51] mind when we started iPhone tracking we
[12:54] uh changed these things here the
[12:56] tracking settings all of
[12:59] these have to be set up in vtube studio
[13:04] when you're doing webcam tracking so for
[13:07] instance um I think my mouth open
[13:09] sensitivity might be too high so I'm
[13:11] going to lower this and as you can see
[13:14] that actually already helped quite a bit
[13:16] same with the eye open sensitivity
[13:19] Ah that's that does help it helps the
[13:23] blink sensitivity will lower a little
[13:25] bit too
[13:27] um I I do look a little bit insane on
[13:31] this one um and then the mouth smile
[13:33] sensitivity we can probably lower too um
[13:36] so I'm going to start with this and then
[13:38] we will go back here and take a look at
[13:41] my mouth open settings and stuff like
[13:43] that this one instead of having a
[13:45] problem with my mouth not um staying
[13:47] closed I'm having a problem with my
[13:49] mouth closing too much so what I'm
[13:52] actually going to want to do is I think
[13:55] lower uh this number
[14:00] so that is an idea of how you can fix
[14:03] that I'm going to actually make
[14:07] it1 still not quite
[14:12] right3 not quite
[14:22] ne01 not
[14:25] quite. you really got to play with
[14:27] things um and and then I'm going to make
[14:29] the top part let's
[14:33] go8 or something like that seems a
[14:35] little bit better same thing with the
[14:37] mouth smile currently I'm stuck in mouth
[14:41] smile hell I'm so far up so I'm actually
[14:45] going to make this like0 five or
[14:47] something like that instead oh my God
[14:51] iph iPhone is what I'm used to At This
[14:54] Point webcam tracking is insane man
[14:58] there so many problem problems with this
[15:00] I'm going to have to do a full webcam uh
[15:02] tracking setup I'm just going to focus
[15:04] on the mouth for now though I'm also
[15:06] going to
[15:08] reset this to default values um just
[15:13] to actually let me go to simple lip sync
[15:16] just for a
[15:17] moment so okay
[15:21] so my God I'm sorry I'm just I'm
[15:24] laughing at myself I'm I look so insane
[15:26] right now I look crazy oh oh my God
[15:30] webcam is wild okay so just a little bit
[15:34] of an update I changed a couple of
[15:35] things um this is my normal model with
[15:39] webcam tracking as you can see the
[15:40] webcam is tracking my face and stuff
[15:43] like that already um these are the
[15:45] settings that I ended up at and actually
[15:48] I swapped from automatic to manual which
[15:52] seemed to help a lot with the parameter
[15:54] setup that I'm doing on here and then I
[15:56] am also still using the simple lip sync
[15:59] right now so let me go ahead and go into
[16:01] my face settings on here and show you
[16:03] what I've got for the mouth smile this
[16:05] is the same as I use my default mouth
[16:08] open this is the same as I use my
[16:10] default mouth X I haven't changed
[16:13] anything in here from like my normal
[16:15] default stuff um technically I've still
[16:19] got my like mouth X and stuff but if I
[16:22] try and wiggle my mouth it doesn't do
[16:24] anything because I am in I'm completely
[16:26] in webcam mode now if if I swap from
[16:30] webcam to normal tracking you can get an
[16:33] idea of the difference between the two
[16:35] So currently camera is
[16:38] on camera is off now I'm back to uh my
[16:41] iPhone tracking again so that is a
[16:45] difference it's quite a big difference
[16:48] but that's a difference between webcam
[16:50] and iPhone tracking iPhone tracking is
[16:52] just always going to be better like I
[16:53] mentioned
[16:56] um but you can at least set it up to be
[16:59] better and look a lot more smooth
[17:02] overall so definitely mess with your
[17:04] settings if you want to try and copy the
[17:07] settings that I've got here and see if
[17:08] that works for you that's fine but make
[17:10] sure that everyone's face is different
[17:12] right so you got to make sure you do
[17:14] edits for yourself I had to do certain
[17:17] things because things like I have a a
[17:20] little bit of a lazy eye sometimes I've
[17:23] got I've got problems with my lips being
[17:24] too big Etc just make the changes that
[17:27] you need and keep twe
[17:29] until it you think it looks good so this
[17:31] isn't set up for an entire model there
[17:33] are other things that I like to make
[17:35] changes on on model setup so I will do
[17:38] probably a second video for that later
[17:40] on especially things like uh tweaking
[17:43] your eye settings so that you smile more
[17:46] tweaking things like um adding in a Tail
[17:49] Wag sort of thing stuff like that a lot
[17:52] of this stuff will be done by your model
[17:55] Creator before you get the model so you
[17:58] only only really need to make changes if
[18:01] this is like your first time ever
[18:03] setting up a model and you rigged it
[18:04] yourself or something or um if you are
[18:08] noticing that you're having a lot of
[18:10] issues with your face tracking just not
[18:12] being what you want it to be so just
[18:15] make sure that you are trying to keep in
[18:19] contact with your rigger if you're
[18:20] having any issues with tracking um talk
[18:23] to them about it as well as trying to
[18:25] make changes see if there's anything
[18:27] that they can help you with ask
[18:30] questions don't ask me questions I don't
[18:32] know how your model's
[18:34] rigged so yeah anyway I hope it helps um
[18:39] see you later byebye