BETTER Mouth Tracking in VTube Studio (iPhone 10+ AND Webcam!)

[00:00] so you want better mouth tracking I can

[00:02] probably help with that hi what's up I'm

[00:04] cozy Chu and if you don't know me

[00:06] already I actually made one of these

[00:08] guides before but on my main Channel and

[00:10] now I'm just remaking it uh with a

[00:12] little bit more depth I wanted to make

[00:15] sure that there was a guide out there

[00:18] specifically for mouth tracking and like

[00:20] changing settings and stuff like that

[00:22] for both iPhone and webcam keep in mind

[00:25] that this is something specifically for

[00:27] vtube studio and it's not for any other

[00:30] tracking programs and it's not for 3D

[00:32] and it's only for 2D avatars all right

[00:35] okay so we're going to start with iPhone

[00:36] tracking but we're going to start with

[00:37] something really specific with iPhone

[00:39] tracking that I don't think a lot of

[00:40] people talk about and that is the actual

[00:43] settings on your iPhone for your

[00:45] tracking so you'll have to go to the app

[00:47] you'll have to go to this little camera

[00:48] button here and you'll have to scroll

[00:50] down to specifically these settings the

[00:54] blink sensitivity eye open sensitivity

[00:56] mouth open sensitivity all of this is

[00:58] controlled specifically by your iPhone

[01:01] settings it's not controlled in app and

[01:04] if you change any of these settings on

[01:06] Steam then it's not actually going to do

[01:09] anything so you'll have to change if you

[01:11] want your blink sensitivity to be higher

[01:13] if you want your eye open sensitivity to

[01:15] be higher all of this stuff on here is

[01:18] controlled by your iPhone so make sure

[01:21] that you make these changes here first

[01:24] um same thing with if you if you don't

[01:27] want your eye blinking linked or if you

[01:29] do want your eye blinking linked or

[01:31] something like that you have to control

[01:32] that through here not through the Steam

[01:35] app there's also a blink fix thing if

[01:38] you need to mess with that I'm going to

[01:40] talk about mostly mouth tracking just

[01:43] make sure that you mess around with your

[01:45] settings in the iPhone app as well as

[01:50] your actual vtube studio app sorry if my

[01:52] tracking is a little wonky it's because

[01:53] I'm waving my my hand in front of

[01:56] my

[01:57] iPhone a lot of those set are really

[02:00] self-explanatory so I'm not really going

[02:02] to go over them I have created a

[02:04] different export of my model that is

[02:07] specifically for iPhone tracking that

[02:09] doesn't have any settings on it so I'm

[02:11] going to start from a completely fresh

[02:13] State as if I just have uploaded this

[02:15] model for the first time usually I start

[02:17] with the auto setup because that's a

[02:18] good starting place the auto setup sets

[02:21] things up kind of weirdly sometimes if

[02:23] we zoom in here you can see there's

[02:26] something weird going on with my cheek

[02:27] puff I find this happens every single

[02:30] time this might not be the case for you

[02:32] um from a model that you got from a

[02:34] different rigger just if it does happen

[02:37] if this is something that's going on for

[02:39] you make sure you go into your settings

[02:41] into little person icon at the top

[02:43] scroll down and try to find where your

[02:46] cheek puff setting is mine

[02:50] is down near the

[02:53] bottom um here we go so it says Pam

[02:56] cheek and right now it's linked to mouth

[02:58] smile what you need the the input to be

[03:01] is instead cheek puff this is an iPhone

[03:05] only thing so if if you don't have an

[03:08] iPhone then don't worry about it but

[03:11] we'll get to that in the actual webcam

[03:14] part okay so you can turn that on for me

[03:18] I also need to do something like this to

[03:20] make sure that it doesn't do a cheek

[03:21] puff like at the start and

[03:25] then that's what happens when I like

[03:27] puff my cheeks out and stuff I'm

[03:28] actually going to make this like point

[03:30] or something I don't

[03:31] know okay so the first problem that I

[03:33] like to solve after my cheek puff is I

[03:35] like to make sure that my mouth can

[03:38] close usually I have a lot of issues

[03:39] with that because I have really big lips

[03:41] so I'll go to the mouth open parameter

[03:45] and I'll go down to the bottom here and

[03:47] I will make the bottom just something

[03:48] like

[03:50] 0.03 something really

[03:52] small so that my mouth closes and make

[03:56] sure that it completely closes no matter

[03:58] what so that's my first issue second um

[04:03] if you watch this red dot on both the

[04:07] mouth smile and mouth open parameters

[04:10] you can get an idea of how much you're

[04:13] getting out of your mouth tracking as

[04:15] you can see on the mouth form it only

[04:18] really stays in the middle and then goes

[04:21] up and then on the mouth open it only

[04:24] really stays on the bottom and hits the

[04:27] middle so I'm not getting the most out

[04:29] of my mouth tracking right now so the

[04:31] best way to deal with this is

[04:34] usually to go to this in part

[04:38] here and if you want to make it so that

[04:40] it goes lower if you want you know the

[04:43] the bottom point to actually hit this

[04:45] bottom here you want to make the bottom

[04:48] number a little bit bigger so I'm going

[04:49] to make mine. 3 so that starts

[04:54] me at a more default expression it

[04:56] starts me not quite frowning but I can

[04:59] still like hit a frown point with my

[05:02] Expressions um but I'm I'm not

[05:05] constantly smiling and only able to

[05:08] smile next thing I'll usually do to get

[05:11] the most out of my my parameters is to

[05:14] change the top number and I usually

[05:16] lower this a tiny bit my default is to

[05:18] go3 on the bottom and 08 on the top and

[05:21] you can see that offers a pretty large

[05:24] range of motion depending on whatever

[05:27] Expressions I'm using and you could see

[05:29] the Red Dot like really bumping around

[05:31] all over the place that's what I

[05:34] like then we can go back to the mouth

[05:36] open parameter you can see that usually

[05:39] it's only really close to the bottom and

[05:42] it only really hits up to about the

[05:43] middle portion sometimes that's fine

[05:46] sometimes you don't want your mouth to

[05:47] open that wide that's okay if you do

[05:49] want your mouth to open wider then

[05:51] you're going to want to lower this top

[05:53] number again I'm going to go for

[05:55] something just to show you I'm going to

[05:57] go for something like 7 so so you can

[06:00] see that it hits the top range of my

[06:03] possible mouth open a lot quicker and a

[06:06] lot easier I usually I think for this

[06:09] model go for about 0.9 so it's a little

[06:11] bit higher but it's not like super

[06:14] overboard um and that is the easy way to

[06:19] set up your mouth tracking parameters as

[06:22] you could see I already have a much

[06:24] better mouth tracking um setup there is

[06:28] one other thing that you can mess with

[06:30] to try and make this better you can turn

[06:32] on the advanced lip sync which is using

[06:35] my voice to try and tart each one of

[06:38] these shapes um personally I found that

[06:44] uh if I just calibrate each of these

[06:47] like letters as I'm making a sound with

[06:50] the right shaping for my mouth it

[06:52] usually makes it a little bit more

[06:53] accurate so for

[06:55] instance for this is a a o uh are

[06:59] different sounds that it's working with

[07:02] so I'll hit

[07:03] calibrate

[07:05] ah and I'll try and make the right mouth

[07:08] shape as I'm trying to do these

[07:10] different

[07:14] sounds No that's not quite what I

[07:19] want I'm not quite hitting it but you

[07:22] know what I'm kind of going for there

[07:23] with the with the teeth closed and stuff

[07:25] then

[07:30] [Music]

[07:35] [Laughter]

[07:37] o

[07:41] o and then you could do uh silence

[07:43] calibration too you could just be

[07:48] like so yeah that ends up calibrating a

[07:51] little bit better to my voice I found so

[07:53] it it listens to whatever sounds I'm

[07:55] making and it tries to set them to the

[07:58] right mouth parameters there is a way

[08:00] for a live TD riger to set it up to have

[08:02] each one of these different parameters

[08:05] its own thing um and then you you

[08:09] usually do this setup in model sort of

[08:11] thing here I've never really done that

[08:14] so I can't really explain to you how to

[08:16] do that there may be another video out

[08:18] there that does I personally have never

[08:20] really needed it because I usually get

[08:22] pretty good mouth tracking without that

[08:24] so I'm not really going to worry about

[08:26] it this time around uh maybe next time

[08:30] but this is basically um set up for

[08:35] iPhone that that's pretty much how I

[08:36] would deal with my mouse tracking on

[08:38] iPhone if we pull back up my other

[08:42] model I

[08:44] have this one is already set up with all

[08:47] my tracking how I usually do it you can

[08:50] kind of see that you can kind of see

[08:54] that the Mal tracking is about the same

[08:56] it's a little bit different on this one

[08:58] but that's just because I I've tweaked

[08:59] it a little bit more than that the the

[09:01] settings are slightly different than

[09:03] they are um in the you know default

[09:05] setup I I mess with my settings a

[09:08] lot but it's a lot better than it was

[09:12] and you could play with the numbers and

[09:13] you could try and make things you know

[09:15] closer to however you want them to be

[09:17] just in case you were curious what my

[09:19] default settings are for this model

[09:21] right now um and how I've have tweaked

[09:23] things uh you can kind of

[09:25] see it's not quite what I was saying

[09:30] for the default one that I was making

[09:31] but it's pretty close um I also

[09:34] technically changed the uh maximum on my

[09:37] mouth open um technically it'll say at

[09:40] the top 1.2 Max that is the total like

[09:44] Max points of the rigging um you'll

[09:48] usually want that number to match with

[09:51] what the actual Max is because that'll

[09:53] usually help you get the most out of

[09:54] your tracking um I think on the the

[09:57] default one that I put in it was it

[09:59] something like 2.1 so make sure that

[10:02] that is about uh what the actual Max is

[10:05] there or if not quite at least pretty

[10:08] close in number okay and the last face

[10:11] tracking thing that you'll have to deal

[10:13] with with an iPhone specifically the

[10:16] this is something that only iPhones can

[10:18] do along with like the cheek puff thing

[10:21] is mouth X you may or may not have this

[10:24] rigging um just make sure that you know

[10:26] if you if you want this sort of rigging

[10:28] if you want the cheek puff if you want

[10:29] the mouth X you're telling the person

[10:31] who is doing your rigging um that you

[10:33] want these features so that they can

[10:35] make sure that they're in there but uh

[10:37] mouth X is

[10:39] basically the side to side motion of

[10:41] your mouth so um you may need to set up

[10:45] this parameter in general it might not

[10:47] even be here um this is on my default

[10:49] model so I've already set it up and this

[10:52] is on the iPhone default model um where

[10:56] it actually already automatically set it

[10:58] up so that I can

[11:00] move side to side and stuff but I like

[11:02] to make it a little bit easier for me to

[11:05] actually hit the the mouth um X because

[11:09] I think it's a little bit nicer if I

[11:11] don't have to like move my face as much

[11:14] I I make pretty small facial motions

[11:16] usually so it's a little bit easier if I

[11:19] set it up to uh account for that so what

[11:22] I'll usually do is I'll make the top

[11:25] like3 and the bottom like netive .3 that

[11:28] keeps it still in the middle but makes

[11:31] it a lot easier to hit the mouth X

[11:34] parameter without like straining your

[11:38] face okay so now we've set up for iPhone

[11:42] so I'm going to set up for webcam so I'm

[11:45] going to do the auto setup

[11:50] again oh my God I look so mad so my

[11:54] webcam is set up above my uh normal like

[11:59] PC setup my iPhone is usually set below

[12:02] so I've got like a a monitor that you

[12:04] know it it sits up a little bit oh my

[12:06] God I look so crazy right

[12:08] now um I usually find that webcam

[12:12] tracking is better from above and for

[12:15] some reason iPhone tracking is better

[12:16] from below I don't know why that is

[12:19] that's just what I've found uh let me

[12:22] fix this freaking cheek puff I'm just

[12:24] going to get rid of it because like I

[12:27] said that does not work on webcam oh my

[12:30] God look at how crazy look at how

[12:33] insane look how actually insane I

[12:37] look okay so um on set this to 1.2 Max

[12:43] okay that already helped quite a bit but

[12:45] um for webcam tracking my mouth actually

[12:48] opens a little bit too far now keep in

[12:51] mind when we started iPhone tracking we

[12:54] uh changed these things here the

[12:56] tracking settings all of

[12:59] these have to be set up in vtube studio

[13:04] when you're doing webcam tracking so for

[13:07] instance um I think my mouth open

[13:09] sensitivity might be too high so I'm

[13:11] going to lower this and as you can see

[13:14] that actually already helped quite a bit

[13:16] same with the eye open sensitivity

[13:19] Ah that's that does help it helps the

[13:23] blink sensitivity will lower a little

[13:25] bit too

[13:27] um I I do look a little bit insane on

[13:31] this one um and then the mouth smile

[13:33] sensitivity we can probably lower too um

[13:36] so I'm going to start with this and then

[13:38] we will go back here and take a look at

[13:41] my mouth open settings and stuff like

[13:43] that this one instead of having a

[13:45] problem with my mouth not um staying

[13:47] closed I'm having a problem with my

[13:49] mouth closing too much so what I'm

[13:52] actually going to want to do is I think

[13:55] lower uh this number

[14:00] so that is an idea of how you can fix

[14:03] that I'm going to actually make

[14:07] it1 still not quite

[14:12] right3 not quite

[14:22] ne01 not

[14:25] quite. you really got to play with

[14:27] things um and and then I'm going to make

[14:29] the top part let's

[14:33] go8 or something like that seems a

[14:35] little bit better same thing with the

[14:37] mouth smile currently I'm stuck in mouth

[14:41] smile hell I'm so far up so I'm actually

[14:45] going to make this like0 five or

[14:47] something like that instead oh my God

[14:51] iph iPhone is what I'm used to At This

[14:54] Point webcam tracking is insane man

[14:58] there so many problem problems with this

[15:00] I'm going to have to do a full webcam uh

[15:02] tracking setup I'm just going to focus

[15:04] on the mouth for now though I'm also

[15:06] going to

[15:08] reset this to default values um just

[15:13] to actually let me go to simple lip sync

[15:16] just for a

[15:17] moment so okay

[15:21] so my God I'm sorry I'm just I'm

[15:24] laughing at myself I'm I look so insane

[15:26] right now I look crazy oh oh my God

[15:30] webcam is wild okay so just a little bit

[15:34] of an update I changed a couple of

[15:35] things um this is my normal model with

[15:39] webcam tracking as you can see the

[15:40] webcam is tracking my face and stuff

[15:43] like that already um these are the

[15:45] settings that I ended up at and actually

[15:48] I swapped from automatic to manual which

[15:52] seemed to help a lot with the parameter

[15:54] setup that I'm doing on here and then I

[15:56] am also still using the simple lip sync

[15:59] right now so let me go ahead and go into

[16:01] my face settings on here and show you

[16:03] what I've got for the mouth smile this

[16:05] is the same as I use my default mouth

[16:08] open this is the same as I use my

[16:10] default mouth X I haven't changed

[16:13] anything in here from like my normal

[16:15] default stuff um technically I've still

[16:19] got my like mouth X and stuff but if I

[16:22] try and wiggle my mouth it doesn't do

[16:24] anything because I am in I'm completely

[16:26] in webcam mode now if if I swap from

[16:30] webcam to normal tracking you can get an

[16:33] idea of the difference between the two

[16:35] So currently camera is

[16:38] on camera is off now I'm back to uh my

[16:41] iPhone tracking again so that is a

[16:45] difference it's quite a big difference

[16:48] but that's a difference between webcam

[16:50] and iPhone tracking iPhone tracking is

[16:52] just always going to be better like I

[16:53] mentioned

[16:56] um but you can at least set it up to be

[16:59] better and look a lot more smooth

[17:02] overall so definitely mess with your

[17:04] settings if you want to try and copy the

[17:07] settings that I've got here and see if

[17:08] that works for you that's fine but make

[17:10] sure that everyone's face is different

[17:12] right so you got to make sure you do

[17:14] edits for yourself I had to do certain

[17:17] things because things like I have a a

[17:20] little bit of a lazy eye sometimes I've

[17:23] got I've got problems with my lips being

[17:24] too big Etc just make the changes that

[17:27] you need and keep twe

[17:29] until it you think it looks good so this

[17:31] isn't set up for an entire model there

[17:33] are other things that I like to make

[17:35] changes on on model setup so I will do

[17:38] probably a second video for that later

[17:40] on especially things like uh tweaking

[17:43] your eye settings so that you smile more

[17:46] tweaking things like um adding in a Tail

[17:49] Wag sort of thing stuff like that a lot

[17:52] of this stuff will be done by your model

[17:55] Creator before you get the model so you

[17:58] only only really need to make changes if

[18:01] this is like your first time ever

[18:03] setting up a model and you rigged it

[18:04] yourself or something or um if you are

[18:08] noticing that you're having a lot of

[18:10] issues with your face tracking just not

[18:12] being what you want it to be so just

[18:15] make sure that you are trying to keep in

[18:19] contact with your rigger if you're

[18:20] having any issues with tracking um talk

[18:23] to them about it as well as trying to

[18:25] make changes see if there's anything

[18:27] that they can help you with ask

[18:30] questions don't ask me questions I don't

[18:32] know how your model's

[18:34] rigged so yeah anyway I hope it helps um

[18:39] see you later byebye