[00:00] The brand new sleep algorithm shipped with all Fitbit and Google devices like the Fitbit Air right here but also the Pixel Watch is supposed to be 15% better. But can we verify that in independent testing?
[00:14] Well, that's exactly what we'll do in this video and we'll also take an initial look at the new Google Health app and Google Code to see if this has also improved the experience. Now, I'm not allowed to actually test anything on the Fitbit Air itself, but I do a detailed
[00:30] test of every aspect of the Fitbit Air and that video will be released in a few days on Tuesday. That includes different exercises like spinning, running, cycling and weightlifting so you can see how it performs for all those exercises, sleep stage tracking and also other aspects
[00:46] like automatic workout attractions, comparison to the whoop strap, anime sit and even step counting. definitely want to be back for that video because that's going to provide you with the most comprehensive data-driven and in-depth testing out there. Now by the way for those of you that are
[01:01] new to the channel my name is Rob and I'm a postdoctoral scientist specializing in biological data analysis and that's also all focused on in this video data analysis. But like I said at the end of this video I'll also give you my initial impressions of the new Google Health app and
[01:16] Google Health Coach so you can decide whether or not that's helpful for you and if it might be a good pairing for you with the Fitbit Air. But let's now get to the data and the testing. As I said, I'm not allowed to test the sleep stage tracking specifically on the Fitbit Air yet, however,
[01:31] luckily, the same sleep stage tracking is already available on other Google and Fitbit devices. That means that the exact same sleep stage tracking algorithm is used on this Pixel Watch. And given that the basic measurements needed for the sleep stage tracking are really easy at night,
[01:46] I don't expect major differences between any of the Fitbit and Google products, at least not any recent ones. So let's have a look at how the new sleep stage tracking of Fitbit slash Google compares to the old sleep stage tracking.
[02:00] And I actually want to start with this overview right here. Now, regular viewers will be familiar with it, but basically I tested the sleep stage tracking of the new Fitbit algorithm against the ZMesh EEG headband, which is an EEG monitor that can measure my brainwaves
[02:15] and was specifically designed for sleep stage tracking in scientific studies. It's sort of a silver standard, not perfect, but good enough to give us an impression of the performance of different devices. And I then calculate the average sensitivity over the different sleep stages,
[02:30] so D-sleep, light sleep, and REM sleep. We'll look at those details in a second, but first I want to show you the differences between the old and new algorithm. Now, the average sensitivity is here along the horizontal axis, but I also want to look at the worst sleep stage.
[02:43] So which of those three C-stages is it worth? And that number is going to be along those vertical axes. So we want the devices to be as far to the top right as possible. And actually, we have the old testing of the Pixel watches.
[02:55] So the 45 and 41 millimeter version right here. So this is the old algorithm. And this is the result for the new algorithm. So there seems to be a minor improvement in this initial test, at least. So that's already a good sign.
[03:08] The fact that indeed, compared to what Google claims, there does appear to be a minor improvement. Google says 15%, now 15% sounds like a lot, but it's 15% compared to the old algorithm.
[03:20] So not a 15 improvement along this axis but like a 1 multiplication of the old number Now in a second I actually want to compare Google values to the ones they put online as their official testing to my testing
[03:33] but let's first look at the detailed sleep stage tracking of this Pixel Watch 4 that I retested. And that is shown in this confusion matrix right here with the reference on top, so these are the speed stages according to the reference,
[03:47] and the sleep stages according to the new algorithm here on the vertical axis. And what we want is these values here on the diagonal to be as close to 100% as possible.
[03:59] And they're actually quite good. Deep sleep was detected at an 86% sensitivity, which is actually really good. Light sleep was pretty good at about 78%. And REM sleep, at least in my testing, was the worst at 71%.
[04:14] But it's still pretty good compared to much of the competition. I just want to show you one example night. This is actually last night. So on top here we have the reference and on the bottom here we have the pixel watch 4 with the new
[04:29] Algorithm so we have the sleep stages here along the vertical axis and if you first look at deep sleep The first deep sleep segment agrees almost perfectly But the pixel watch does detect a bit of extra deep sleep now
[04:41] This is actually a sort of typical sleep taste pattern for me where I was one or two two sometimes deep sleep segments only and less so near the end of the night also compared to other people even at least that's what it seems to me sort of subjectively and the pixel watch
[04:55] did get back a little bit of extra but not that much mainly this segment is extra i would say now REM sleep agreement was also quite okay so i actually had somewhat fragmented sleep with more
[05:07] awake time here at the end of the night but there was still a really good agreement here between the different REM sleep segments. Only here in the beginning was there a slight disagreement, so it was a bit shifted here according to the Pixel Watch 4, but
[05:20] otherwise actually pretty solid. So roughly the same segments were overall detected, so I'm quite happy with that performance. But as I said, I actually want to compare this performance to what Sizzit actually claims the performance is.
[05:33] And this is actually their white paper that they sort of published, not in a scientific journal, but on their own website. And this is similar to what we we're looking at in this overview right here so these are similar numbers but now as published
[05:45] by Fitbit so it's now flipped so we have the reference here and here on the bottom is the new algorithm of Fitbit and Google but again we want these numbers right here to be as high as
[05:59] possible and specifically I want to focus on light sleep deep sleep and REM sleep again we see actually very similar numbers so now in a way deep sleep and REM sleep are flipped compared to
[06:12] my testing google actually sort of shows that their deep sleep is a little bit worse than their REM sleep whereas for me this was the other way around but if we now plot this data in my plot we do actually see it ends up at almost the identical location right here because that
[06:29] average agreement of REM and deep was sort of flipped but overall this does seem to more or less confirm the testing that Google and Fitbit did. As I said, I cannot show the Fitbit Air yet but I have a good feeling about it.
[06:41] Now regular viewers will be familiar with the overall top performers. The fact that Google and Fitbit are amongst those hasn't changed, they might have just upgraded their sleep stage tracking on average a little bit The other 4 major good performers in my testing are the Apple Watch the Oura Ring the Whoop Strap and the 8 Sleep Pops So we stick now with the same top five brands
[07:04] but as I said, potentially with a slight improvement in the algorithm of Google and Fitbit, which will hopefully translate into better data also ending up in your app, so in the new Google Health app. So let's take a quick look at my first experience of the Google Health app with the
[07:20] Fitbit Air because that I'm allowed to share. I'm not allowed to test the data but I am allowed to give you my initial impression of how this app actually performed. Okay so let's zoom in a bit to the actual app and the app experience. So the first thing I noticed is that there's really a
[07:34] focus on your weekly metrics. So instead of focusing initially at least on the day-to-day variation Google and Fitbit really tried to focus on your weekly overview which has its up and
[07:46] downsides I would say. So if we zoom in here we can see that for this week I already reached my weekly target of cardio and here you can see some of the different exercises that I did. Some of
[07:58] these were auto-tracked, some of these were manually tracked but more on the auto-tracking in the future video. Then there's the sleep stage tracking which potentially has improved so I hope it has. Now one potential quite powerful thing in this app now is it can learn from you and what
[08:16] your activity profile sort of look like so if it auto detects certain workouts like analytical or rowing it learns from your movement next time to actually classify that automatically so tag it
[08:28] automatically as that workout so that's actually really good what i would have slightly preferred is for the focus on the first home screen to be more on your health so that when you open the app in the morning that's your first overview like is anything out of range was there anything off
[08:42] that doesn't appear to be the case I always start in this screen right here of course the google coach if you have freemium does give you some summaries right here I'll get to those in a second I like to always check my basic metrics compared to some of the competitors it is a bit more of a
[08:57] cluttered view I would say so there's a lot of data which is good and it's hard not to make that cluttered but sometimes it still feels like a bit of an overload compared to some of the competition maybe I still have to get used to it.
[09:09] And part of that has to do with this textual coach thing. For me, this isn't always the right way to start. I sometimes just first want to see the numbers and have some first impression of it
[09:21] and only get a more detailed view when I ask for it. But so far, a lot of the individual logging things immediately seem to get some text with it. So this is, for instance, a textual thing to my sleeping, which is on top of the actual data.
[09:35] Now I would have preferred for this right here to be front and center because this is the important part. And then if I wanted to, to get the extension of the text. Because I just respond better to numbers and visuals than to text myself.
[09:49] With the new AI model being developed, it's almost natural for companies to put this text here on top. But I would prefer this view right here as the initial view or maybe the health view and not to have this text on top.
[10:01] And in general, I feel like the text is very much on focus. So you get a lot of descriptions of each of the things that you did. And I'm not sure if this is just a bit too much for me.
[10:13] I would prefer shorter summaries and only to get extended text if I click for it. But this is again very personal Overall it a very complete app and it does have all the data that i looking for i would just like more of a visual representation of things so for the text to be secondary
[10:30] and the data and the graphs to be the main thing and that i can click through and get to the things actually i look at so here it's nice i click on the sleep tab i get my sleep info i can look at
[10:42] how things are compared to normal and then if i wanted to then i put off for more detailed interpretations from Gemini. Again, it's not bad at all. It's actually really good, but there are still some minor improvements. For me, it would take away from the clutter to have
[10:56] this text being secondary. And for me, also, this weekly overview is nice, but it wouldn't have to be on top the moment I open the app in the morning. Maybe during the day that makes sense, but my first view, in my opinion, should always be something like this. And then as the day
[11:10] progresses, when the app knows I've actually viewed this, then we could switch to this. Overall, I quite like the app, though it's not quite as clean as some of the competition. And then for the sleep face tracking tab, this actually looks kind of nice.
[11:22] So here I can see all the data and the text at once, and it's not too much text necessarily. So here the balance is a little bit better in my opinion. Also here there's always text with a graph and not just a graph.
[11:35] And now we can actually ask coach about something. So I did quite a lot of running and cycling this week for my marathon training plan. what do you recommend this weekend not to overload and overtrain so now it's compiling its answer
[11:47] now it actually came up with an answer right here indeed i only did a single run of 8k but my goal was 40 kilometers so it does say i need more time on feet and it's non-negotiable
[11:59] but then it actually also says maybe it's not non-negotiable keep the running easy so it actually recommends four to five kilometers and it says i'm not failing by not hitting the 40 kilometers. I'm not exactly sure how I feel about that particular advice because it did contradict
[12:15] itself in a way but what I said in the end did make sense. I would go personally for a slightly longer run because I still have a whole weekend ahead but of course I could have a conversation with the coach and we would probably end up where I wanted it to end up. This is just my first
[12:29] experience I think compared to some other marathon plans where they just generate it for you here there's more of a conversation which on the good side gives you more influence but if like me you don't know much about creating marathon plans maybe that's also a bad thing i'm really curious
[12:45] also to hear about your experiences have you already tried the new google health app and do you have premium or not because of course i'll briefly discuss that as well the differences between premium and non-premium in my full review of the google fitbit air now i hope that this video
[13:00] is useful for you if you want to help me out it would really help if you like this video and subscribe to the channel i'm trying to get to 400k i know for the algorithm it doesn't matter that much more how much subscribers you have but I do think it's a nice milestone.
[13:13] And if you want to support in other ways using one of my affiliate links down in the description below usually gives you the best discount possible whether it be on the whoop strap, maybe you like the aura ring, my favorite sleep improvement device the AC pod, maybe
[13:27] you're gonna buy on Amazon anything like this is it air. If you want to support using my affiliate links never cost you any extra and almost always gives you the best discount possible. Now given that you watched this review of the new sleep stage algorithm present on the
[13:40] new Pixel watches and the Fitbit Air, I think you will like this video that actually tells you more about what the Fitbit Air can and cannot do and this video on the A3 part.