---
title: 'I tried YouTube A/B Thumbnail Testing in 8 videos and found BIG problems'
source: 'https://youtube.com/watch?v=-NVy6AjhfoA'
video_id: '-NVy6AjhfoA'
date: 2026-06-15
duration_sec: 0
---

# I tried YouTube A/B Thumbnail Testing in 8 videos and found BIG problems

> Source: [I tried YouTube A/B Thumbnail Testing in 8 videos and found BIG problems](https://youtube.com/watch?v=-NVy6AjhfoA)

## Summary

The video critiques YouTube's A/B thumbnail testing feature, arguing that it measures watch time share instead of click-through rate (CTR), which is misleading. The creator tested the feature on 8 videos and found it ineffective, especially for small channels with low impressions.

### Key Points

- **Initial impression of thumbnail testing** [00:00] — The creator initially thought thumbnail testing compares CTR, but YouTube actually uses watch time share to determine the winner.
- **YouTube's explanation** [00:22] — YouTube states that the winning thumbnail is selected based on the highest share of a video's watch time, arguing that great thumbnails help viewers understand the video and make informed decisions.
- **Critique of watch time metric** [01:57] — The creator argues that watch time depends on video delivery, not the thumbnail, and gives an example of the same script delivered with different energy levels leading to different watch times.
- **YouTube's ranking implication** [04:15] — YouTube's statement about ranking videos based on watch time share suggests that channel ranking exists, similar to Google's algorithm, giving big channels an advantage.
- **Small channels disadvantage** [05:50] — Thumbnail testing requires a large sample size, which small channels lack due to low impressions, making the feature useless for them. Tests often take two weeks and end inconclusively.
- **MrBeast's confusion** [06:46] — Even MrBeast initially thought the feature measured CTR, as shown in his social media posts, indicating confusion about the metric.
- **Current usage** [07:20] — The creator still uses the feature with subtle thumbnail changes, but results are often equal, and only once did YouTube declare a winner after one day.
- **Call for transparency** [08:40] — The creator urges YouTube to be more transparent and work with small channels to develop features that actually help them grow, as big channels already have advantages.

### Conclusion

YouTube's A/B thumbnail testing is flawed because it measures watch time instead of CTR, and it disproportionately benefits large channels with high impressions. The creator calls for YouTube to prioritize small channels and be more transparent about how the feature works.

## Transcript

hello hello hello hope you're doing well
all right YouTube thumbnail testing or
AB testing as some would call it it has
been out now for a couple of months and
I have had the chance to do some testing
in eight of my videos and my honest
opinion here is it is good at least
initially until I realized that it's
horrible the way YouTube implemented
this let me start by asking you a simple
question what exactly is thumbnail
testing to you it is basically comparing
two or three thumbnails than checking to
see which one has a higher click through
itate am I right that was my initial
thought and I instinctively think that
that is what it is for so when I did my
testing I looked at test results see
which has the higher percentage then
select that as the winner that is how
the feature is meant to help I give
YouTube Three thumbnails they test it
out for me by swapping at random times
then run the test for me if you have
been using Tu buddies AB thumbnail test
that is how it works as well in fact
that is what anyone with common sense
would think when we say AB testing ask
any U iux designer and they will tell
you that too but YouTube for whatever
reason decided that the thumbnail should
measure watch duration instead of Click
through rate I mean why how on Earth can
a thumbnail decide someone's watch
duration so this is what YouTube has to
say how is the winning thumbnail
selected the winning thumbnail is
selected based on the highest share of a
video's watch time but why is watchtime
share used great thumbnails don't just
get viewers to click they also help
viewers understand what the video is
about so that they can make informed
decision of what to watch all right fair
enough by selecting based on watchtime
share the winning thumbnail is the one
that gets viewers to both click and stay
to watch now that doesn't make any sense
at all people click on a video because
the thumbnail or the title made them
curious but what makes them stay to
watch the video is about the
presentation the flow of the video the
storytelling the engagement so what I'm
seeing here is even if there's a channel
out there who is in the same niche as I
am who creates similar thumbnails as me
who has the same type of information in
our videos we may get the same
click-through rate but that does not
mean we will get the same watch through
ation I'll show you imagine if I'm
sharing a tip to making more money on
YouTube so yeah guys today I made $100
on YouTube and what I did is to make
sure I create my video then I upload it
and then I share it on other social
media platforms and when my video gains
Traction in other social media platforms
they ended up clicking on my chat link
and watching my video on YouTube then
getting me more views
so I made $100 like that extremely dull
not much hand gestures no soul no
expression whatsoever now compare that
to this so today I made $100 on YouTube
and I cannot believe what I did actually
works what I did is to make sure I
create my video then upload that video
to YouTube and then I share it to other
social media platforms these social
media platforms will then get views on
their own attracts new audiences for me
and get those who are interested to
watch my YouTube video via the link I
shared and when my video gain Traction
in other social media platforms they
ended up clicking on my shared link and
watching my video on YouTube than
getting me more views on YouTube and
increasing my revenue for that video it
is as simple as that I didn't believe
that would work and I hope you can try
this out and see the traction as well
same message but the watch time varies
depending on the delivery right so I
honestly don't think YouTube got this
right at all the thumbnail has got
nothing to do with watch duration even
if the Creator nailed the thumbnail and
delivered the promised value but YouTube
did not end their explanation there they
have more to say they state that
additionally YouTube's search and
Discovery systems consider both when
ranking videos choosing thumbnails based
on watchtime share will benefit both you
and your viewers the way I read that
sentence it is just covering up
something they don't understand I feel
like the team who came up with this
feature needs to be content creator
themselves hence I don't buy what they
say it isn't benefiting me nor my
viewers because the metrics are
unrelated having said that I do find one
very interesting fact about this and
that is how YouTube does rank our video
this is something that I have had
suspicion on for years that's I think
that some videos are ranked much like
the Google algorithm and I strongly
believe that big channels that have
created one or more successful videos
their channels are ranked secretly so
say if I were to create a video of the
same topic as Nate black then my video
will not get as much impression as his
because his channel is ranked higher
than mine and therefore the videos he
creates will have the advantage of
getting ranked higher of course this is
just my specul but that particular
sentence from YouTube about ranking
videos it kind of tells me that it does
exist and is hidden from us anyways
that's what I found from YouTube
thumbnail testing and set to say I
didn't even notice it until recently but
not only that another big problem with
YouTube thumbnail testing is it doesn't
benefit small channels in order to have
thumbnail testing perform efficiently
you actually need a big sample siiz data
for example most of my thumbnail testing
takes at least two weeks to conclude and
often it ends with no conclusion so
that's a big waste of time but that's
because my sample size is small meaning
my videos aren't getting enough
impressions for YouTube to do
substantial thumbnail testing and who is
responsible for giving me more
impressions YouTube of course and here's
the thing if you're a small Channel and
you don't get as many Impressions
because your channel isn't ranked yet
then this whole thumbnail testing is
useless to you your sample size is just
too small to make any proper conclusion
which is very counterintuitive because I
thought thumbnail testing is for small
channels but turns out it is really for
the big players again I know shocking
right I don't know why but when I first
heard about thumbnail testing from
YouTube from the big content creators
out there who got invited to test this
feature like Mr Beast I was given the
impression that they are to measure
click rate and when I look back at what
Mr Beast posted when he first started
testing this I think even he got
confused about what YouTube's AB
thumbnail testing is all about the post
he made on Twitter or X or whatever he
is clearly talking about click through
rate and not watch share look that being
said I do still use it these days for
every video I launch I create three
thumbnail versions and I upload them for
testing this three thumbnails contain
very subtle changes like font color or
face expression but that's about it then
I will test it out and 99% of the time I
will get almost equal percentages which
means it doesn't matter which thumbnail
I pick they all give similar results
there is one time where YouTube gave me
a winning thumbnail after testing just
one day but that happen only once and I
actually don't know why just take a look
yourself they're all the same nail I
just tilt one and turn another one
upside down I guess it does increase
click through rate but again I don't see
how that will affect watch time unless
of course the formula is as simple as
taking the clickthrough rate plus the
watch time then giving a final score to
that thumbnail to determine the winner
so that means assuming watch time stays
constants because the video style is
only capable of doing so much then the
deciding factor of a winning thumbnail
automatically becomes the click through
rate so I guess in a way it is still the
click through rate and we're back to the
correct way of AB testing look you can
see how confusing this is and YouTube
needs to be a little more transparent
and be straightforward maybe for once
stop working with big channels who have
the edits advantage and work with small
channels who really want to see if new
features introduced can indeed help
channels grow big channels are all
already established they don't need
these tools but small channels do so the
very least you can do YouTube is to
spend those billions of dollars you have
to implement a feature that is made
specifically for small channels big
channels can help sustain YouTube they
don't need as much help small channels
though if you help us grow our channel
to become the next big Channel then that
benefits YouTube in the long run because
the pool of big channels just gets
bigger it's a win-win situation if you
ask me