[0:00] Have you ever thought about how video streaming
is possible?
[0:03] Let's think about how big a typical 1080p
video is: 1920x1080 pixels, 24-bits each,
[0:10] 30 frames per second...
[0:11] That's almost 1.5 gigabits per second.
[0:14] How can you transmit that much data, over
the air, in real time?
[0:18] The answer is video compression.
[0:21] You might have heard of codecs.
[0:23] A codec is a piece of software that encodes
and decodes data.
[0:27] The encoding part compresses the data, making
it easier to store and transmit.
[0:32] And the decoding part reverses this process,
recreating the original data as closely as
[0:37] possible.
[0:38] Use of codecs is not limited to video, they
can be used to encode and decode many types
[0:43] of signals.
[0:44] But for now, let's focus on how video codecs
work.
[0:46] In the previous video, we talked about how
still images are compressed.
[0:50] In short, images are compressed by throwing
out the information that is less visible to
[0:54] the human eye and storing redundant data more
efficiently.
[0:58] We can easily extend image compression to
video compression by compressing a video frame
[1:03] by frame.
[1:04] This approach is called spatial or intra-frame
coding.
[1:07] Even doing that alone would significantly
reduce the file size, but we can actually
[1:12] do much more than that.
[1:14] In a typical video, many consecutive frames
tend to be nearly identical.
[1:18] We can make use of this temporal, inter-frame
redundancy to further compress a video.
[1:23] First, let's think of an extreme case where
nothing moves in a video.
[1:27] Instead of storing each one of these identical
frames we can simply tell our encoder to keep
[1:32] the first frame and repeat it N times.
[1:34] That would save us a lot of space.
[1:36] Now let's think of a more realistic case where
only some parts in the video don't change.
[1:41] This time we can do the same thing, but more
locally, by dividing the frames into blocks
[1:46] and repeating only the blocks that don't change.
[1:49] What if all blocks change between consecutive
frames but some change a lot and some change
[1:53] a little?
[1:54] Instead of checking whether a block has changed
or not we can search for a given block in
[1:58] the next frame within a neighborhood.
[2:01] This process is called block motion estimation.
[2:03] How does this help with compression?
[2:05] Well, instead of saving every frame, we can
save a reference frame and the motion vectors
[2:10] for the blocks.
[2:11] The motion vectors tell us how we should move
the blocks to closely match the next frames.
[2:15] This is called motion compensation.
[2:18] Although motion compensation can reduce the
difference between two consecutive frames
[2:21] greatly, it is usually not enough by itself
to fully recreate the next frame.
[2:26] So, in addition to the motion vectors, we
should also save the frame differences between
[2:30] the actual and motion compensated frames.
[2:33] These differences are known as residual frames.
[2:36] When it's time to play this video, the decoder
predicts the current frame by taking the previous
[2:40] reference frame, compensating for the motion
using the motion vectors, and adding the residual
[2:45] frame.
[2:46] You might ask couldn't we just save the original
frames instead of saving the residual frames?
[2:51] We could, but residual frames have much less
information than the full reference frames.
[2:56] Therefore they are highly compressible.
[2:58] Let's overview the entire process.
[3:01] Traditional video compression algorithms represent
a video as a sequence of reference frames
[3:05] followed by residual frames.
[3:06] There are two types of compression done here:
intra-frame coding and inter-frame.
[3:11] In this video, I focused mostly on the inter-frame
coding part which achieves a high compression
[3:16] efficiency by exploiting the similarities
between consecutive frames.
[3:21] Intra-frame coding, on the other hand, compresses
a frame by throwing out visually redundant
[3:25] information within the frame and storing the
rest more efficiently.
[3:30] Check out my previous video to learn more
about how this is done.
[3:33] The methods I covered here are the very basics
that are used by many codecs including the
[3:37] mainstream H.264 codec, which is also known
as MPEG-4 AVC.
[3:43] Modern video codecs, including H.264, H.265,
and VP9, use sophisticated methods to balance
[3:49] the level of compression and perceptual image
quality without introducing too much computational
[3:54] complexity.
[3:56] Although video compression algorithms we use
today are pretty mature, video compression
[4:00] is still an active area of research.
[4:03] Researchers have already been experimenting
with machine learning models that have the
[4:06] potential to perform better than today's block-based
hybrid encoding standards.
[4:11] It's not easy to beat today's encoding standards
since they had decades to mature and to be
[4:16] tuned in many possible ways.
[4:18] I still think that an end to end trainable
codec will eventually outperform the traditional
[4:22] compression methods, by optimizing perceptual
image quality while minimizing the file size.
[4:29] That's all for today!
[4:30] I hope you liked it.
[4:31] If you have any comments or questions, let
me know in the comments section below.
[4:34] Subscribe for more videos.
[4:35] As always, thanks for watching, stay tuned,
and see you next time.