using ai to interpolate 30fps to 60fps – flowframes tutorial

[00:01] hey everyone today i wanted to show you

[00:02] a pretty cool tool i found that uses ai

[00:05] to generate frames between your frames

[00:06] essentially making it so that you can

[00:08] feed in a 30 fps video and get a 60fps

[00:11] version out of it and arguably best of

[00:13] all it's free and it's called flow

[00:14] frames

[00:15] when it comes to interpolating frames

[00:17] which is just the act of having a frame

[00:18] a and b and then creating a new frame in

[00:20] between those two there's already a

[00:22] couple ways we can go about doing that

[00:23] without flow frames but in my opinion

[00:25] they're just not as good after effects

[00:26] is a built-in feature called optical

[00:28] flow that you might have used before and

[00:29] there's also one that you've likely

[00:30] heard especially if you're hitting up

[00:32] some sick call of duty montages back in

[00:34] the day and that one's called twixtor

[00:35] which is a paid plugin so real quick

[00:37] i'll set up a comparison between optical

[00:39] flow twixtor and flow frames where i

[00:41] take this clip from into the spider

[00:43] verse and slow it down to 50 speed in

[00:45] each case these tools are going to be

[00:46] interpolating the frames to make the

[00:48] slow motion look smoother to use optical

[00:50] flow to slow down our clip all we need

[00:52] to do is drag it into a comp make sure

[00:53] the composition is at least two times

[00:55] longer than the clip itself then right

[00:57] click the clip go to time time stretch

[00:59] and then stretch it to 200 percent now

[01:01] doing that alone is going to stretch the

[01:03] clip to twice as long but if i step

[01:04] through frame by frame you can see that

[01:06] it isn't interpolating frames we get a

[01:08] real frame then a paused frame that a

[01:09] real frame that a pause frame and so on

[01:11] to actually use optical flow to

[01:13] interpolate the frames creating a new

[01:14] frame in between the real ones we just

[01:16] need to hit this frame blend switch

[01:17] twice once changes it to standard frame

[01:20] blending which is just essentially

[01:21] putting the next frame over the current

[01:23] frame and setting the opacity to 50 but

[01:25] hitting the frame blend switch that

[01:26] second time switches it to the optical

[01:28] flow frame blending where after effects

[01:29] is actually trying to interpolate

[01:31] between your frames to create a new

[01:32] frame now with that all rendered out you

[01:34] can see that optical flow is mostly

[01:36] garbage but if things are already moving

[01:38] very slow it does a pretty good job like

[01:40] in these two shots

[01:41] and here's a comparison between no

[01:42] interpolating at all on top and using

[01:44] optical flow below so there definitely

[01:47] is a time and place where using it can

[01:48] work for you just most of the time it's

[01:50] not going to be a good option next let's

[01:52] take a look at twixtor again here we're

[01:54] going to import our clip into our comp

[01:55] making sure the comp size is at least

[01:57] two times longer than the footage but

[01:59] instead of time stretching the clip

[02:00] we're gonna have to nest our comp so

[02:02] dragging the composition into the new

[02:03] comp button we've got that sorted now on

[02:05] this new nested comp i'll apply the

[02:07] twixtor effect i'll enable gpu and i'll

[02:09] set the speed to 50

[02:10] now rendering this out you can see that

[02:12] twixtor definitely does a better job

[02:14] than optical flow when it comes to the

[02:15] faster moving parts if i put optical

[02:17] flow up top and two extra on the bottom

[02:19] you can see that it's pretty clear who

[02:20] the winner is now let me show you the

[02:22] main event the show stopper the one the

[02:25] okay enough i'm simply gonna drag the

[02:26] clip onto flow frames making sure it's

[02:28] set to two times interpolation and hit

[02:30] interpolate it does its thing and then

[02:32] we've got our video with double the

[02:33] frames now just to be clear what we get

[02:35] out of flow frames isn't slow motion

[02:37] this clip is the same speed as it was

[02:39] going in but it has double the frames

[02:40] because flow frames interpolated a frame

[02:42] in between every real frame making it

[02:44] look a lot smoother that was a lot of

[02:45] frames in the same sentence so when we

[02:47] import this into after effects and we

[02:48] make a new comp we need to go into the

[02:50] comp settings and make sure the frame

[02:51] rate is set to the same frame rate as

[02:53] the original clip in other words half of

[02:55] what our new interpolated clips frame

[02:57] rate is then again making sure the

[02:58] comp's length is at least twice as long

[03:00] as our clips we hit ok then we'll just

[03:03] right click our clip go to time time

[03:04] stretch and stretch it to 200 and we're

[03:07] done this is our flow frames

[03:08] interpolated slow motion i'll show you a

[03:10] comparison with twixtor in a second but

[03:12] flow frames is handling the

[03:13] interpolation of faster movement so much

[03:15] better here we've got twixtor up top and

[03:17] flow frames on the bottom if i step

[03:18] through frame by frame to show you how

[03:20] they both deal with faster movement here

[03:21] we can see we're on our real frame and

[03:23] then next we've got an interpolated

[03:24] frame with twixtor we've got some heavy

[03:26] ghosting and even some of the frame

[03:27] being pulled down up top here and none

[03:29] of that with the flow frames version if

[03:30] i go forward a little bit more oh boy

[03:32] lots of ghosting and the edges of the

[03:34] screen being warped inwards on both the

[03:35] top and bottom of the twixtor version so

[03:38] you get the point now i'm sure optical

[03:39] flow is decent if your footage is moving

[03:41] very slow already twixtor is much better

[03:43] at handling the faster moving

[03:45] interpolations but flow frames takes the

[03:47] cake in both fast and slow moving

[03:48] scenarios it's also just a really great

[03:50] way to quickly and easily take any media

[03:52] you have and double its frame rate so

[03:54] that it looks a lot smoother you can do

[03:56] it with anime video games movies renders

[03:58] it won't work flawlessly with every

[04:00] render but you could save yourself a lot

[04:01] of time if you just export your stuff at

[04:03] half the frame rate and then use flow

[04:05] frames to double it back up to where it

[04:06] needs to be because sometimes you're

[04:08] looking at like 10 minutes of frame on

[04:09] more complicated scenes so if you've got

[04:11] a render where things aren't moving

[04:12] super fast flow frames might be a good

[04:14] option for you in terms of saving a

[04:16] bunch of time as a bit of a trickier

[04:18] example here i've got a render for a new

[04:20] version of my intro that i'm working on

[04:21] i rendered it at 30fps because that's

[04:23] what all my videos are rendered at

[04:25] except for this one but if i re-render

[04:27] the intro at 15 fps here's what we've

[04:29] got there's a lot of fast movement at

[04:30] the beginning especially so i feel like

[04:32] flow frames won't absolutely nail it but

[04:34] if we chuck it in there and ask it to

[04:36] double the frames back up to 30 fps

[04:38] here's what it spits out now like i said

[04:40] at the beginning there seems to be a bit

[04:41] of artifacting but other than that it

[04:43] pretty much nails it a good workflow i

[04:44] can imagine doing is rendering my stuff

[04:46] out at 15 fps running it through flow

[04:48] frames to bring it up to 30 and then any

[04:50] area where i see flow frames having

[04:52] trouble i can just re-render that

[04:53] section in blender at 30fps and comp it

[04:56] back over top to cover up any issues

[04:58] that way i'm pretty much getting away

[04:59] with rendering at 15fps and only having

[05:01] to re-render at a full 30fps for the

[05:03] sections that flow frames can't

[05:05] interpolate smoothly which in the end

[05:06] will save me a ton of time now as far as

[05:08] where to get it i'll leave a link in the

[05:09] description it's absolutely free to

[05:11] download and you can choose to support

[05:12] the creator on patreon in order to get

[05:14] access to early updates after you

[05:16] download and install it you pretty much

[05:17] just need to drag a video file onto the

[05:19] program and it'll sort out most of the

[05:21] settings for you all you really need to

[05:22] worry about is how much to interpolate

[05:24] by the output mode and two settings

[05:25] called detuplication and scene changes

[05:28] setting the interpolation amount is

[05:29] pretty self-explanatory it shows you the

[05:31] fps of the video on the left and on the

[05:33] right it shows you what the output fps

[05:34] will be then you just choose the

[05:36] multiplier the output mode is also

[05:38] pretty obvious if you can work after

[05:39] effects i have confidence in your

[05:41] ability to sherlock homes what this

[05:42] could possibly mean and the last two

[05:44] settings of note are deduplication and

[05:46] scene changes deduplication is for if

[05:49] you have footage that's 60fps for

[05:50] example but if you step through it frame

[05:52] by frame you actually see that every

[05:54] other frame is a paused frame that would

[05:56] mean your footage is actually 30fps but

[05:58] it's just been rendered out at 60fps for

[06:00] no reason deduplication basically just

[06:02] tries to remove those paused frames

[06:04] before interpolating which will lead to

[06:05] a much smoother output lastly the

[06:07] setting for scene changes just exists to

[06:09] try and detect

[06:10] scene changes that way it won't try and

[06:13] interpolate a frame in between two

[06:14] scenes if i turn the setting off and

[06:16] then run the spider-verse clip through

[06:17] again you can see that between these two

[06:19] scenes we get a mushy morphed frame

[06:21] which you probably don't want so i'd

[06:23] leave that setting on and that's pretty

[06:24] much all i had to say about flow frames

[06:26] if you're into programs like this that

[06:28] you can toy around with and help with

[06:29] your editing i made a similar one on a

[06:31] program called ebsynth that i recommend

[06:33] you check out it's one of the most

[06:34] viewed videos on the channel so i guess

[06:36] people like it now that we're done with

[06:38] the tutorial let's get to the part

[06:39] you've all been waiting for the no

[06:41] contacts poll you know the part where i

[06:43] do whatever you guys tell me to do on a

[06:44] twitter poll even though you have no

[06:45] context for what i'm asking well this

[06:47] video's poll i asked miles toby andrew

[06:49] tum or

[06:51] miles tom toby andrew and 66.7 percent

[06:54] of you which is like six people in this

[06:56] case chose the second option so now get

[06:59] ready buckle down for a little

[07:00] spider-man edit that goes through them

[07:02] in the order from the pole with scenes

[07:04] interpolated to double the frame rate

[07:06] for more fluid fluidity

[07:17] ugh

[07:35] ugh

[07:57] pizza time wow

[07:59] amazing there are there are no words

[08:01] there are no words if you want to

[08:03] participate in the next videos no

[08:04] context poll make sure to follow me over

[08:06] on twitter and turn on notifications for

[08:08] my tweets because i don't tweet super

[08:10] often and the poll might get buried in

[08:11] the rest of your feed if you don't now

[08:13] let me take a second to thank this video

[08:15] sponsor skillshare if you weren't aware

[08:17] skillshare is an online learning

[08:18] community with thousands of classes for

[08:20] creators they've got a ton of topics you

[08:22] can explore but the ones that i'm

[08:23] personally drawn to were film and video

[08:25] of course productivity and creative

[08:27] writing a class that i watched recently

[08:28] that i think you might enjoy is intro to

[08:30] virtual filmmaking in unreal engine by

[08:32] cinematic captures he goes over how to

[08:34] set up a virtual camera rig in unreal

[08:36] finding assets lighting rendering and

[08:38] more virtual filmmaking has always been

[08:40] super interesting to me so i'm glad i

[08:42] got to learn a little bit more about it

[08:43] if any of that sounds good to you then

[08:45] give his course on skillshare a go

[08:46] skillshare is curated specifically for

[08:48] learning so you don't have to worry

[08:49] about ads and they're constantly

[08:51] launching a new premium classes for

[08:52] members so you can stay focused and

[08:54] follow where your creativity takes you

[08:56] if you're interested in joining the

[08:57] first thousand of my subscribers to

[08:58] click the link in the description we'll

[09:00] get a one month free trial of skillshare

[09:01] so you can start exploring your

[09:03] creativity today thanks skillshare