4K/60 FPS Explained - Frame Interpolation & Upscaling Compared (Aiarty Vs. Comfy UI)

[00:00] Hello once again audience. Welcome back

[00:03] to another video. And today we're going

[00:06] to do something different. I'm going to

[00:07] show you guys how to upscale a video and

[00:10] interpolate a video to 60 frames per

[00:13] second. But first, please subscribe to

[00:16] the channel if you haven't already. I

[00:18] would greatly appreciate it. This video

[00:20] costs a lot of time and effort to make

[00:22] and I would really appreciate if you

[00:24] could just subscribe to the channel to

[00:26] get me to a th00and subscribers. I mean,

[00:29] we got a bunch of views already and a

[00:30] bunch of watch time. We just need the

[00:33] subscriber count. So, I'm going to

[00:36] explain real quick what upscaling means

[00:39] and what video interpolation also means.

[00:42] Upscaling is the process of increasing

[00:44] the resolution of an image or a video

[00:47] without reducing the quality or creating

[00:51] ugly stretches.

[00:53] And

[00:54] uh in this case you can also add more

[00:57] details by adding an AI model to improve

[01:00] the quality or add more details or more

[01:03] sharpness whatever you want.

[01:05] And video interpolation is the process

[01:07] of adding more frames per second to a

[01:11] video. So basically generating in

[01:12] between frames uh for your video so you

[01:16] can make them buttery smooth.

[01:19] And uh basically what I tried to do

[01:22] there was uh run a compui workflow to

[01:26] upscale a video

[01:29] uh which is getting split into separate

[01:31] images and that's the the video right

[01:34] there which I showed in the intro and

[01:36] those images are getting upskilled

[01:38] separately. However, due to my PC being

[01:41] kind of poopy and not that great uh it

[01:45] crashed. So, what I'm going to do is I'm

[01:47] going to manually

[01:50] um split the images. I'm pretty sure

[01:52] there is a compi workflow node somewhere

[01:56] which allows you to do parts of the

[01:59] batch.

[02:01] Uh so what I did first I was splitting

[02:04] the audio from the video because if you

[02:07] put images uh together you don't have

[02:10] the audio. So, you need to keep the

[02:11] audio from the original video uh

[02:14] somewhere safe, which is what I did

[02:16] first. And next up, I uh I use a little

[02:19] command using ffmpeg uh which is the

[02:22] program I use for video and audio

[02:26] uh processing, editing,

[02:29] uh just trying to combine videos and

[02:31] whatever if I get stuck uh grabbing

[02:34] images from videos, etc., etc. That's

[02:36] what FFmpeg It's open source library and

[02:40] what I'm doing now is I'm running a

[02:42] command to uh extract all the frames of

[02:46] the video and create PGs from them. And

[02:51] uh you can see it takes a little bit of

[02:53] time and it should be around 300 images

[02:56] because it's 300 frames obviously.

[02:59] Uh, don't worry. I'll put all the

[03:01] commands in the description because uh,

[03:04] you're not going to remember everything

[03:05] I type. And I guess typing it well,

[03:07] pausing the video is also kind of

[03:08] annoying. So, don't worry. I put

[03:09] everything into the description. Um, so

[03:12] now we have the little uh, images here.

[03:15] I'm just going to put them in a separate

[03:17] folder for safekeeping

[03:20] so I can easily use that folder in a

[03:22] comfy UI.

[03:24] And besides Scumview, I also have

[03:26] Lossless Scaling for frame

[03:28] interpolation, which is like a Steam

[03:31] program which you can use for screen uh

[03:35] applications or games to create 60 fps

[03:38] without actually having to do any video

[03:39] editing. Uh, and I'm also going to show

[03:42] ART video enhancer, which is a paid AI

[03:45] tool. I got a free trial. I'm going to

[03:46] show you guys how it works. Uh, I'm not

[03:49] sponsored, by the way. I have to say I'm

[03:51] not sponsored. Uh, I hope they sponsor

[03:54] me. So, if you're watching the ARD, yo,

[03:56] sponsor your boy, please.

[03:59] So, what I did here is, uh, basically I

[04:01] had a little conf workflow and I grabbed

[04:06] a batch of images and I upskilled them

[04:09] to, uh, 4K resolution uh, without the

[04:13] use of the model. I do have a model in

[04:15] this little preview, but uh actually

[04:20] using that model kind of tenfolds the

[04:23] time. This already took over an hour to

[04:25] do in total. So, I'm just using like

[04:27] little batches here to to upskill them.

[04:32] And uh you can see that it takes about a

[04:36] minute for like 50 I want to say.

[04:40] uh maybe a bit longer but the process of

[04:43] like generating the images and doing all

[04:45] that stuff setting up confiu is such a

[04:48] drag uh I'm using Pinocchio

[04:52] and I think it has like 18 gigs of

[04:54] storage usage just for nodes and other

[04:58] libraries and whatnot. It's huge and uh

[05:02] it really eats up my PC.

[05:05] Uh, I I had a previous version of this

[05:08] video and there was just static noise

[05:10] and just the viewing experience wasn't

[05:13] that great. Honestly, I'm sorry. It's

[05:16] just cuz um because of my PC was kind of

[05:19] dying. So, that's why I'm re-recording.

[05:22] If you saw the other video, that's why

[05:24] I'm re-recording

[05:26] for uh for better audio because I I

[05:29] checked it out. I'm like, dude, this is

[05:30] awful. I cannot just put this out. And

[05:33] also I added some some nice new tweaks

[05:36] during the results section. So I I'm

[05:39] really happy how that turned out and

[05:41] might be a bit easier to spot the

[05:43] differences now.

[05:45] So uh yeah, you can see I think this

[05:48] takes like 2 minutes for 70 uh and then

[05:51] it's sourced to disk. So let's say it

[05:54] takes about two minutes for 60 and it's

[05:57] uh 300 frames.

[05:59] That's about uh uh 10 minutes for

[06:03] upscaling.

[06:05] Uh which isn't too bad uh considering

[06:11] ARD also takes about 10 minutes.

[06:14] Uh but there is a quality difference

[06:16] because ARD is using a

[06:20] AI model to increase the details and I'm

[06:23] not doing that here because that would

[06:25] probably make the runtime close to one

[06:28] hour maybe two hours

[06:31] uh just for for the image upscaling and

[06:34] then I'd still have to do 4K and

[06:38] I combining it back into a video. So,

[06:42] and for the sake of the video, I'm not

[06:44] using a model, but you could use a model

[06:46] if you have more RAM. Uh, I have a GTX

[06:50] 1070

[06:52] and it's doing fine. It's got 8 gigs of

[06:55] VRAM and it's it's holding up fine. The

[06:57] problem is,

[06:59] uh, mostly the RAM. I'm running out of

[07:01] RAM. I have 16 gigs of RAM, which is

[07:03] fine like like

[07:05] uh like eight years ago when I got the

[07:07] PC, but nowadays it's not. And and Comy

[07:12] UI just eats everything up and

[07:16] uh like I'm not sponsored by by ARD, but

[07:19] ARD has better um system resource

[07:24] management basically. So I'm just

[07:26] showing off the um the image there. it.

[07:30] It's upscaled to 4K now. It looks maybe

[07:33] a little bit sharper. I don't know.

[07:34] We're going to find out at the results

[07:37] section later in the video. And what I'm

[07:40] doing now is I'm uh combining

[07:44] the

[07:46] uh frames back into a video. And I'm

[07:49] specifying using the minus r uh

[07:53] parameter. I'm I'm specifying the frame

[07:55] rate basically. And I'm also setting up

[07:58] a empty audio clip. Uh because when you

[08:03] want to merge like concatenate videos or

[08:05] if you want to add audio to an existing

[08:08] video, it's sometimes it kind of

[08:11] uh like doesn't want to because you

[08:14] don't have an audio track on the video.

[08:17] So it's like just just a GIF basically.

[08:20] It treats it like a GIF and a GIF can't

[08:22] have sound. So if you have an empty

[08:23] audio track uh by using the 128k thingy

[08:28] um you ensure that you are not going to

[08:31] run into issues later

[08:33] uh adding audio or combining videos

[08:35] which do have audio and videos with jon

[08:37] of audio. So I had like um a banner

[08:42] project before. I had like a banner and

[08:44] I tried to merge it uh with like an

[08:46] actual video but the banner didn't have

[08:48] any sound. It was like just one frame of

[08:50] banner. um and um that caused issues. So

[08:55] that's why I know you you find out stuff

[08:57] like that. Uh again, don't worry. I'm

[08:59] just putting everything into the

[09:01] comments.

[09:02] And you can see this takes quite a long

[09:04] time. Uh probably because each of the

[09:06] images is like five megs big, five

[09:08] megabytes, over 300 images. That's um do

[09:13] the math, 1.5 gigs of images.

[09:17] uh it's it's adding into a video and

[09:21] honestly the video turns out to be like

[09:24] 55 megabytes or something. It's not

[09:26] really that big. Uh and ARD basically

[09:30] has the same thing. You know it it takes

[09:31] a 2 gig raw video and it converts it

[09:34] into like 55 or 20 megabytes video 4K.

[09:38] So it does do some very good

[09:40] optimization there. uh which I think

[09:43] it's H 264 265 something.

[09:48] So

[09:50] uh now I'm going to be

[09:55] uh let me check.

[10:00] I

[10:03] grab the video and I grab the audio.

[10:06] Yeah. And I combine them back to one. So

[10:09] now we are finished with our 4K video.

[10:13] We we've upscaled it perfectly and we're

[10:16] going to combine it with the other. This

[10:18] is a very fast process cuz basically

[10:21] you're just adding the track to to it.

[10:25] And this is how it looks like now. And

[10:27] at the end of the video, I'll be making

[10:29] comparisons so you can compare for

[10:31] yourself. And let me know in the

[10:33] comments if you think it looks any

[10:34] different. Uh and if so, which version

[10:37] do you think looks better? I'm really

[10:39] excited to hear you guys opinion. So

[10:41] this again is the um

[10:44] load video note which is the first one

[10:47] on the left there. Uh you can load a

[10:49] video in and then you can kind of

[10:52] interpolate it using another node. Uh

[10:55] basically I'm multiplying it to two

[10:57] meaning it will get uh 60 frames instead

[11:00] of 30. And then uh like the first node

[11:03] is actually splitting the video into

[11:05] images which uh basically I was doing

[11:08] myself before as well. Uh and then the

[11:11] the middle node is taking those images

[11:14] and then adding in between frames. And

[11:17] then the last node is actually combining

[11:22] the images back into a video. So

[11:26] basically

[11:28] I could work with only getting the in

[11:31] between frames and then generating the

[11:33] file myself. But the last node is not

[11:35] the issue. The first node is the issue

[11:38] which again crashed my PC because I

[11:41] don't have enough RAM. So I had to

[11:45] create smaller bits of video. So I I cut

[11:50] all the 300 frames into little parts.

[11:55] and like 50 frames per video, which is

[11:58] like 1.8 seconds, I guess, on a 30 frame

[12:02] video, 30 frames per second video. And I

[12:05] cut them in six parts cuz it crashed.

[12:09] And uh this this worked. And it's

[12:13] basically the same command as before.

[12:14] We're just trying to to create another

[12:16] video with the um 30 frames per second

[12:21] parameter, the minus or 30. empty audio

[12:24] track just cuz. And uh uh yeah, we

[12:28] create six of them. And I have a very

[12:30] bad naming schemes, you know, B1, A1,

[12:33] C1. Uh I just want to like get this done

[12:36] and I remove all the files afterwards.

[12:38] So like when I'm doing stuff like this,

[12:40] I usually have very default commands in

[12:42] a in a notepad or something

[12:45] uh with like a b c uh and they are

[12:48] sequenced. So I know like oh I'm at C

[12:51] step which is the third step. So I'll

[12:53] try to do the same in the comments as

[12:55] well for you guys. So you can easily

[12:57] copy paste it in sequence for one

[12:59] project and then uh you have like a a

[13:02] nice finished end result.

[13:05] So uh this like 50 frames not too long

[13:08] takes uh almost 30 seconds. Yay.

[13:12] And now I put it in into the um load

[13:16] video again. Uh I do notice that you can

[13:19] kind of limit the frames. Uh maybe you

[13:22] can set a different starting frame. Uh

[13:24] and you can kind of batch it like that.

[13:29] That might work. But I'm also scared

[13:31] that it's because it's a larger video,

[13:33] it's trying to still load some stuff

[13:35] into memory, which is going to cause it

[13:38] to die again, and I don't want it to die

[13:40] again because just sucks so much.

[13:44] Um like I didn't get a blue screen. My

[13:47] entire PC just completely froze. But you

[13:49] can see that the first step was finished

[13:51] way faster now, which again, less RAM

[13:54] needed. So, it all fits into my little

[13:57] 16 gigs memory. Uh, if you have more

[14:00] RAM, I I guess you could do this a lot

[14:03] easier with bigger videos. Let's look at

[14:06] the the the usage there. It is already

[14:10] for 50 frames already almost capping my

[14:13] my memory. It's taking 30 gigs of my

[14:15] swap file on my hard disk. like you can

[14:18] set up a certain section of your hard

[14:20] disk for extra RAM, which is really

[14:23] slow, but you can see that's also taking

[14:25] like 30 gigs, 28 to 33. Um, and that's

[14:30] that's besides my 15 to 16 gigs actual

[14:33] RAM usage. So, it's almost 50 gigs of

[14:36] RAM usage for a 1 second video for

[14:40] interpolation. So, that's you need like

[14:43] 64 gigs at least to do this stuff.

[14:46] Uh, so I finished all the uh little

[14:48] parts there

[14:50] and I'm combining them in order. You can

[14:52] have a little text file with ffmpeg

[14:55] and it will combine those videos in

[14:58] order and it will just concatenate those

[15:00] videos uh without any transitions. You

[15:03] might be able to do a transition with

[15:05] some fancy stuff maybe. I don't know.

[15:08] You would have to look it up. But if you

[15:10] don't have a video editor, but you want

[15:12] to have like a transition between clips,

[15:15] uh you could maybe do something like

[15:17] that. And also

[15:19] um you can easily combine clips using

[15:22] ffmpeg. You don't need a fancy video

[15:24] editor. You don't need Adobe Premiere

[15:27] for just combining clips. Uh also Adobe

[15:31] Premiere sometimes just crashes and

[15:32] doesn't want to combine videos and and

[15:34] just creates corrupted stuff. Uh, Fmpe

[15:38] is really stable for video creation, but

[15:40] if you want to do animations and like

[15:44] fancy text and whatnot, you might need a

[15:46] video editor.

[15:48] So, I combined the 60 fps video now as

[15:51] well. There we go. It's now 4K 60 fps.

[15:55] There is no sound again because we do

[15:57] have to add the sound to the video

[16:01] again, which as we know from before is a

[16:04] very fast process. So,

[16:07] almost done with the 4K 60 fps uh AI

[16:10] version.

[16:12] I am also going to create a

[16:16] nonupskilled 60fps version. See how that

[16:19] runs and I'll show the comparison later.

[16:24] That was me. Sorry, I pressed the

[16:26] recording button. Uh that's not your

[16:28] Windows doing a little plink sound.

[16:29] That's just me. I'm sorry. I hate it

[16:31] when YouTubers make a sound like Discord

[16:33] notification or whatnot and they don't

[16:35] mention it. This was me, okay? So, I'm

[16:37] sorry. Uh, don't don't be scared. So,

[16:40] again, we're we're combining the audio

[16:42] with the um with the video. And like I

[16:47] said before, I was going to try to do a

[16:49] 60 fps version with out upscaling. And

[16:53] that one took maybe like 20 to 30

[16:58] minutes. Uh, but it didn't crash my RAM.

[17:01] It just took a very long time. Like I

[17:02] was still able to watch some YouTube and

[17:05] and open Discord. I could I could do

[17:06] stuff while it was running. So,

[17:10] it's really good. Uh, that's the wrong

[17:13] version. Here we go. This is the uh the

[17:16] sound version again.

[17:18] 60 fps with sound.

[17:21] It looks It looks decent. I guess I'll

[17:23] have to compare it with uh with the

[17:25] original one later.

[17:28] Uh, but that's basically it. It took us

[17:29] about 15 minutes and 70 minutes even.

[17:33] And now I'm going to show you ARD video

[17:36] enhancer. I'll put the link in the

[17:38] description. Again, I'm not sponsored.

[17:40] Uh, but it's very easy. It's very

[17:42] optimized and it checks your hardware

[17:44] and everything. You can upscale using a

[17:47] model. You have three models of choice.

[17:49] I'm not sure if you can get other models

[17:50] somehow. You can upscale to 4K. You can

[17:53] use step mode for higher quality. You

[17:56] can use turbo for faster processing. Uh

[18:01] but it is uh uh it is lower quality. So

[18:05] here you can see like I have a little

[18:07] slider. You can see the edge of the uh

[18:10] ridge basically gets a little sharper.

[18:12] Let's look at Maido for a sec. Um yeah,

[18:16] he does get sharper. Like clearly

[18:17] sharper. Let's zoom in a little bit.

[18:19] There you go.

[18:21] Uh so you can see he does get a little

[18:23] sharper. You do have to be careful with

[18:25] the strength though, especially when you

[18:27] have a very low res image. Uh sometimes

[18:30] AI is adding some weird details like

[18:33] with text. I've had instances where uh

[18:37] the C turns into a D or a O. And uh as

[18:42] you can see, you can also increase the

[18:43] frame rate very easily. Uh you can

[18:46] change the color. Uh you can use HDR uh

[18:50] audio D noiseis like whatever you want.

[18:53] and you just press the the uh export

[18:56] button. You can even do a batch export

[18:58] of multiple videos using the same

[19:00] settings. So, you can see it takes about

[19:02] 10 minutes for a 10-second video. Uh

[19:06] it's a lot less setup. It's very

[19:09] much faster. So, I hope you enjoyed this

[19:12] and I hope you enjoy the results in a

[19:14] little bit. Uh because yeah, I think

[19:18] they look pretty cool. So, please

[19:20] consider subscribing to the channel if

[19:21] you have not already. I would greatly

[19:23] appreciate it and I hope to see you all

[19:24] next time. Let me know in the comments

[19:26] which versions you like the most and if

[19:29] you notice any big differences or not.

[19:32] So, thank you for watching. I hope you

[19:35] found this useful or interesting at

[19:37] least. And see you tomorrow with Maple

[19:40] Story again. I'm sorry. Okay, bye-bye

[19:43] everyone.

[19:46] [music]

[19:56] >> [music]

[20:08] >> Don't

[20:36] >> [music]

[21:06] [music]

[21:43] >> And now a shout out to our members. On

[21:46] the extreme tier we have Dustin H. On

[21:49] the normal tier we have Daniel Chavez.

[21:53] Thank you all so much.

[21:55] See you all next