Interpolate infinite number of frames with small VRAM usage in ComfyUI AI, RIFE, FILM

[00:00] hello to this new tutorial today I want

[00:02] to show you how you can interpolate big

[00:04] piles of frames without running into

[00:07] memory issues I got question in the

[00:09] comments how to do this and I researched

[00:14] it a little bit and I realized that

[00:17] there is nothing like this workflow so

[00:20] far so I created something which is

[00:25] based on already generated images in a

[00:30] folder and this workflow runs through

[00:33] this folder step by step and generates

[00:36] the interpolated frames between every

[00:39] frame pair step by step without loading

[00:43] the whole folder so that your rum is not

[00:47] filled with all the frames and will not

[00:50] overrun so only one interpolation is

[00:54] done at once and this workflow goes

[00:57] through your folder of images Step by

[01:00] step this is based on a wild mix of

[01:04] custom comy UI notes which I will list

[01:08] in the description below it was quite a

[01:10] wild ride to find the notes which work

[01:13] well together but in the end everything

[01:15] worked out so in the future I might make

[01:18] a new version an updated version of this

[01:21] workflow with more simple notes or maybe

[01:26] a little bit easier to work with but for

[01:28] now this is working it should help a lot

[01:31] of you to work through big piles of

[01:34] images and interpolate the whole bunch

[01:37] of them through without having memory

[01:40] issues let's start from scratch I want

[01:43] to start with loading our images I use

[01:46] the load image badge note from the V

[01:50] note

[01:51] suit so add note or suit

[01:56] IO load image badge

[02:03] I Define the path of all the frames I

[02:08] want to interpolate

[02:10] between and we use the single image mode

[02:15] here we need to duplicate this since we

[02:19] need two frames to interpolate between

[02:24] and we need to pack our

[02:27] frames to a batch

[02:34] so we pipe both outputs into our batch

[02:39] images node

[02:41] to have a batch which we then can fill

[02:45] into our interpolation

[02:48] node I I use the

[02:53] RVE interpolation note

[03:01] I keep the settings uh like this for

[03:10] now and then I want to save the

[03:14] results for this I choose the V notepad

[03:23] again and choose the image save note of

[03:27] the V pack this note gives us a

[03:30] possibility to define a folder where we

[03:34] put our interpolated frames into we need

[03:39] to Define which frame the load image

[03:43] batch node will load in this case uh the

[03:47] first note is uh has the index zero so

[03:52] that means it's loading the

[03:55] first

[03:57] frame and this uh note needs the index

[04:01] of one this means it's loading the

[04:04] second

[04:06] frame in

[04:08] programming um you point up from zero

[04:13] zero means one so in programming and

[04:20] informatics uh you're starting at zero

[04:23] just just at a side

[04:27] note and this should give us results

[04:33] already I will have a test so as a base

[04:36] I take some simple animate diff uh

[04:40] animation I did before and we see out of

[04:43] those two frames there have been created

[04:46] three frames if we push this up to 10 we

[04:50] get 11 frames so this is only the first

[04:54] step this is the basic interpolation

[04:56] this only uses the first and the second

[04:59] frame for now to go through our whole

[05:02] pile of images we need to do some

[05:05] Automation and special things now let's

[05:08] first start by making an

[05:11] automation to crank up our index we need

[05:15] to count

[05:17] up this value and this value one by one

[05:24] so that we later can run through

[05:28] everything sing frame frame for this I

[05:30] use the incremental note out of the

[05:33] musarat

[05:40] noes this gives us the possibility every

[05:43] time we run the patch the number in this

[05:47] node will be increased and then we can

[05:49] fill in this number to our load image

[05:53] batch nodes we have to set this to

[05:56] increment and now since we need

[06:01] this note have the number one higher

[06:04] than in this note so this the zero this

[06:08] one when this is one I need this to be

[06:11] two so we have

[06:13] to add one to the output of this note to

[06:18] fill the value into this note we could

[06:22] take two of those

[06:23] notes uh with the difference of

[06:26] one to fill uh in the second

[06:31] note but I find it easier like this way

[06:35] so you only have to adjust one number in

[06:39] the start of the workflow so this is

[06:43] easier for the addition of one to this

[06:47] number I use the number operation Noe

[06:51] out of the V not

[06:58] suit since this is taking numbers I have

[07:02] to convert this

[07:05] integer to a

[07:11] number since this is not an input we

[07:14] have to converted to an

[07:19] input so right click on this arrow and

[07:23] choose convert a to input now we can

[07:27] fill in our integer

[07:30] and it will be converted to a

[07:36] number so we're filling this number a

[07:39] into this number operation and we choose

[07:43] addition as an action and now we need

[07:47] another number which is added to number

[07:49] a

[07:51] so we use the into number again

[08:00] put it to one and fill it inside our

[08:04] number

[08:08] operation and now we need

[08:11] to convert the index settings of our

[08:17] notes to inputs also right click on this

[08:21] arrow and convert index to input and do

[08:26] this for the first note for the first

[08:28] batch note

[08:30] also convert index to input and now we

[08:35] can fill the integer value into our

[08:40] index and we can fill this integer to

[08:45] the first one so this seat is zero so in

[08:50] the first PA of this workflow we have a

[08:53] zero filled into the load image batch

[08:56] node and it will load the first image

[08:59] image out of the

[09:01] folder for the second batch it it will

[09:05] add

[09:06] one this one to our zero so we get one

[09:10] as an output and this gets number one as

[09:13] an index and will load the second

[09:16] image so sorry if this is a little bit

[09:20] complicated uh with the zero and the one

[09:25] and the shift of one number but uh it is

[09:28] like it is in mathematics and in

[09:30] informatics in programming like this and

[09:33] I cannot change it but you will get

[09:36] familiar with it if you're not used to

[09:38] it this control after generate will

[09:41] increment our number one step after each

[09:45] generation and this will step through

[09:48] our whole folder first time we use it

[09:51] there will be index zero index one the

[09:54] next time this will one so we will have

[09:57] index one and index two this means the

[10:01] second frame out of the folder and the

[10:04] third frame will be loaded and

[10:06] interpolated between then this will be

[10:08] two that means the search frame will be

[10:12] interpolated to the force frame and so

[10:16] on so to automate this we need to use

[10:20] the extra options in the Q prompt and we

[10:23] need to check the auto que option so

[10:26] after the after every gener Generation

[10:30] The Prompt will be started again and the

[10:33] number will be increased into one and it

[10:36] will go through all your images step by

[10:39] step and not all at once so they will

[10:42] not be loaded everything into your rum

[10:44] and this is much more RAM saving as if

[10:48] you have to load everything to your room

[10:51] but we're not finished yet we have one

[10:54] problem this RVE Noe will output uh our

[10:57] first frame and our second frame and all

[11:00] the frames in between every time we run

[11:03] this workflow that means that there will

[11:06] be some frames doubled so when we

[11:09] generate between first and second and

[11:12] then between second and third the second

[11:14] frame will be doubled and then when we

[11:17] generate between third and fourth uh the

[11:20] third frame will be doubled and the

[11:22] fourth frame and the fifth frame and the

[11:24] sixth frame and so forth this is because

[11:27] the ri interpolation node

[11:29] always all frames out so the first the

[11:33] second and the frames in between and we

[11:35] have a doubling of those end and first

[11:38] frames every run now it gets a little

[11:41] bit complicated we need some if and else

[11:45] Logic for this we need to split our

[11:48] image batch and separate the first frame

[11:52] and only output the last frames with all

[11:56] the starting frames because of the

[11:58] duplication we need to make some space

[12:01] here and run our batch of 11 images into

[12:05] Split Image batch node it's part of the

[12:08] VHS notes this can split our batch which

[12:12] is consisting of 11 image in this case

[12:15] and can output uh two batches and split

[12:18] it at a certain point so in this case I

[12:20] split it after index one this gives us

[12:23] the first image of the batch to image a

[12:26] and the 10 images after the go to image

[12:30] B so let's have a image preview here and

[12:34] an image preview here to see what we get

[12:39] out of this note so let's

[12:42] generate so what you see here is we get

[12:45] our first frame here separated and then

[12:48] the frames after that get outputed here

[12:51] so this is 10 frames this is one and

[12:54] this gives us all the 11 frames here

[12:57] since the first generation generat the

[12:59] images with the end frame we don't need

[13:03] this end frame which is the same as the

[13:05] starting frame for the Next Generation

[13:07] we don't need the starting frame we can

[13:10] split it off we only need those frames

[13:13] since this would be double we have to

[13:15] check if this worklow is running the

[13:17] first time if so we put out every 11

[13:20] frames if it is not we only put out

[13:24] those 10 frames without the starting

[13:27] frame to prevent the doubling we do this

[13:30] with the image input switch note from

[13:35] the v no

[13:44] suit this um image input switch is quite

[13:49] simple

[13:51] depending of the Boolean input true or

[13:55] false it's sending image a through to

[13:58] the output or image B so we pipe our

[14:04] splitted images into the image a port

[14:08] and the unsplit images to the image B

[14:13] port and now we need the Boolean

[14:16] information true or false for this we

[14:20] need

[14:21] to make a logic uh operation and compare

[14:25] our values so we use the comp Paran note

[14:29] out of the comi logic

[14:37] pack the bo part goes to the Boolean

[14:42] input of this

[14:43] note you already can connect the images

[14:47] to the image safe

[14:49] node and now we need some information

[14:53] here so at first I connect

[14:57] our n

[15:00] index into the compare node to the a

[15:05] input

[15:07] and for the

[15:10] bport I take another incremental

[15:17] Noe so connect this to the bport and set

[15:22] it to zero and set it to

[15:26] fixed so we are comparing our initial

[15:32] index which goes to a with the number we

[15:36] put to be so the number is zero and we

[15:41] look if a is bigger than b so if our

[15:47] index is zero we

[15:50] know it is not bigger than zero it is

[15:54] equal so this is false so it gives false

[15:58] to the image input

[16:00] switch and this means our image B will

[16:04] go through our

[16:06] output if we process our workflow

[16:10] another time the index will be

[16:13] one and therefore it is bigger than our

[16:17] zero so that means the compare node will

[16:20] output

[16:22] two and this gives the input switch the

[16:25] information to to bypass the image a

[16:30] output to the safe note that means the

[16:34] reduced the splitted batch without the

[16:37] starting image so that we don't have

[16:39] duplicate

[16:41] images and this will be done until our

[16:44] batch is done so what is really

[16:47] important for the incremental note at

[16:50] the beginning is that this max value is

[16:53] bigger than the amount of images you

[16:55] want to process so if you have let's say

[16:59] 200 images this should be bigger than

[17:02] 200

[17:04] so just put in a super big number and

[17:07] you will be fine let's have another

[17:12] look so let's put the multiplier a

[17:15] little bit

[17:18] smaller and let's put our seat again

[17:22] back to

[17:25] zero and

[17:30] we should be good to go let's give it a

[17:33] try let's disable the auto que again for

[17:37] a test this should output our first

[17:39] image the interpolated image in between

[17:42] and the last image now if we run it for

[17:46] the first time

[17:48] so that should the output should be

[17:50] three

[17:52] images so there we are we have our three

[17:56] images every other

[17:59] generation should output two images only

[18:03] depending on this

[18:06] logic and so that we have no duplicate

[18:10] images so let's do it another one it

[18:13] should only output two

[18:16] images there we go two images another

[18:19] one should only output two images

[18:24] again and there we go it's going through

[18:27] our batch

[18:30] step by step and calculating the images

[18:34] between every pair of images so we can

[18:39] hit the auto que feature again and let

[18:43] it run

[18:44] through so we're ending with this error

[18:47] message there is no object left to

[18:49] process this workflow can used for as

[18:52] many images as you like so yeah let's

[18:55] have a look at the results really smooth

[18:58] because of the

[19:01] interpolation and I cannot see any

[19:06] double images since this uh workflow is

[19:09] quite complicated I decided to put it on

[19:12] my GitHub page so that you can download

[19:15] it uh maybe a quick walk through how you

[19:20] would use it it's important to set your

[19:23] folder path

[19:27] here on both

[19:29] uh load image batch

[19:32] notes should both be the same path then

[19:36] you need to adjust the seat settings to

[19:44] zero and

[19:46] then set your output pass and then you

[19:50] should be good to go I put a link to the

[19:53] file on my GitHub in the description

[19:55] below as well as links to all the custom

[20:00] notes you need I hope this walk through

[20:04] this tutorial helps you and please

[20:06] subscribe and like the video I hope to

[20:09] see you soon goodbye