Frame Interpolation Large Motion (FILM)

Published: February 5, 2023 | Last Modified: May 13, 2025

Tags: ai video film frame interpolation tensorflow animation python machine learning

Categories: Python Machine Learning

"The official Tensorflow 2 implementation of our high quality frame interpolation neural network. We present a unified single-network approach that doesn’t use additional pre-trained networks, like optical flow or depth, and yet achieve state-of-the-art results. We use a multi-scale feature extractor that shares the same convolution weights across the scales. Our model is trainable from frame triplets alone."

Prerequisites

Miniconda

https://docs.conda.io/en/latest/miniconda.html

Git

https://git-scm.com/download/win

Setup

Get Frame Interpolation source codes:

git clone https://github.com/google-research/frame-interpolation.git

cd into the cloned git repository, for example:

cd C:\Users\trima\Documents\GitHub\frame-interpolation

Create the Miniconda virtual environment:

conda create -n frame-interpolation pip python=3.9

Activate the Miniconda virtual environment:

conda activate frame-interpolation

Install requirements:

pip install -r requirements.txt

If FILM is not running on your GPU it’s because cudann is missing. Install it with:

conda install -c anaconda cudnn

Usage

From the Conda Shell, cd to the FILM directory:

Open File Explorer at this directory and copy the frames you want to interpolate to the “photos” folder.

start .

Place the images you would like to interpolate in the “photos” directory and run this command to begin interpolating them:

python -m eval.interpolator_cli --pattern "photos" --model_path pretrained_models\film_net\Style\saved_model --times_to_interpolate 1 --output_video

Rename frames

When rendering thousands of frames, the file names will unfortunately be formatted as frame_001.png, frame_002.png, frame_1000.png, frame_10000.png, etc. ffmpeg can’t make sense of this when the frames are being assembled into a video, so every frame needs to be renamed. Here’s a script that does this:

import os
import re

folder_path = r"C:\Users\Tristan\Documents\GitHub\frame-interpolation\photos\interpolated_frames"  # replace with the path to your folder

# Define a function to extract the number from the filename
def extract_number(filename):
    match = re.search(r'(\d+)', filename)
    return int(match.group(1)) if match else 0

# Get all files in the directory
all_files = [f for f in os.listdir(folder_path) if os.path.isfile(os.path.join(folder_path, f))]

# Filter only the .png files
image_files = [f for f in all_files if f.endswith('.png')]

# Sort using the extract_number function as the key
image_files.sort(key=extract_number)

# Rename each file
for idx, file_name in enumerate(image_files):
    new_name = "{:05d}.png".format(idx)
    old_file_path = os.path.join(folder_path, file_name)
    new_file_path = os.path.join(folder_path, new_name)
    
    os.rename(old_file_path, new_file_path)

print("Renaming complete!")

Assemble frames with ffmpeg

ffmpeg -framerate 60 -i %05d.png -c:v libx264 -pix_fmt yuv420p interpolated-60fps.mp4

Batch Processing

Enter this For loop in the Anaconda Shell to iterate through a folder of folders containing video frames and batch interpolate all of them.

FOR /D %i IN ("C:\Users\<user>\<some>\<directory>\*") DO python -m eval.interpolator_cli --pattern "%i" --model_path pretrained_models\film_net\Style\saved_model --times_to_interpolate 1 --output_video

Use this batch script to copy all “interpolated.mp4” files to the same directory as the script and rename them in sequential order.

@echo off
setlocal enabledelayedexpansion
set /a "count=0"

for /r "." %%a in ("*interpolated.mp4") do (
    set /a "count+=1"
    set "filename=00!count!.mp4"
    copy "%%a" "!filename:~-6!"
)

echo Finished copying !count! files.

Categories

Tags