Using WarpFusion to Restyle Video in Google Colab

Matthew Berman introduces WarpFusion by showing stylized footage and explains how to produce a similar effect. Although the video title uses “text-to-video,” the demonstrated workflow takes an existing video and restyles it using a text prompt and image-generation models.

He says WarpFusion can be run locally or through Google Colab and chooses Colab for the demonstration. In June 2023, he describes obtaining a notebook through creator Sxela’s Patreon for $10 and using a $10-per-month Colab Pro account; he says a local setup can avoid the Colab charge but requires more work. These are his historical descriptions, not a statement of current prices or requirements.

Berman downloads the notebook file from Sxela’s page, opens a new Colab notebook and uploads the .ipynb file. He points out the extensive code, change notes, local installation guide and separate step-by-step instructions provided with it.

In the notebook’s folder preparation section, he runs the cell that mounts Google Drive and authorizes access. Once Drive is connected, he uploads the source video through the file pane. Connecting a drive gives the notebook access to files under the granted permissions.

In the settings section, he chooses an output size of 1280 by 720 and warns that a larger size can exhaust available GPU memory. He copies the uploaded clip’s path into the video input setting so the notebook knows which footage to process.

He obtains model weights from Hugging Face, uploads them to Google Drive, copies the resulting path and inserts it into the notebook’s model setting. The video links to the Stable Diffusion 1.4 original weights; model availability, compatibility and license terms depend on the chosen weights.

The text-prompt section determines the desired appearance. Berman replaces the default with a detailed dragon-man prompt, then selects Runtime and Run all. He says processing may take hours and recommends leaving the tab open; runtime and session behavior depend on the environment.

Before publishing the tutorial, Berman adds an update after discovering he had used an older notebook version. He says newer versions include more features and a graphical settings interface. He also demonstrates finding alternative style models on Civitai and placing their files in Drive, subject to compatibility with the workflow.

The newer interface exposes prompts, negative prompts and step schedules, as well as ControlNet settings that can be enabled or disabled for different source videos. Berman encourages experimentation because those choices can substantially change the visual result.

A saved settings file can be placed in Drive and loaded into the notebook so the user does not need to enter every setting again. Berman also explains that the render can show intermediate previews before the entire video is complete.

If a preview is unsatisfactory, he suggests changing settings and running the relevant step again. Once the style is pleasing, the Create video step assembles the output. He then shows examples from the WarpFusion community and notes that settings can affect consistency across frames.

Berman encourages viewers to experiment and learn from others in the community. His walkthrough records the notebook and services as they existed in 2023; later versions, subscription terms and interfaces can differ.