How to make an AI film?

How to make a good AI story.

Hello My Friend

Runway hosted their 48 hour AI film challenge recently. Looking at the winning entry inspired me to write this breakdown.

We tell stories through different formats - Comic books, novels, films... Same story will have a different structure depending on the format and medium it is told.

This will be a long breakdown, so let’s see what makes a good AI film.


We have to start here.

If you’ve worked with AI, you’ve probably noticed it has certain capabilities and limitations. Development goes quick and AI is leveling up daily and this is what it is capable of today:



It is communication-based.

Scripts, dialogue, character background, plot structure, strategy - Anything goes as long as we can put our thoughts and ideas into words and communicate them.


Without well-designed prompts, the results will be basic and generic.

AI is predicting, generating, guessing. It will sometimes make up things and be incoherent. We need to pay close attention and make creative decisions.



We can generate images of basically anything (depends on the dataset).

We can achieve style and character consistency.

As easy as writing a prompt to get started.


Generating highly specific images requires deeper dive into prompting, controlNET, IP adapters, models, etc.

Some concepts are incredibly difficult to generate.

High-quality output requires extra effort - there can be distortions, inconsistencies, blurriness, etc.



We can generate video from text, images, we can stylize existing videos.

We can control camera movements.

Simple motions are the most consistent.

We can create AI avatars of actors


Generative video is at the very early stages.

Complex movement, human anatomy are difficult to generate.

Many of the advanced AI video models are technical to use.

Some avatars immediately feel fake, with bad movement and speech



We can generate audio from text or transform existing voice into a different one.


Speech can sound flat, especially when generated from text.

Traditional Experience

Understanding the capabilities and limitations of technology is one thing, making a film is another. The best way to leverage new technology is to use existing frameworks with them.

Don’t go reinventing the whole wheel.

Write down the process of making a movie and think about the capabilities and limitations of AI.

  1. The Idea - How can AI help me ideate?

  2. The Script - Which parts can AI assist me with?

  3. The Storyboards - What kind of shots and style can AI generate?

  4. The Cast and Crew - How consistent can the characters and Avatars be?

  5. The Locations - How consistent can AI be?

  6. The Filming - What type of movement can AI create?

  7. The Post-Production - How can AI contribute to it?

In most cases, a combination of traditional and AI tools will work best. In many aspects, AI is too limiting. If we want to realize our full vision, we need to

Collaborate with AI

Follow the vision you have for your story AND also pay attention to what AI is giving you back.

Some ideas are not possible to create with AI. Some ideas are ONLY possible to create with AI.

AI films are very much still a playground. We’re seeing more people experiment with the format and create new structures.

Runway competition winner is a great example of this. They recognized the capabilities and limitations of AI and created a concept that would work excellently with them. They collaborated with AI while leveraging their traditional experience to add the finishing touches to their work with the dialogue boxes and effects.

Have you created your own AI film yet?

If not, take this post as an invitation. The tools are here, will you pick them up?

If you like my work, consider sharing it with others. It helps me inspire and empower more people.

As a thank you, you’ll get one of the downloadables as a gift. 💛