- AI-Enhanced Creator
- Posts
- How to make an AI film?
Hello My Friend
Runway hosted their 48 hour AI film challenge recently. Looking at the winning entry inspired me to write this breakdown.
We tell stories through different formats - Comic books, novels, films... Same story will have a different structure depending on the format and medium it is told.
This will be a long breakdown, so let’s see what makes a good AI film.
Technology
We have to start here.
If you’ve worked with AI, you’ve probably noticed it has certain capabilities and limitations. Development goes quick and AI is leveling up daily and this is what it is capable of today:
Text
Capabilities It is communication-based. Scripts, dialogue, character background, plot structure, strategy - Anything goes as long as we can put our thoughts and ideas into words and communicate them. | Limitations Without well-designed prompts, the results will be basic and generic. AI is predicting, generating, guessing. It will sometimes make up things and be incoherent. We need to pay close attention and make creative decisions. |
Images
Capabilities We can generate images of basically anything (depends on the dataset). We can achieve style and character consistency. As easy as writing a prompt to get started. | Limitations Generating highly specific images requires deeper dive into prompting, controlNET, IP adapters, models, etc. Some concepts are incredibly difficult to generate. High-quality output requires extra effort - there can be distortions, inconsistencies, blurriness, etc. |
Video
Capabilities We can generate video from text, images, we can stylize existing videos. We can control camera movements. Simple motions are the most consistent. We can create AI avatars of actors | Limitations Generative video is at the very early stages. Complex movement, human anatomy are difficult to generate. Many of the advanced AI video models are technical to use. Some avatars immediately feel fake, with bad movement and speech |
Audio
Capabilities We can generate audio from text or transform existing voice into a different one. | Limitations Speech can sound flat, especially when generated from text. |
Traditional Experience
Understanding the capabilities and limitations of technology is one thing, making a film is another. The best way to leverage new technology is to use existing frameworks with them.
Don’t go reinventing the whole wheel.
Write down the process of making a movie and think about the capabilities and limitations of AI.
The Idea - How can AI help me ideate?
The Script - Which parts can AI assist me with?
The Storyboards - What kind of shots and style can AI generate?
The Cast and Crew - How consistent can the characters and Avatars be?
The Locations - How consistent can AI be?
The Filming - What type of movement can AI create?
The Post-Production - How can AI contribute to it?
In most cases, a combination of traditional and AI tools will work best. In many aspects, AI is too limiting. If we want to realize our full vision, we need to
Collaborate with AI
Follow the vision you have for your story AND also pay attention to what AI is giving you back.
Some ideas are not possible to create with AI. Some ideas are ONLY possible to create with AI.
AI films are very much still a playground. We’re seeing more people experiment with the format and create new structures.
Runway competition winner is a great example of this. They recognized the capabilities and limitations of AI and created a concept that would work excellently with them. They collaborated with AI while leveraging their traditional experience to add the finishing touches to their work with the dialogue boxes and effects.
Have you created your own AI film yet?
If not, take this post as an invitation. The tools are here, will you pick them up?
FREE Crash Course on Generative AI: 5 daily lessons delivered directly to your Inbox.
Join our Group: Closed group for professionals to connect, collaborate and share insights.
AI-Enhanced Creator: Power of a Production House: Naturally integrate Language models and Image generators into your work.
Coaching and Workshops: Looking for a hands-on transformation? Book a call and we’ll get to know each other. 🤗
If you like my work, consider sharing it with others. It helps me inspire and empower more people.
As a thank you, you’ll get one of the downloadables as a gift. 💛