Best AI workflows for video & image

ComfyUI workflows

Author

Nejc Susec
May 28, 2024 • Estimated Reading Time: 6 minutes

Hello Friend

Have you thought about trying out ComfyUI?

Not sure where to start or what does it even do?

ComfyUI is the ultimate GenerativeAI interface (for images and animations).

Why? Because you can build a fully custom workflow, controlling absolutely every part that goes into it. Which means you also have to understand what each part does.

Aside from actually writing the code, this is the most technical interface you will encounter with Generative AI.

Is it complicated?

Yes.

Can you still try it out?

Yes.

ComfyUI is open source. Which means, everything around it is shared openly by the creators. Including the workflows.

People truly are incredible in how giving we can be.

What I wish to give you today, is a list of workflows that I have tried out and found useful for production, fun to play with or they opened a new perspective on what creating can be.

To use these workflows you will need ComfyUI and a fairly strong computer.

Here is a guide to help you install it.

Before you get into the complexities of it, take a look at what we can do with it.

Using reference images, we can create an animation that blends them together. Consistency is influenced by the settings and the models we use.

Made by Ipiv

Made by me using the same Workflow

Aleksej and the team at Vantage Films used a fantastic combination of rotoscoping the hand, adding the style with AnimateDiffusion and upscaling it afterwards. I highly recommend watching the full video.

made by Aleksej Lotkov

Similar to Ipov’s morph animation, this workflow morphs between reference images we can put in. This workflow focuses more on consistency between initial images and the generated output.

made by simonxxoo

Using 3D animation as a base and AnimateDiffusion to apply styles on top of it. Combining traditional skill with new AI technology. A fantastic blend.

3D animation

Stylized with Animate Diff

A one click workflow for product photography. Incredible. You put the original image in, AI segments into subject and background. You create the new background with a prompt, AI applies lighting appropriate for the scene to the subject. All the details of the original product are preserved.

Made by Luka Tisler

This workflow is so clean. You have one big slider that controls the sun inside images. Sounds pretty crazy, right?

You choose where the sun will be in your image and AI will light the scene accordingly.

Made by Luka Tisler

The world of AI is exciting and ever growing. The open source community is building incredible tools and workflows daily. Sharing them openly for everyone to use, experiment and improve.

This is but a glimpse into what is possible with AI. Going from experimental to use case specific.

Wherever you are in your use of AI - experimental or use case specific - It’s here. Unlocking new worlds to those who embrace it.

Keep creating and always remember to have fun. ☀️