Best ControlNET uses

How can we control our creative work.

Nejc Susec
January 16, 2024 • Estimated Reading Time: 4 minutes

Hello My Friend

ControlNET has been around for a while now (coming up to a year). It’s one of the major AI features when it comes to image (and subsequently video) generation. With a reason. It offers a level of control (it’s in the name after all) over our creative work.

I want to highlight three use cases today, that transform your creative work.

I teach how to use ControlNET (among other lessons) in my latest course, if you want to learn step by step. You can use the code “2024” for 50% off.

ControlNET

Use an image to drive our AI generations.

First one are compositions. By drawing our composition (by hand, finding reference) we can use AI to build on top of that.

As in this example using a spiral composition:

Here are a few more examples:

www.linkedin.com/posts/nejcsusec_midjourney-aiart-artificialinspiration-activity-7110080277464457216-TTyN?utm_source=share&utm_medium=member_desktop

Second are "Images inside Images.”

We can use text, logo, faces, paintings… as guidance and generate on top of them with AI.

An example with Mona Lisa:

Few more examples here:

www.linkedin.com/posts/nejcsusec_midjourney-aiart-stablediffusion-activity-7115516193305763840-aBIg?utm_source=share&utm_medium=member_desktop

ControlNET + IP Adapter

We can use ControlNET for the composition and IP adapter for the subject/prompt.

With this combination, we can put characters into specific scenes.

As seen in this example:

Result

Few more examples here:

www.linkedin.com/posts/nejcsusec_aiart-aiforgood-aistorytelling-activity-7133001522476453888-TqHL?utm_source=share&utm_medium=member_desktop

ControlNET + Animate Diffusion

Video is a sequence of images and as such, we can use ControlNET to keep the movement consistent

Here are two examples:

www.linkedin.com/posts/nejcsusec_ai-aiart-stablediffusion-activity-7122039110432673792-xDnp?utm_source=share&utm_medium=member_desktop

www.linkedin.com/posts/nejcsusec_aiart-aiforgood-aistorytelling-activity-7140611370466652160-YmUk?utm_source=share&utm_medium=member_desktop

We started with text prompts and now we’ve been adding more and more controls beyond that.

The future of AI is multimodal, we need to look beyond text-only prompts.

Whenever you’re ready, here are 4 ways I can help you:

FREE Crash Course on Generative AI: Fill in the survey and you’ll receive lessons directly to your mailbox.
Downloadable Digital Products: PDFs and video courses used by more than 2000 creatives.
AI-Enhanced Creator: Power of a Production House: Video course for creators and industry professionals teaching AI with autonomy.
1 on 1 Teaching Sessions: Let’s connect the dots between generative AI and your own work.

If you like my work, consider sharing it with others. It helps me inspire and empower more people.

As a thank you, you’ll get one of the downloadables as a gift. 💛