Basics of Comfy UI

Hello Friend

Welcome to this week’s issue of the AI-Enhanced Creator. Thank you for being a subscriber. 💛

I’ve got a few things for you today.

Let’s get to it.

Resources

AI development I am most excited about this week - IC Light workflow for videos. This immediately caught my eye among all the content I see daily. Experimental workflows always get my mind going.

Live event - AI Artists react ep. 3. A new live event coming up Jul 17th at 3pm CEST. Luka Tisler and I will go over some of the coolest creations made in the AI community recently and discuss the technology and workflows use to make them. You can join us on Youtube or LinkedIn.

Free PDF guide - I released a new guide AI-Enhanced Creator - First steps. It goes over the core concepts, mindset and software to help you adopt generative AI in a professional creative environment. You can download it for free here.

Live walkthrough - Basics of Comfy UI. Last week I did an event on the topic of basic of ComfyUI. You can watch it on Youtube or LinkedIn. It is also the topic of the lesson we will get into today.

This Weeks Lesson - ComfyUI Basics

ComfyUI is the place to be.

As you dive deeper into the world of generative AI, you will start to see a lot of the paths lead to ComfyUI.

All the newest models and developments can primarily be used inside ComfyUI.

If you’ve seen the relighting feature in Magnific or Krea release recently - it was available more than a month earlier inside Comfy.

If you want to stay ahead of everyone else, it is THE place to be.

So, how do we get started with it?

Last week, Luka Tisler and I did a live event going over the basics of ComfyUI:

• The prerequisites of running it locally

• Installation options

• Cloud options

• Navigating the interface

• Importing workflows

• Explaining the node system

• Building a sample workflow

You can watch the full event here (it’s about 1 hour long). We received plenty questions from the audience, I highly recommend watching it, as you might encounter the same questions.

Prerequisites

The main prerequisite necessary to run ANY Stable Diffusion interface is Python. More specifically, Python 3.10. Make sure you have installed this exact version before getting into a loop of troubleshooting and frustration.

One click installer

I highly recommend using a one click installer to install ComfyUI. If you are not looking to spend hours inside the terminal and installing everything manually.

There are a few options to choose from, I have personally used Stability Matrix and Pinokio.

It’s as easy as downloading their software and running it. It will also download all the prerequisites necessary to run ComfyUI.

Cloud options

If you don’t have a powerful computer with an Nvidia GPU, you can run ComfyUI through a cloud service.

Here are a few options:

• Runpod

• Run comfy

• Comfy ICU

• Think diffusion

They have an easy setup with premade workflows and you pay for render time with all of these.

Interface

It’s a blank canvas. Literally.

You will be greeted with this basic workflow loaded when you first open ComfyUI.

You can move around nodes as you like, add new ones and connect them into more complex workflows like this one:

Overwhelmed?

Yeah, it can be.

No worries though, as we can start very simple.

Basic workflow

Each node is one piece of technology, when combined creates the workflows and outputs we want.

Understanding these pieces is the difficult and overwhelming part. There are more than 200 ComfyUI nodes at this point. And everyone can create their own custom node, serving the purpose they want. No wonder more nodes are being added daily.

We can add nodes to the canvas by double clicking and searching for the specific node we want.

To start, let’s breakdown a basic workflow and understand what each node does:

• Load checkpoint - we start by loading our checkpoint model

• Clip Text Encode - This is our prompt. We want two of these - one positive one negative.

• Empty Latent Image - This is our canvas. We determine the aspect ratio of our final image.

• KSampler - This is it. The engine. The node that converts all the information into an image. In here we select the steps, the CFG, the scheduler.

• VAE Decode - this decodes the latent noise into pixels.

• Preview image - Our final image is shown here.

Connecting the Nodes

The connections are color coded and labeled Clip goes into Clip, model goes into model, VAE goes into VAE.

When you grab one of the connectors, it will highlight where you can connect it.

To sum up

ComfyUI can get very complicated. It doesn’t have to be though. We can start very simple with basic workflows and build from there.

It all comes down to understanding the underlying technology behind the nodes.

What does a ControlNET do? What does an IP adapter do? What does Segment Anything do?

The more you understand, the bigger workflows you can build.

Make your life easier - Use a one click installer, start with a basic workflow, experiment and explore.

Homework

To give you a bit more motivation and guidance, here is an exercise you can try out.

First - Access ComfyUI. Either locally or through a cloud.

Second - Load the basic workflow. Inspect it, understand the nodes and why they are connected together

Third - Explore. Add ONE new feature to this basic workflow. It can be ControlNET, IP Adapter, Lora, Segment Anything, Animate Diff, whichever technology you are familiar with and know what it does.

Really pay attention and try to understand what the individual nodes do and what kind of building blocks you can create with them.

Most of all, keep creating and always remember to have fun. ☀️

Whenever you are ready, here are 4 ways I can help you:

• FREE Beginner’s Guide: Take your first steps with a 6-lesson course designed to introduce core mindset, skills and software to start working with AI. It's the foundation on which you can build your expertise.

• We have a free comunity of creators just like you. A place to connect, collaborate and learn together. It’s always easier to grow with others.

• AI-Enhanced Creator: Power of a Production House: Building on top of the core concepts, we dive deep into workflows and techniques. With step by step walkthrough and process breakdown.

• Coaching and Workshops: For personal and hands-on learning experience. Focusing on your goals.