If you only have a LoRA for the base model you may actually want to skip the refiner or at least use it for fewer steps. The two-model setup that SDXL uses has the base model is good at generating original images from 100% noise, and the refiner is good at adding detail at 0. GTM ComfyUI workflows including SDXL and SD1. Pastebin. 5 and 2. Upto 70% speed. CivitAI:ComfyUI is having a surge in popularity right now because it supported SDXL weeks before webui. 5s/it as well. Searge SDXL v2. Here's the guide to running SDXL with ComfyUI. 0の概要 (1) sdxl 1. 0. The next step for Stable Diffusion has to be fixing prompt engineering and applying multimodality. I'd like to share Fooocus-MRE (MoonRide Edition), my variant of the original Fooocus (developed by lllyasviel), new UI for SDXL models. 5s/it, but the Refiner goes up to 30s/it. It also works with non. ComfyUI allows setting up the entire workflow in one go, saving a lot of configuration time compared to using base and. 先文成图，再图生图细化，总觉得不太对是吧，而有一个插件能直接把两个模型整合到一起，一次出图，那就是ComfyUI。 ComfyUI利用多重节点，能实现前半段在Base上跑，后半段在Refiner上跑，可以干净利落地一次产出高质量的图像。make-sdxl-refiner-basic_pipe [4a53fd] make-basic_pipe [2c8c61] make-sdxl-base-basic_pipe [556f76] ksample-dec [7dd004] sdxl-ksample [3c7e70] Nodes that have failed to load will show as red on the graph. A number of Official and Semi-Official “Workflows” for ComfyUI were released during the SDXL 0. . Create a Load Checkpoint node, in that node select the sd_xl_refiner_0. 24:47 Where is the ComfyUI support channel. The prompts aren't optimized or very sleek. There is no such thing as an SD 1. In any case, we could compare the picture obtained with the correct workflow and the refiner. Update README. latent to avoid this) Do the opposite and disable the nodes for the base model and enable the refiner model nodes. Two Samplers (base and refiner), and two Save Image Nodes (one for base and one for refiner). 5 + SDXL Refiner Workflow : StableDiffusion. 0. It'll load a basic SDXL workflow that includes a bunch of notes explaining things. And the refiner files here: stabilityai/stable. 9_comfyui_colab (1024x1024 model) please use with: refiner_v0. latent file from the ComfyUIoutputlatents folder to the inputs folder. 0 ComfyUI. ) These images are zoomed-in views that I created to examine the details of the upscaling process, showing how much detail. 5 models in ComfyUI but they're 512x768 and as such too small resolution for my uses. Part 3 - we added the refiner for the full SDXL process. SDXL Base 1. Searge-SDXL: EVOLVED v4. 5的对比优劣。. I also used a latent upscale stage with 1. Now that you have been lured into the trap by the synthography on the cover, welcome to my alchemy workshop! 现在，你已经被封面上的合成图所吸引. I think this is the best balanced I could find. SDXL_1 (right click and save as) workflow has the SDXL setup with refiner with best settings. By becoming a member, you'll instantly unlock access to 67 exclusive posts. Aug 2. 5 + SDXL Base - using SDXL as composition generation and SD 1. Run ComfyUI with colab iframe (use only in case the previous way with localtunnel doesn't work) You should see the ui appear in an iframe. Part 4 - we intend to add Controlnets, upscaling, LORAs, and other custom additions. This is the most well organised and easy to use ComfyUI Workflow I've come across so far showing difference between Preliminary, Base and Refiner setup. 9. 3 ; Always use the latest version of the workflow json. ComfyUI for Stable Diffusion Tutorial (Basics, SDXL & Refiner Workflows) Control+Alt+AI 818 subscribers Subscribe No views 1 minute ago This is a comprehensive tutorial on understanding the. You can add “pixel art” to the prompt if your outputs aren’t pixel art Reply reply irateas • This ^^ for Lora it does an amazing job. Final Version 3. 9 in ComfyUI, with both the base and refiner models together to achieve a magnificent quality of image generation. 2 noise value it changed quite a bit of face. SDXL Models 1. o base+refiner model) Usage. The latent output from step 1 is also fed into img2img using the same prompt, but now using. Fix. To simplify the workflow set up a base generation and refiner refinement using two Checkpoint Loaders. download the SDXL VAE encoder. Yes 5 seconds for models based on 1. 最後のところに画像が生成されていればOK。. Also, use caution with. I describe my idea in one of the post and Apprehensive_Sky892 showed me it's arleady working in ComfyUI. Natural langauge prompts. I noticed by using taskmanager that SDXL gets loaded into system RAM and hardly uses VRAM. 0 + LoRA + Refiner With Comfy UI + Google Colab fot FREEExciting news! Introducing Stable Diffusion XL 1. My current workflow involves creating a base picture with the 1. 0 with ComfyUI. My research organization received access to SDXL. Double click an empty space to search nodes and type sdxl, the clip nodes for the base and refiner should appear, use both accordingly. 2 comments. 0 almost makes it. 0 Refiner Automatic calculation of the steps required for both the Base and the Refiner models Quick selection of image width and height based on the SDXL training set XY Plot ControlNet with the XL OpenPose model (released by Thibaud Zamora)To use the Refiner, you must enable it in the “Functions” section and you must set the “refiner_start” parameter to a value between 0. 5B parameter base model and a 6. There are several options on how you can use SDXL model: How to install SDXL 1. You can disable this in Notebook settings sdxl-0. With usable demo interfaces for ComfyUI to use the models (see below)! After test, it is also useful on SDXL-1. 25:01 How to install and use ComfyUI on a free. Now that you have been lured into the trap by the synthography on the cover, welcome to my alchemy workshop! 现在，你已经被封面上的合成图所吸引. How to AI Animate. (introduced 11/10/23). They compare the results of Automatic1111 web UI and ComfyUI for SDXL, highlighting the benefits of the former. 0. 手順1：ComfyUIをインストールする. 0 workflow. Part 4 (this post) - We will install custom nodes and build out workflows with img2img, controlnets, and LoRAs. 0 links. For upscaling your images: some workflows don't include them, other workflows require them. com is the number one paste tool since 2002. Updating ControlNet. 5 clip encoder, sdxl uses a different model for encoding text. . Otherwise, I would say make sure everything is updated - if you have custom nodes, they may be out of sync with the base comfyui version. if it is even possible. Try DPM++ 2S a Karras, DPM++ SDE Karras, DPM++ 2M Karras, Euler a and DPM adaptive. Here Screenshot . Hypernetworks. 5 and 2. I'ma try to get a background fix workflow goin, this blurry shit is starting to bother me. Your results may vary depending on your workflow. The result is a hybrid SDXL+SD1. Install this, restart ComfyUI and click “manager” then “install missing custom nodes” restart again and it should work. That is not the ideal way to run it. It is not AnimateDiff but a different structure entirely, however Kosinkadink who makes the AnimateDiff ComfyUI nodes got it working and I worked with one of the creators to figure out the right settings to get it to give good outputs. It detects hands and improves what is already there. . 0 Alpha + SD XL Refiner 1. Prior to XL, I’ve already had some experience using tiled. (early and not finished) Here are some more advanced examples: “Hires Fix” aka 2 Pass Txt2Img. I also used the refiner model for all the tests even though some SDXL models don’t require a refiner. 5 and send latent to SDXL BaseIt has the SDXL base and refiner sampling nodes along with image upscaling. You can disable this in Notebook settingsYesterday, I came across a very interesting workflow that uses the SDXL base model, any SD 1. 0 Alpha + SD XL Refiner 1. Part 2 ( link )- we added SDXL-specific conditioning implementation + tested the impact of conditioning parameters on the generated images. Drag the image onto the ComfyUI workspace and you will see the SDXL Base + Refiner workflow. 9. For me, this was to both the base prompt and to the refiner prompt. Host and manage packages. if you find this helpful consider becoming a member on patreon, subscribe to my youtube for Ai applications guides. After an entire weekend reviewing the material, I think (I hope!) I got the implementation right: As the title says, I included ControlNet XL OpenPose and FaceDefiner models. Put into ComfyUImodelsvaeSDXL and ComfyUImodelsvaeSD15). Then inside the browser, click “Discover” to browse to the Pinokio script. json and add to ComfyUI/web folder. 1. Click “Manager” in comfyUI, then ‘Install missing custom nodes’. 🧨 DiffusersExamples. The denoise controls the amount of noise added to the image. go to img2img, choose batch, dropdown refiner, use the folder in 1 as input and the folder in 2 as output. 本机部署好 A1111webui以及comfyui共用同一份环境和模型，可以随意切换使用。. 0 BaseYes it’s normal, don’t use refiner with Lora. 9 and Stable Diffusion 1. Download the SD XL to SD 1. update ComyUI. 12:53 How to use SDXL LoRA models with Automatic1111 Web UI. All the list of Upscale model is. So I used a prompt to turn him into a K-pop star. Adjust the workflow - Add in the. All images are generated using both the SDXL Base model and the Refiner model, each automatically configured to perform a certain amount of diffusion steps according to the “Base/Refiner Step Ratio” formula defined in the dedicated widget. Most UI's req. Fully configurable. Fooocus, performance mode, cinematic style (default). 🧨 Diffusers Examples. 手順3：ComfyUIのワークフローを読み込む. stable-diffusion-xl-refiner-1. 0 - Stable Diffusion XL 1. 5. How to install ComfyUI. 0. Selector to change the split behavior of the negative prompt. Save the image and drop it into ComfyUI. NOTE: You will need to use linear (AnimateDiff-SDXL) beta_schedule. Updated ComfyUI Workflow: SDXL (Base+Refiner) + XY Plot + Control-LoRAs + ReVision + ControlNet XL OpenPose + Upscaler . You can't just pipe the latent from SD1. 5/SD2. 0_comfyui_colab (1024x1024 model) please use with: refiner_v1. Not a LORA, but you can download ComfyUI nodes for sharpness, blur, contrast, saturation, sharpness, etc. My advice, have a go and try it out with comfyUI, its unsupported but its likely to be the first UI that works with SDXL when it fully drops on the 18th. A detailed description can be found on the project repository site, here: Github Link. . i miss my fast 1. The refiner is although only good at refining noise from an original image still left in creation, and will give you a blurry result if you try. Explain the Ba. SDXL Refiner 1. But, as I ventured further and tried adding the SDXL refiner into the mix, things. To update to the latest version: Launch WSL2. 0 Download Upscaler We'll be using. Once wired up, you can enter your wildcard text. If you don't need LoRA support, separate seeds, CLIP controls, or hires fix - you can just grab basic v1. stable-diffusion-webui * old favorite, but development has almost halted, partial SDXL support, not recommended. If you have the SDXL 1. 23:06 How to see ComfyUI is processing the which part of the. The refiner is trained specifically to do the last 20% of the timesteps so the idea was to not waste time by. Also how to organize them when eventually end up filling the folders with SDXL LORAs since I cant see thumbnails or metadata. 0. ComfyUI is a powerful and modular GUI for Stable Diffusion, allowing users to create advanced workflows using a node/graph interface. Contribute to fabiomb/Comfy-Workflow-sdxl development by creating an account on GitHub. 🧨 DiffusersThe way to use refiner, again, I compared this way (from on of the similar workflows I found) and the img2img type - imo quality is very similar, your way is slightly faster but you can't save image without refiner (well of course you can but it'll be slower and more spagettified). These are what these ports map to in the template we're using: [Port 3000] AUTOMATIC1111's Stable Diffusion Web UI (for generating images) [Port 3010] Kohya SS (for training) [Port 3010] ComfyUI (optional, for generating images. Embeddings/Textual Inversion. Part 2 ( link )- we added SDXL-specific conditioning implementation + tested the impact of conditioning parameters on the generated images. The only important thing is that for optimal performance the resolution should be set to 1024x1024 or other resolutions with the same amount of pixels but a different aspect ratio. Hires isn't a refiner stage. I think the issue might be the CLIPTextenCode node, you’re using the normal 1. The refiner is although only good at refining noise from an original image still left in creation, and will give you a blurry result if you try to add. ago. How To Use Stable Diffusion XL 1. but ill add to that, currently only people with 32gb ram and a 12gb graphics card are going to make anything in a reasonable timeframe if they use the refiner. Images. The following images can be loaded in ComfyUI to get the full workflow. 0已更新！遥遥领先~快来看看更新内容及使用体验~，免费开源AI音乐：文本生成音乐，使用Riffusion实现音乐实时生成，【AI绘画】SDXL进阶篇：如何生成不同艺术风格的优质画面SDXL VAE: Optional as there is a VAE baked into the base and refiner model, but nice to have is separate in the workflow so it can be updated/changed without needing a new model. If. A good place to start if you have no idea how any of this works is the: with sdxl . Fooocus-MRE v2. 0 model files. Custom nodes extension for ComfyUI, including a workflow to use SDXL 1. SD1. A hub dedicated to development and upkeep of the Sytan SDXL workflow for ComfyUI he workflow is provided as a . There’s also an install models button. Despite relatively low 0. . Custom nodes extension for ComfyUI, including a workflow to use SDXL 1. you are probably using comfyui but in automatic1111 hires. tool guide. 0. . 5对比优劣ComfyUI installation. refiner_v1. A little about my step math: Total steps need to be divisible by 5. It will crash eventually - possibly RAM but doesn't take the VM with it - but as a comparison that one "works". The question is: How can this style be specified when using ComfyUI (e. I also have a 3070, the base model generation is always at about 1-1. Place LoRAs in the folder ComfyUI/models/loras. All the images in this repo contain metadata which means they can be loaded into ComfyUI with the Load button (or dragged onto the window) to get the full workflow that was used to create the image. It provides a super convenient UI and smart features like saving workflow metadata in the resulting PNG images. 手順4：必要な設定を行う. Use the Refiner as a checkpoint in IMG2IMG with low denoise (0. 9 comfyui (i would prefere to use a1111) i'm running a rtx 2060 6gb vram laptop and it takes about 6-8m for a 1080x1080 image with 20 base steps & 15 refiner steps edit: im using Olivio's first set up(no upscaler) edit: after the first run i get a 1080x1080 image (including the refining) in Prompt executed in 240. ComfyUI_00001_. SDXL Models 1. Restart ComfyUI. 9 refiner node. You can type in text tokens but it won’t work as well. make a folder in img2img. IThe sudden interest with ComfyUI due to SDXL release was perhaps too early in its evolution. I tried with two checkpoint combinations but got the same results : sd_xl_base_0. It would be neat to extend the SDXL dreambooth Lora script with an example of how to train the refiner. Outputs will not be saved. refiner_output_01030_. Warning: the workflow does not save image generated by the SDXL Base model. When all you need to use this is the files full of encoded text, it's easy to leak. So I created this small test. In summary, it's crucial to make valid comparisons when evaluating the SDXL with and without the refiner. Step 1: Update AUTOMATIC1111. It provides workflow for SDXL (base + refiner). This is great, now all we need is an equivalent for when one wants to switch to another model with no refiner. ComfyUI got attention recently because the developer works for StabilityAI and was able to be the first to get SDXL running. #ComfyUI is a node based powerful and modular Stable Diffusion GUI and backend. Works with bare ComfyUI (no custom nodes needed). You will need a powerful Nvidia GPU or Google Colab to generate pictures with ComfyUI. Hires. The only important thing is that for optimal performance the resolution should be set to 1024x1024 or other resolutions with the same amount of pixels but a different aspect. I upscaled it to a resolution of 10240x6144 px for us to examine the results. 5 512 on A1111. What a move forward for the industry. 3. This is often my go-to workflow whenever I want to generate images in Stable Diffusion using ComfyUI. Designed to handle SDXL, this ksampler node has been meticulously crafted to provide you with an enhanced level of control over image details like never before. 1. Two Samplers (base and refiner), and two Save Image Nodes (one for base and one for refiner). 0 with ComfyUI's Ultimate SD Upscale Custom Node in this illuminating tutorial. import json from urllib import request, parse import random # this is the ComfyUI api prompt format. . A second upscaler has been added. He puts out marvelous Comfyui stuff but with a paid Patreon and Youtube plan. 5 models) to do. The method used in CR Apply Multi-ControlNet is to chain the conditioning so that the output from the first Controlnet becomes the input to the second. After testing it for several days, I have decided to temporarily switch to ComfyUI for the following reasons:. 0 and Refiner 1. . The SDXL base model performs significantly better than the previous variants, and the model combined with the refinement module achieves the best overall performance. Part 4 (this post) - We will install custom nodes and build out workflows. 0 base and refiner models with AUTOMATIC1111's Stable Diffusion WebUI. Play around with different Samplers and different amount of base Steps (30, 60, 90, maybe even higher). SDXL has 2 text encoders on its base, and a specialty text encoder on its refiner. Using SDXL 1. To use the refiner model: Navigate to the image-to-image tab within AUTOMATIC1111 or. g. The Tutorial covers:1. SDXL VAE. It allows users to design and execute advanced stable diffusion pipelines with a flowchart-based interface. A CLIPTextEncodeSDXLRefiner and a CLIPTextEncode for the refiner_positive and refiner_negative prompts respectively. The difference is subtle, but noticeable. This uses more steps, has less coherence, and also skips several important factors in-between I recommend you do not use the same text encoders as 1. Simply choose the checkpoint node, and from the dropdown menu, select SDXL 1. Subscribe for FBB images @ These configs require installing ComfyUI. ComfyUI is also has faster startup, and is better at handling VRAM, so you can generate. 5 and 2. 0_0. ComfyUI, you mean that UI that is absolutely not comfy at all ? 😆 Just for the sake of word play, mind you, because I didn't get to try ComfyUI yet. 1 latent. Table of Content. I think this is the best balanced I. 0 base checkpoint; SDXL 1. 0; Width: 896; Height: 1152; CFG Scale: 7; Steps: 30; Sampler: DPM++ 2M Karras; Prompt: As above. 23:06 How to see ComfyUI is processing the which part of the workflow. ComfyUI with SDXL (Base+Refiner) + ControlNet XL OpenPose + FaceDefiner (2x) ComfyUI is hard. safetensors. Hi there. Fixed SDXL 0. I wanted to share my configuration for ComfyUI, since many of us are using our laptops most of the time. And to run the Refiner model (in blue): I copy the . The issue with the refiner is simply stabilities openclip model. Stable Diffusion XL 1. i've switched from a1111 to comfyui for sdxl for a 1024x1024 base + refiner takes around 2m. Upcoming features:Automatic1111’s support for SDXL and the Refiner model is quite rudimentary at present, and until now required that the models be manually switched to perform the second step of image generation. I normally send the same text conditioning to the refiner sampler, but it can also be beneficial to send a different, more quality-related prompt to the refiner stage. . 0. AI_Alt_Art_Neo_2. Step 3: Download the SDXL control models. json file to ComfyUI window. ComfyUI ControlNet aux: Plugin with preprocessors for ControlNet, so you can generate images directly from ComfyUI. RTX 3060 12GB VRAM, and 32GB system RAM here. This produces the image at bottom right. 0 was released, there has been a point release for both of these models. I need a workflow for using SDXL 0. 1. When I run them through 4x_NMKD-Siax_200k upscaler for example, the. 详解SDXL ComfyUI稳定工作流程：我在Stability使用的AI艺术内部工具接下来，我们需要加载我们的SDXL基础模型（改个颜色）。一旦我们的基础模型加载完毕，我们还需要加载一个refiner，但是我们会稍后处理这个问题，不用着急。此外，我们还需要对从SDXL输出的clip进行一些处理。The chart above evaluates user preference for SDXL (with and without refinement) over SDXL 0. StabilityAI have release Control-LoRA for SDXL which are low-rank parameter fine tuned ControlNet for SDXL which. 5 + SDXL Base+Refiner is for experiment only. The base model was trained on the full range of denoising strengths while the refiner was specialized on "high-quality, high resolution data" and denoising of <0. @bmc-synth You can use base and/or refiner to further process any kind of image, if you go through img2img (out of latent space) and proper denoising control. 9 Tutorial (better than. Second picture is base SDXL, then SDXL + Refiner 5 Steps, then 10 Steps and 20 Steps. My 2-stage (base + refiner) workflows for SDXL 1. For me, it has been tough, but I see the absolute power of the node-based generation (and efficiency). 0_controlnet_comfyui_colab (1024x1024 model) controlnet_v1. 0 for ComfyUI | finally ready and released | custom node extension and workflows for txt2img, img2img, and inpainting with SDXL 1. The fact that SDXL has NSFW is a big plus, i expect some amazing checkpoints out of this. Control-Lora: Official release of a ControlNet style models along with a few other. com is the number one paste tool since 2002. SDXL afaik have more inputs and people are not entirely sure about the best way to use them, also refiner model make things even more different, because it should be used mid generation and not after it, and a1111 was not built for such a use case. If it's the best way to install control net because when I tried manually doing it . I hope someone finds it useful. Here's where I toggle txt2img, img2img, inpainting, and "enhanced inpainting" where i blend latents together for the result: With Masquerades nodes (install using comfyui node manager), you can maskToregion, cropByregion (both the image and the large mask), inpaint the smaller image, pasteByMask into the smaller image, then pasteByRegion into. 9 Tutorial (better than Midjourney AI)Stability AI recently released SDXL 0. Includes LoRA. I have updated the workflow submitted last week, cleaning up a bit the layout and adding many functions I wanted to learn better. Also, you could use the standard image resize node (with lanczos or whatever it is called) and pipe that latent into sdxl then refiner. Img2Img works by loading an image like this example image, converting it to latent space with the VAE and then sampling on it with a denoise lower than 1. 0 is “built on an innovative new architecture composed of a 3. In the Comfyui SDXL workflow example, the refiner is an integral part of the generation process. SDXL base → SDXL refiner → HiResFix/Img2Img (using Juggernaut as the model, 0. Here are the configuration settings for the SDXL. Step 1: Download SDXL v1. Per the announcement, SDXL 1. Pastebin is a. AP Workflow 3. Place upscalers in the. I also automated the split of the diffusion steps between the Base and the. 5 models. I suspect most coming from A1111 are accustomed to switching models frequently, and many SDXL-based models are going to come out with no refiner. Stability. Eventually weubi will add this feature and many people will return to it because they don't want to micromanage every detail of the workflow. Note that for Invoke AI this step may not be required, as it’s supposed to do the whole process in a single image generation. google colab安装comfyUI和sdxl 0. SDXL VAE. safetensors”. 0を発表しました。 そこで、このモデルをGoogle Colabで利用する方法について紹介します。 ※2023/09/27追記 他のモデルの使用法をFooocusベースに変更しました。BreakDomainXL v05g、blue pencil-XL-v0. 11:02 The image generation speed of ComfyUI and comparison. The base doesn't - aesthetic score conditioning tends to break prompt following a bit (the laion aesthetic score values are not the most accurate, and alternative aesthetic scoring methods have limitations of their own), and so the base wasn't trained on it to enable it to follow prompts as accurately as possible. To download and install ComfyUI using Pinokio, simply go to and download the Pinokio browser. download the SDXL models. The SDXL base checkpoint can be used like any regular checkpoint in ComfyUI. 1 0 SDXL ComfyUI ULTIMATE Workflow Everything you need to generate amazing images! Packed full of useful features that you can enable and disable on the. Here's what I've found: When I pair the SDXL base with my LoRA on ComfyUI, things seem to click and work pretty well. Search for “post processing” and you will find these custom nodes, click on Install and when prompted, close the browser and restart ComfyUI. git clone Restart ComfyUI completely. 0. Step 2: Install or update ControlNet. Traditionally, working with SDXL required the use of two separate ksamplers—one for the base model and another for the refiner model. . Example workflow can be loaded downloading the image and drag-drop on comfyUI home page. If the refiner doesn't know the LoRA concept any changes it makes might just degrade the results.