Flux Support #1176
Replies: 35 comments 32 replies
-
|
I assumed that I need to download the node Anyline of TheMistoAI
And x-flux-comfyui of the Xlabs-AI
|
Beta Was this translation helpful? Give feedback.
-
|
no, you don't need either of those, just the nodes in https://github.com/Acly/krita-ai-diffusion/wiki/ComfyUI-Setup extra nodes are only required for GGUF (I should add that to the instructions) |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for that, i really appreciate that much work |
Beta Was this translation helpful? Give feedback.
-
|
Is it possible to use the --fast flag from Comfy in the plugin too? I have a 40x series card. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for this. I have two Flux questions:
Thank you |
Beta Was this translation helpful? Give feedback.
-
Not yet in the release. Added here: 13e613f |
Beta Was this translation helpful? Give feedback.
-
|
run into generation error, seems to be missing clip. I have all my clips as I am using OK in Comfyui but is it mixing teh T5 up?
followed by a lot of block and shape size mismatch log pre error:
I have in the clip folder: clip_g.safetensors Do I need to rename t5 base or t5xxl_fp8_e4m3fn.safetensors to t5.safetensors |
Beta Was this translation helpful? Give feedback.
-
|
It picks the first thing with "t5" in the name. Try deleting/moving/renaming t5_base.safetensors so it will use one of the other two. (What is t5_base used for?) |
Beta Was this translation helpful? Give feedback.
-
|
With that name change (either file or in resources) |
Beta Was this translation helpful? Give feedback.
-
|
I swapped the name in resources.py as test and now it is generating:
|
Beta Was this translation helpful? Give feedback.
-
There are models with custom trained CLIP L TEs. One example: Accompanying custom TE found here: |
Beta Was this translation helpful? Give feedback.
-
|
Flux controlnet inpainting(just find this don't know, if it helps make flux model more usable): |
Beta Was this translation helpful? Give feedback.
-
|
Implemented in 1ca5ed9 Results are better than without the Inpaint CN, but not great. SDXL still works better in general. But it's an alpha model, there is hope an improved version won't be too far off. |
Beta Was this translation helpful? Give feedback.
-
|
Added instructions on how to install the inpaint model in the main post. From version 1.25.0, it will be used when there's a selection and strength is at 100%. |
Beta Was this translation helpful? Give feedback.
-
|
I try to add controlnet to flux but seems it has not yet been installed in Interstice cloud. Hope it can be added soon. |
Beta Was this translation helpful? Give feedback.
-
|
I'm trying and something is wrong at the moment |
Beta Was this translation helpful? Give feedback.
-
|
how did you load gguf? |
Beta Was this translation helpful? Give feedback.
-
|
ok, ok work for me |
Beta Was this translation helpful? Give feedback.
-
|
Hello, I have tested flux-Canny-ControlNet-v3. Safetensors and MistoLine (mistoline-flux. dev-v1), and both of them have poor image quality, which cannot compare to XL's ControlNet. Do you have any recommended Flux ControlNet for testing? |
Beta Was this translation helpful? Give feedback.
-
|
Use low strength to avoid poor image quality. But they don't follow very closely (even with high strength). It took 6+ months for great ControlNet to come out for XL. I think we just have to wait. |
Beta Was this translation helpful? Give feedback.
-
|
Hello Acly @Acly ,
|
Beta Was this translation helpful? Give feedback.
-
|
Hi syl1130, how did you make the Union Pro work in Krita ai diffusion? |
Beta Was this translation helpful? Give feedback.
-
|
Hello Acly @Acly , I have added translations for the text added in version 1.24.0. |
Beta Was this translation helpful? Give feedback.
-
|
I have install MistoLine ,but i could not find in krita,but I can find canndy or depth |
Beta Was this translation helpful? Give feedback.
-
|
Got an error which is strange that flux model recognized as SD 1.5 |
Beta Was this translation helpful? Give feedback.
-
|
I kinda feel like I have to ask. Is there any specific reasoning why there aren't some opt in for experimental features list where you can straight up install the different Flux models from. The main reason why I have that question is because there are a few common things that you straight up need the Flux models to fix even if you are using an SD model for a specific style like hands and feet's if you don't want to sit down and redraw them by hand. The two main benefits that I can see from it is that you guys will be able to know that all the users have at least the provided base configuration and the second is that you can attach FAQ plus link to this thread. Also, if I'm not completely incorrect you could probably gray out the FP16 models for anyone that rock 4GB of VRAM. Also, stuff like this would probably be unnecessary to ask if you just got the correct configuration |
Beta Was this translation helpful? Give feedback.
-
|
I'd recommend switching to using this control net union: https://huggingface.co/Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro |
Beta Was this translation helpful? Give feedback.
-
|
I choose option 1 and just dropped Flux in models/checkpoints |
Beta Was this translation helpful? Give feedback.
-
|
people seem to like this pixelwave flux finetune https://www.reddit.com/r/StableDiffusion/comments/1gcbn4t/comment/ltswixq/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button Are there any other fine tunes that you have been looking at? |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Please can we get this :) |
Beta Was this translation helpful? Give feedback.
-
|
There's Flux1 dev NF4 model for low VRAM (6-12Gb) devices. |
Beta Was this translation helpful? Give feedback.
-
|
Did you manage to launch it? |
Beta Was this translation helpful? Give feedback.
-
|
In Krita? Not yet. |
Beta Was this translation helpful? Give feedback.
-
|
Added instructions for Flux Tools: Redux, Depth & Canny Lora, and (very rudimentary) Fill support You need v1.29.0 to use them. |
Beta Was this translation helpful? Give feedback.
-
|
Why am i getting this error? the models are all in the correct folder! |
Beta Was this translation helpful? Give feedback.
-
|
Not listed. 
|
Beta Was this translation helpful? Give feedback.
-
the model its there, but the log say it is not found:
|
Beta Was this translation helpful? Give feedback.
-
|
It might be because style_models is missing from extra_model_paths.yaml Try either of these:
(This applies if you're using the managed server, for custom Comfy install I have no idea) |
Beta Was this translation helpful? Give feedback.
-
Thanks God it worked! :))) Thanks a lot for yout reply and time! For anyone insterested, the solution provided by Acly: edit ComfyUI/extra_model_paths.yaml and add style_models: style_models to the krita-managed section
|
Beta Was this translation helpful? Give feedback.
-
This works for me. |
Beta Was this translation helpful? Give feedback.
-
|
flux canny tool on Krita is not generating proper images. does anyone have a solution for this? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Acly, the Redux tool has quite a strong reference even in the lowest strength setting, which ignores the prompt. Have you considered this approach that bypasses this? https://github.com/kaibioinfo/ComfyUI_AdvancedRefluxControl It also seems to solve the non-square image issue of CLIP that is present in the Redux tool. |
Beta Was this translation helpful? Give feedback.
-
|
Yea that's already used (lowest setting uses downscale of 4 I think, it's quite weak in my experience and definitely follows prompt) |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I did some additional testing, and it indeed follows the prompt at the lower settings. I think the main issue was that I used shorter prompts—it seems to perform better with longer ones. Additionally, I used it with Shuttle 3 diffusion, which, while it works, doesn’t follow the prompt as closely as Flux Dev does. Anyways, thanks again for the clarification and for all your amazing work! |
Beta Was this translation helpful? Give feedback.
-
|
Edit : opened #1457 I only get noise in outpainting
This is a sub 1024 image, mostly defaults
Tried 4 different quants of flux fill Alimama controlnet has been renamed as .safetensor.disabled I'm at a loss. Tried Fill / Expand / Add Object, always get noise |
Beta Was this translation helpful? Give feedback.
-
|
Use one or the other method for inpainting (not both): |
Beta Was this translation helpful? Give feedback.
-
|
Would it be possible to update the RunPod template to the latest version? Would be much appreciated, thank you for the template. Another Runpod request would be to add start commands to automatically download the full versions of Flux (FP16 unet, FP16 T5, canny, depth, fill, redux). If Runpod is tricky to maintain, I'd be happy to sign up for other services too (yours too) if they could offer this. Many thanks |
Beta Was this translation helpful? Give feedback.
-
|
When using Krita, it generates a black image, but there is no problem when using ComfyUI alone, I suspect it is related to the following code, please help got prompt |
Beta Was this translation helpful? Give feedback.
-
|
I tried to get the Flux Fill to work but I'm getting : |
Beta Was this translation helpful? Give feedback.
-
|
I got the same error |
Beta Was this translation helpful? Give feedback.
and others
-
Flux: Current State
Plugin version: 1.29.0
Experimental Support
Flux ecosystem is very active, but a lot of extension models are early beta versions or simply don't exist. Please keep that in mind and prefer to comment about models here rather than opening issues.
Model Download
There are various options to get started with Flux, they differ in complexity and hardware requirements.
Flux can be used with the managed ComfyUI installed by the plugin, or a custom install. Either way you need to download the models to the indicated folder where your models are stored. For managed installs, the models folder is located directly inside your server install folder.
Option 1: Full Checkpoint
The easiest way is to download a full checkpoint which includes everything in one package. Choose any of the following models:
/models/checkpoints/models/checkpointsThe models must be placed into the
/models/checkpointsfolder. Then you will be able to choose them from the Checkpoint list in the Style settings.16 GB VRAM recommended, but it will work with less.
Option 2: Separate Diffusion Model
Many distributions of Flux contain only the diffusion model. In order to use the model you also need the text encoders and image auto-encoder, as separate files. Because all variants of Flux share the same text/image encoders, this gives you more flexibility and saves disk space.
Install the GGUF custom node
This is only needed if you have a custom ComfyUI install and want to use GGUF files. Please install city96/ComfyUI-GGUF. You can also find them in ComfyUI Manager.
Download the Text Encoders
Clip-L (clip_l.safetensors) →
/models/clipT5-XXL (t5xxl_fp8_e4m3fn.safetensors) →
/models/clipYou can also alternatively use T5 quantizations from city96/t5-v1_1-xxl-encoder-gguf -
Q5_K_Mis a good trade-off.Download the VAE
Flux AE (ae.safetensors) →
/models/vaeDownload a Diffusion Model
Pick one, depending on your available VRAM and desired quality:
These files need to go into the
/models/diffusion_modelsfolder!The plugin will show them in the same place as regular checkpoints and you can use them in the same way, as long as you have all prerequisite models.
Inpaint (Fill, Expand, Add/Remove Object)
Use one or the other method for inpainting (not both):
Option 1: AlimamaCreative ControlNet
This can be used in combination with any Flux [dev] model and works similar to existing inpainting models.
/models/controlnetPlease rename the file to
FLUX.1-dev-Controlnet-Inpainting-Beta.safetensors.If you don't want to use the inpaint model, use the "Generate (Custom)" option and uncheck "Seamless".
Option 2: Flux Fill
Download the Flux Fill model (safetensors or GGUF) and put it into
/models/diffusion_models. You need to select it as your model checkpoint in the Style settings.Now use this style for inpainting (Fill/Expand/Add). You must have a selection and 100% strength, it won't work for anything else.
Redux (Reference Control Layer)
To make use of the Reference control layers with Flux, download the "Redux" model from Flux.1-Tools.
/models/clip_vision/models/style_modelsControlNet
The following models are supported:
/models/controlnet/models/controlnet/models/controlnet/models/controlnet(rename!)They are meant to be used with the "dev" version of Flux and don't work well with "schnell". They're also not available on cloud for now.
Flux Tools Lora
You can use the Lora provided by Flux-Tools as a replacement for the Depth and Canny control models listed above. They will be preferred if present.
/models/loras/models/lorasCommon Issues
The Flux workload is not installed
Try restarting Krita if you copied in a new model while it was running
Text encoder or VAE not found
Make sure models are in the correct folder and have the correct name. Otherwise they cannot be found.
Please check client.log file for more info.
Future Work
Beta Was this translation helpful? Give feedback.
All reactions