The different quality between ComfyUI and Diffusers ?

[vuongminh1907]

I had a problem using InstantID (https://github.com/instantX-research/InstantID), which uses Diffusers as its base. Additionally, I tried ComfyUI (https://github.com/cubiq/ComfyUI_InstantID), and the quality of the images improved better I think.

I discussed this with Cubiq, and he mentioned that there are no differences in how they applied the IP Adapter (cubiq/ComfyUI_InstantID#206).

Can you explain this issue to me? Perhaps it’s related to the Sampler in ComfyUI and Diffusers.

[asomoza]

You're not providing any code, we don't even know if you're using their repo or the community pipeline or at least which model you're using.

Also the images are really small to tell a difference, better if you post the actual generations.

Finally, this is the most important part:

and the quality of the images improved better I think.

The way you're phrasing this is that you're not sure? At the end of the day, if an image is better or not depends on the person that's looking, the images are really small for me to tell for sure, but I think the diffusers ones look more realistic and the comfyui ones more fake and plastic, but that just means that you like more the "smooth" kind of images.

[vuongminh1907]

Thanks for your reply, Firstly, if u want to reproduce images, you can follow these steps:

1. ComfyUI code:

• Checkpoints ip adapter , controlnet I used from this repo: https://github.com/cubiq/ComfyUI_InstantID
• Model Base: RealVisXL_V4.0
• File config I used:
  InstantID_basic.json

2. Diffusers Code:

• Clone this repo: https://github.com/instantX-research/InstantID
• pip install -r gradio_demo/requirements.txt
  python gradio_demo/download_models.py
• Then I run infer.py ( Replace face image and use RealVisXL_V4.0)

Otherwise, I have tested sufficient images to compare:

• Case 1:
  Prompt: comic character. graphic illustration, comic art, graphic novel art, fsdvibrant, highly detailed
  Negative prompt: photograph, deformed, glitch, noisy, realistic, stock photo

• Case 2:
  Prompt: city street, closeup photo of person in black clothes, serious face, very realistic, in London background
  Negative prompt: (octane render, render, drawing, anime, bad photo, bad photography:1.3), (worst quality, low quality, blurry:1.2), (bad teeth, deformed teeth, deformed lips), (bad anatomy, bad proportions:1.1), (deformed iris, deformed pupils), (deformed eyes, bad eyes), (deformed face, ugly face, bad face), (deformed hands, bad hands, fused fingers), morbid, mutilated, mutation, disfigured

I guarantee that the result is not cherry-picked. I found that the similarity of the Comfy code is better. I explored the differences between the two approaches and realized the problem comes from the way they apply Cross Attention.

However, I experimented with the impact of the IP adapter in InstantID and found it to be not significant. I wonder if the reason might be due to the Sampler used in the two methods?

[asomoza]

Thanks for the follow up, I was asking for real sized images because it will take time for me to install everything and test it, and I can't do that if this isn't a issue with diffusers.

I tried your case 1 prompt in the original space (I removed the fsdvibrant).

I get these images:

No style	Vibrant

They don't look bad and are totally different to what you get, probably because of the model you're using which is for realistic images.

In this case, since you have in your prompt this: graphic illustration, comic art, graphic novel art, the diffusers one is more right in my opinion, the ones you get from ComfyUI don't have any comic art in most of them.

Also you have something weird happening with the aspect ratios, the ComfyUI ones gets cropped but the diffusers ones seems to get stretched.

There was an issue with the DPM scheduler that was fixed in the 0.26 release if that's what you're using, but apart from that, the samplers pretty much have the same quality than ComfyUI or Automatic1111.

Since you're using an external repo, and one that uses an old version of diffusers (0.25) you'll get better answers there, we can't really provide support for external libraries that use custom code that is not maintained by us, still I'll test this when I have time because I'm intrigued, but sadly it won't be soon.