SDXL Inpainting

[GitHub1712]

Can we use the new diffusers/stable-diffusion-xl-1.0-inpainting-0.1 model?
Someone got it working in webui already?

[sipie800]

+1

[Renaldas111]

Also interested in loading the model in A1111.

[captainzero93]

+1 and can it be merged with merged models to make custom inpainting models too ?

[ZeroCool22]

So...?

[mykeehu]

Any news?

[lwz1357]

I also want to use this model, tried format conversion but no results, has anyone used it?

[kalle07]

sdxlInpainting01Official_v01-inpainting.safetensors

its HALF so its dont work, if i start with --no-half, my 16GB RTX VRAM runs out of memory

[Littleor]

Hello, I'm currently looking for guidance on where or how to download the sdxlInpainting01Official_v01-inpainting.safetensors. I located the model on https://huggingface.co/diffusers/stable-diffusion-xl-1.0-inpainting-0.1, but I'm unsure about how to integrate it with the WebUI. Any assistance or directions would be greatly appreciated. Thank you!

[kalle07]

i only know it for SD1.5
go to img2img tab, choose your inpaint model ...
choose maybe inpaint, load a pic (resize same dimensions), maske it, write a prompt and generate

[lucasjinreal]

So/..............?

[gel-crabs]

Sadly SDXL inpaint currently only works with diffusers, there needs to be support in SGM (https://github.com/Stability-AI/generative-models) for it to work. Let them know here. (Stability-AI/generative-models#45)

There is an inpainting safetensors and instructions on how to create an SDXL inpainting model here (https://huggingface.co/benjamin-paine/sd-xl-alternative-bases), however it won't work until SGM gets a yaml file for SDXL inpainting.

[andyRyuSk]

any update?

[wangqyqq]

I made a pull request #14390 here to support sdxl-inpaint model. The model can be download at wangqyqq/sd_xl_base_1.0_inpainting_0.1.safetensors

[wangqyqq]

the error is happened when writing text info to image. this has nothing to do with my code.
I wrote the codes based on the newest version webui 1.7.0. Did you update your codes to 1.7.0？

[kalle07]

thy ... updated ... now it work s : D
now its time to play and may create new inpaints

you have any idea when its regulary merged in automatic1111 ?
and you can tell me what is new in 1.7. 0 or it give a version history ?
ah and how i can check if an inpaintmodel that i merged is caclculated right with your yaml file ?

[wangqyqq]

I'm not the developer of the repository so i don't know when.
You can check the changelog for what's new in every version.
If it works, you can think it calculate right.

[sipie800]

not working here, merge your code into 1.7. While load your safetensor file, error says,

RuntimeError: Error(s) in loading state_dict for DiffusionEngine:
size mismatch for model.diffusion_model.input_blocks.0.0.weight: copying a param with shape torch.Size([320, 9, 3, 3]) from checkpoint, the shape in current model is torch.Size([320, 4, 3, 3]).

[wangqyqq]

this error indicates that you may not merge codes rightly. Because if you don't merge the codes, you will see this error too.
There are 4 files to be update, you could check it again.

[ZeroCool22]

Still not working...

Loading weights [fe1b97fe65] from C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\models\Stable-diffusion\sd_xl_base_1.0_inpainting_0.1.safetensors
Creating model from config: C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\repositories\generative-models\configs\inference\sd_xl_base.yaml
changing setting sd_model_checkpoint to sd_xl_base_1.0_inpainting_0.1.safetensors: RuntimeError
Traceback (most recent call last):
  File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\options.py", line 146, in set
    option.onchange()
  File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\call_queue.py", line 13, in f
    res = func(*args, **kwargs)
  File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\initialize_util.py", line 174, in <lambda>
    shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
  File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\sd_models.py", line 783, in reload_model_weights
    load_model(checkpoint_info, already_loaded_state_dict=state_dict)
  File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\sd_models.py", line 658, in load_model
    load_model_weights(sd_model, checkpoint_info, state_dict, timer)
  File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\sd_models.py", line 375, in load_model_weights
    model.load_state_dict(state_dict, strict=False)
  File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\sd_disable_initialization.py", line 223, in <lambda>
    module_load_state_dict = self.replace(torch.nn.Module, 'load_state_dict', lambda *args, **kwargs: load_state_dict(module_load_state_dict, *args, **kwargs))
  File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\modules\sd_disable_initialization.py", line 221, in load_state_dict
    original(module, state_dict, strict=strict)
  File "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DiffusionEngine:
        size mismatch for model.diffusion_model.input_blocks.0.0.weight: copying a param with shape torch.Size([320, 9, 3, 3]) from checkpoint, the shape in current model is torch.Size([320, 4, 3, 3]).

[wangqyqq]

I committed codes here, you need to merge them to your code to load and inference sdxl-inpaint model.
There are 4 files to be updated.
If you directly replace with the 4 files, your webui need to be updated to version 1.7.0 first.

[ZeroCool22]

Sorry i never did hat before, can you give me the instructions? I do it via CMD?

[sipie800]

I do use 1.7.0 and replace 4 files, and get the same error. There are something other than these.

[wangqyqq]

1. git clone -b sdxl-inpaint https://github.com/wangqyqq/stable-diffusion-webui.git
2. cd stable-diffusion-webui
3. bash webui.sh
4. download sdxl-inpaint model to stable-diffusion-webui/models/Stable-diffusion