Amazing outpainting code for SD

[1blackbar]

https://www.reddit.com/r/StableDiffusion/comments/xbjnnu/huge_outpainting_in_1_step_without_erased_colors/

[oobabooga]

This and #291 are huge improvements.

Pasting the code linked in the Reddit thread for completeness:

Code available here:

https://github.com/parlance-zz/g-diffuser-bot/tree/g-diffuser-bot-diffuserslib-beta

Explanation: Getting good results in/out-painting with stable diffusion can be challenging. Although there are simpler effective solutions for in-painting, out-painting can be especially challenging because there is no color data in the masked area to help prompt the generator. Ideally, even for in-painting we'd like work effectively without that data as well. Provided here is my take on a potential solution to this problem.

By taking a fourier transform of the masked src img we get a function that tells us the presence and orientation of each feature scale in the unmasked src. Shaping the init/seed noise for in/outpainting to the same distribution of feature scales, orientations, and positions increases output coherence by helping keep features aligned. This technique is applicable to any continuous generation task such as audio or video, each of which can be conceptualized as a series of out-painting steps where the last half of the input "frame" is erased. For multi-channel data such as color or stereo sound the "color tone" or histogram of the seed noise can be matched to improve quality (using scikit-image currently) This method is quite robust and has the added benefit of being fast independently of the size of the out-painted area. The effects of this method include things like helping the generator integrate the pre-existing view distance and camera angle.

[ThereforeGames]

This also relies on "shaping the init/seed noise" similar to the new inpainting code that was posted today ( #288 ).

I wonder which does a better job at this step?

[Dawgmastah]

This also relies on "shaping the init/seed noise" similar to the new inpainting code that was posted today ( #288 ).

I wonder which does a better job at this step?

This one is probably better for outpainting since it sort of mathemathically refines the..style
and not so much the exact initial image?

But maybe we can just compare or they do the same thing, I havent looked into detail

[DenkingOfficial]

@1blackbar, image that you attached is not created by the same author of the post that you also attached.

Original creator of the image:

Code author:

They are different people.

So I guess the code, which is used to outpaint attached picture is different.

[Dawgmastah]

You are correct, the example outpainting in the first post isnt with this code, it was according to the author a more painstaking process (with editing software I assume) but these examples (with WIP builds) are:

https://imgur.com/a/S6g5SmI

[TheEnhas]

This is much better and closer to DALL·E 2's outpainting IMO, seems like every day there's more and more advances it's great

[1blackbar]

Oh yes, there were two guys that i read at the same time !

[vinyvince]

Talking about inpainting, currently i can't found a way to extend an image (like with the poor map script and extand cropping of picture ) and use a custom mask in the same time. When switch to redrawing fill image to inpainting, the editing feature of the guiding image is not available anymore.

[oobabooga]

I would like to be annoying and tag @AUTOMATIC1111, as I believe that this is very promising.

[ntrpnr]

I'd love to see this here as well.

[oobabooga]

Here is a new, relevant resource:

https://github.com/lkwq007/stablediffusion-infinity
https://twitter.com/lkwq007/status/1570089301194711040

[C43H66N12O12S2]

implemented