img2img alternative script - study

[Ehplodor]

EDIT : previous study : #314

base picture generated by txt2img :
a living room with a big white wall
Steps: 50, Sampler: DPM adaptive, CFG scale: 7, Seed: 1402990190, Size: 512x512, Clip skip: 2

sending to img2img... with img2imgalt script :

test n°1 :
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler, CFG scale: 1, Seed: 733074592, Size: 512x512, Denoising strength: 1, Clip skip: 2, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 1, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

test n°2 : decode cfg scale from +1 to -1
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler, CFG scale: 1, Seed: 908896074, Size: 512x512, Denoising strength: 1, Clip skip: 2, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: -1, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

test n°3 : cfg scale (decode and main) at +3
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler, CFG scale: 3, Seed: 104285665, Size: 512x512, Denoising strength: 1, Clip skip: 2, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 3, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

test n°4 : decode cfg scale from +3 to -3
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler, CFG scale: 3, Seed: 1321855819, Size: 512x512, Denoising strength: 1, Clip skip: 2, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: -3, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

test n°5 : cfg scale (both decode and main) at +7
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler, CFG scale: 7, Seed: 3352041221, Size: 512x512, Denoising strength: 1, Clip skip: 2, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 7, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

test n°6 : decode cfg scale from +7 to -7
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler, CFG scale: 7, Seed: 2427111055, Size: 512x512, Denoising strength: 1, Clip skip: 2, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: -7, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

[Ehplodor]

So clearly decode CFG scale should be negative IMHO.
That has already been reported elsewhere...
#569
#546

[Ehplodor]

Euler Ancestral tests with img2imgalt script :

test n°7 :
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 3864474382, Size: 512x512, Denoising strength: 1, Clip skip: 2, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: -7, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

test n°8 : decode cfg scale from -7 to zero
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 557598640, Size: 512x512, Denoising strength: 1, Clip skip: 2, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 0, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

test n°9 : lower denoising strength from 1 to 0.9
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 1668527541, Size: 512x512, Denoising strength: 0.9, Clip skip: 2, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 0, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

[Ehplodor]

clip skip from 2 to 1

test n°10 :
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 3378976777, Size: 512x512, Denoising strength: 0.9, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 0, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

[Ehplodor]

more flowers now that no more skipped clip layer

[Ehplodor]

without sigma adjustment

test n°11
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 3838348395, Size: 512x512, Denoising strength: 0.9, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 0, Decode steps: 50, Randomness: 0, Sigma Adjustment: False

[Ehplodor]

I don't know what to think about this one versus the previous...

[Ehplodor]

reusing seed from test n°10
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 3378976777, Size: 512x512, Denoising strength: 0.9, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 0, Decode steps: 50, Randomness: 0, Sigma Adjustment: False

[Ehplodor]

the image is more similar to the original WITH sigma adjustment

[Ehplodor]

trying DPM Adaptative

Without sigma adjustment
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: DPM adaptive, CFG scale: 7, Seed: 3378976777, Size: 512x512, Denoising strength: 0.9, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 0, Decode steps: 50, Randomness: 0, Sigma Adjustment: False

With sigma adjustment
a living room with a big white wall (((painted with huge colorful flowers)))
Steps: 50, Sampler: DPM adaptive, CFG scale: 7, Seed: 3378976777, Size: 512x512, Denoising strength: 0.9, Mask blur: 4, Decode prompt: a living room with a white wall, Decode negative prompt: , Decode CFG scale: 0, Decode steps: 50, Randomness: 0, Sigma Adjustment: True

[Ehplodor]

here, sigma adjustment doesn't have as dramatic effetcs as previously with Euler A

[Ehplodor]

Not much luck with trying to cover the walls with painted flowers... But (re) discovering that

1. samplers other than Euler can be used for interesting effects.
2. Decode cfg scale should be negative
3. Sigma adjustment seems desirable, but that remains open
4. With Euler, more consistent results with main cfg scale positive and decode cfg scale = - main cfg scale, and denoising strength set to maximum (1.0)
5. With Euler A, lower denoising strength a bit to get more consistent results. Also, main cfg scale can be disconnected from decode cfg scale

[EtherealIntellect]

The white wall seems really hard for most img2img methods, have you tried adding noise with the noise slider ? I feel that might help some - tho i havent found the time to test it. Novelai has an "add noise" built into their user interface, i'm guessing because anime often has large flat colored areas too

I just posted an outpainting focused #3232 - but covering the wall with noise (or a transparent noise layer) might give it something/a texture to draw the flowers out of better.

[Ehplodor]

Yes ! Great work ! I like the "stuff trick" very much !