how to set the noise and strength separately like novelAI when I'm using img2img?

[Pure-island]

[mezotaken]

related #4988
There is no way to set it currently, and i cant really understand how it's supposed to work.
Process works in such a way, that initial gaussian noise with mean=0 and sigma=1 is used, and then it is being reconstructed every step by specific schedule after denoising.
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Custom-Scripts#alternate-noise-schedules
there is a script to override sigma values, but it doesnt work as i expected.
Manually setting lower variance for initial noise didnt work for me either, i just got blurry nothing, seems to me that there must be "just the right amount" of it, otherwise high denoise value will destroy the picture itself.
We need an expert to make sense of it.
I feel like this parameter could still use initial N(0,1) noise but then during reconstruction only noise_strength*scheduled_noise will be added back.

[ghost]

I believe you can do this by adding more of the initial noise that's generated for img2img (it looks like NovelAI does something similar). If you want to try this you can modify this function:

stable-diffusion-webui/modules/sd_samplers.py

Lines 447 to 458 in ce049c4

	def sample_img2img(self, p, x, noise, conditioning, unconditional_conditioning, steps=None, image_conditioning=None):
	steps, t_enc = setup_img2img_steps(p, steps)
	if p.sampler_noise_scheduler_override:
	sigmas = p.sampler_noise_scheduler_override(steps)
	elif self.config is not None and self.config.options.get('scheduler', None) == 'karras':
	sigmas = k_diffusion.sampling.get_sigmas_karras(n=steps, sigma_min=0.1, sigma_max=10, device=shared.device)
	else:
	sigmas = self.model_wrap.get_sigmas(steps)
	sigma_sched = sigmas[steps - t_enc - 1:]
	xi = x + noise * sigma_sched[0]

Add this line before the line that sets xi:

x = x + (noise * 0.2)

That being said, I think the implementation must be slightly different somewhere, because that actually gives worse, more "noisy" results for me. Also when the noise is set to 0 (which would essentially be the value it's set at in A1111's webui currently), it produces different results than when NovelAI's noise is set to 0. To me it feels more like the A1111's outputs currently are close to what NovelAI's are when the noise is set, which makes me think maybe the extra noise is already being added somewhere else, but I can't say for sure.

[mezotaken]

yup, did it like that exactly, just modified generated noise itself bit before it being passed to this function. Works terrible no matter if its more or less noise variance than 1.

stable-diffusion-webui/modules/processing.py

Lines 357 to 363 in ce049c4

	xs.append(noise)
	if sampler_noises is not None:
	p.sampler.sampler_noises = [torch.stack(n).to(shared.device) for n in sampler_noises]
	x = torch.stack(xs).to(shared.device)
	return x

I kinda preferred it here because there's just one place to set it for every sampler including txt2img, but 4 places to set it in sd_samplers. But we won't be able to sum it from there, that's the downside.

And well, as you suggested to add more noise it results in more noisy output, thats understandable and that's what i expected. Slight overnoise will bring more details, sure. But i kinda hoped to play with less than normal std amount of noise and high denoising value, to preserve original and refine it strongly in the same time,

Main thing i struggle to understand is that when there's LESS initial noise (even if by small amount) than expected by scheduler computing sigmas, quality is by far worse.

Tested it a bit more, and slight noise increase indeed extends detail without any quality drops
With basic noise:

With 0.1 additional noise:

Welp, seems like you're correct: this parameter just adds a bit more noise than expected to create additional details.