[FEATURE REQUEST] Cross Attention Control

[PhreakHeaven]

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
I came across something that seems like it would be a huge addition (if possible). It's called Cross Attention Control; you're probably already aware of it. If it is able to be implemented, that would be awesome, considering it's potential to help with fine-tuning.

Problem is, I'm not familiar with code that utilizes, for example, Colab and/or Jupyter, so I'm unaware of the possibility of this occurring. I mainly join any Discord servers and subreddits that I find related to SD and end up randomly stumbling upon all of these different projects people are developing and imagine the potential of such features packed into this project. Let me know what you think! Thanks!"

Describe alternatives you've considered
Don't know enough to add info here!

Additional context
Here's an example I found:

[JustAnOkapi]

I think this might be the same thing as negative prompt plus weights

[JustAnOkapi]

Oh no this looks like something that can be a custom script.
would have to convert the notebook to a script
bloc97/CrossAttentionControl#7

[JustAnOkapi]

I also dont see why you cant just img2img it especially with the new alternative img2img

[PhreakHeaven]

I actually just realized there was a new img2img that seems to be quite similar...not sure if the Cross Attention Control is any different then. Many people here would know more than me about this; just thought the CAC was pretty cool when I ran across it, so if it is the same thing, then no need for it to be implemented here, I suppose 😬

[ExponentialML]

Cross Attention Control a little bit different than the img2img alternate solution that was recently integrated. They're supposed to work very well in tandgem, but block97 hasn't gotten around to it yet. I know that Doggettx has created a solution that was inspired by his work that should be easy to implement with this repository.

[PhreakHeaven]

I saw a comment by bloc97 on the Reddit link (AUTOMATIC1111 called it ebaumsworld.com for some reason, lol...maybe a joke I'm not aware of?) that was added to the README.md:

Awesome, I can't wait to combine this with cross attention control, this will actually allow people to edit an image however they want at any diffusion strengths! No more the problem of img2img ignoring the initial image at high strengths. I will take a look at the code tomorrow...

I think I need to start a new 'Issue' because I followed the instructions from the README, and that Reddit post (they say the same thing), and I DID finally get a very similar image (as well as the desired change, which needs to be inputted into the normal 'Prompt' field up top) with the alternate img2img, BUT, it only worked after I removed the attentions (() and []) from the 'Original Prompt' field in the script area below; my original prompt had attentions, of course. I messed with it for like 1.5 hours before that, and kept getting no similar generations, so I decided to see if 'Interrogate'-generated prompt would work any better, and it did, however, the background is like large noise, so it didn't resolve the wall behind the subject properly (maybe steps need to be lowered...?)

[C43H66N12O12S2]

implemented

[cmp-nct]

@C43H66N12O12S2 You mean the img2img noise finder ? Isn't that a totally different solution ?
(Here is the current code on cross attention prompt to prompt editing: https://github.com/bloc97/CrossAttentionControl/blob/main/CrossAttention_Release_NoImages.ipynb)

[PhreakHeaven]

@C43H66N12O12S2

Wait...how do we use it if it's implemented? I haven't seen anything about it...

[Omegastick]

@C43H66N12O12S2 I don't think this is implemented. If you look at Google's implementation, this is different to img2img alt and the prompt scheduler.

[aungkhant0911]

So this Promp2Prompt feature, as called by Google, HASN'T been implemented in the webui so far right?

[illtellyoulater]

What's the current status of this?