-
Notifications
You must be signed in to change notification settings - Fork 27.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis #1417
Comments
This seems to rely on parsing the input prompt to seperate nouns and tokenize each one seperately. I lack any experience with such a thing - though I tried anyways and failed. It also uses a NLP model and depends on stanza. As far as attention goes, I believe this would be sufficient.
@AUTOMATIC1111 would you be interested in this? |
Where would that go, I'd like to try it! |
@differentprogramming It won't work. The hard part is in txt2img |
I tried to run the sample version but it dies: 2022-10-01 23:53:04 INFO: Use device: gpu During handling of the above exception, another exception occurred: Traceback (most recent call last): |
That error has nothing to do with the code, you're just failing to connect to huggingface, probably because you didn't supply login info |
Up. This is a must IMHO |
Is your feature request related to a problem? Please describe.
Don't think this is a duplicate of anything else and shouldn't be confused with #1325. This is related to the problems showcased in these images provided by the research paper that will be linked below. Anyone who uses SD on a frequent basis should know some of these issues far too well.
Describe the solution you'd like
Implementation of the changes made to txt2img.py and attention.py to reduce these problems that show up in AI image generation. Obviously this shouldn't replace the default and should be an option offered as opt-in with plenty of warning that it will produce different results than what is currently produced.
Appropriate links to the research page, paper, and zip file that contains their modified txt2img.py and attention.py
https://openreview.net/forum?id=PUIqjT4rzq7
https://openreview.net/pdf?id=PUIqjT4rzq7
https://openreview.net/attachment?id=PUIqjT4rzq7&name=supplementary_material
The text was updated successfully, but these errors were encountered: