k-diffusion-euler #1019

hlky · 2022-10-27T11:44:34Z

k_euler and k_euler_a

HuggingFaceDocBuilderDev · 2022-10-27T11:47:50Z

The documentation is not available anymore as the PR was closed or merged.

patil-suraj

Thanks a lot for the @hlky , very cool addition!

The code looks good to me, will try it out a bit more and comment here. Also it looks like the new tests are failing, happy to take a look at it.

But should be good to merge for me once the tests are green!

src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py

tests/pipelines/stable_diffusion/test_stable_diffusion.py

patil-suraj · 2022-10-28T09:16:09Z

Hey @hlky would it be okay if I go push commits to the PR to fix the tests ? The PR should be good to merge once the tests pass :)

hlky · 2022-10-28T09:52:24Z

@patil-suraj Sure, no problem :)

anton-l

Thank you @hlky, excited to see these schedulers in action!

src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py

src/diffusers/schedulers/scheduling_euler_discrete.py

patil-suraj · 2022-10-28T14:47:02Z

Hey @hlky , it seems like I can't push to your branch, if it's okay with you, could you allow me to push commits to your branch so I could fix the tests ?

Otherwise, happy to merge this and open a follow-up PR to fix the tests. Let me know. Thank you :)

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

…users into k-diffusion-euler

tonetechnician · 2022-10-28T16:16:09Z

This is great! Thanks @hlky and @patil-suraj

One thing I noticed is that in the scheduler's step function you're generating random noise, and not seeded noise. This causes the output to be non-deterministic even when the same seed is used.

I've made a commit here which I'd be happy to push to this branch to your PR if you give access to me too. Or can make a PR to your repo. Or can wait for merge and then make a new PR with the modification, it's not a big one.

One thought though is at each step a new "random" noise should be generated in which case I think maybe we can have some kind of cyclical counter that offsets the generator's seed in the step function so it's continually random. I'm not sure it'll affect the quality of the result doing this, but would affect output

src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py

tests/test_scheduler.py

patil-suraj · 2022-10-28T16:34:50Z

Hey @tonetechnician you are right, and actually that's why the tests were failing. If we pass the generator we need to make sure that it is generic across all schedulers as we want all schedulers in diffusers to be interchangeable. Let's wait to see what @patrickvonplaten @anton-l @pcuenca has to say. I have proposed two ways on the comment above.

Also fine to merge this now, and open a follow-up PR to add this :)

One thought though is at each step a new "random" noise should be generated in which case I think maybe we can have some kind of cyclical counter that offsets the generator's seed in the step function so it's continually random. I'm not sure it'll affect the quality of the result doing this, but would affect output

Very good point! The cool thing about Generator is that, each time it's used in a random function it's state gets advanced, so if we pass generator we can be sure that it will be random each time.

src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

tests/test_scheduler.py

src/diffusers/schedulers/scheduling_ipndm.py

anton-l

LGTM!

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

patrickvonplaten

Amazing work @hlky and @patil-suraj 🥳

patrickvonplaten · 2022-10-31T15:20:33Z

Merging as it blocks #890

patil-suraj · 2022-10-31T16:02:15Z

Thanks a lot @hlky and @tonetechnician !

WASasquatch · 2022-11-02T17:00:35Z

I hope you guys know Euler_A is non-determinisitc in this implementation, and do not work with generator seeds. Random result every time.

andrea-gatto · 2022-11-03T15:00:06Z

@WASasquatch I'm able to get reproducible results on GPU. You need to set your generator
generator = torch.Generator("cuda").manual_seed(seed)
and pass it to the pipeline
You'll get an error because here (and same for euler ancestral)

diffusers/src/diffusers/schedulers/scheduling_euler_discrete.py

Line 217 in 269109d

    
           noise = torch.randn(model_output.shape, dtype=model_output.dtype, generator=generator).to(device)

torch.randn expects the generator to be on cpu. This can be fixed by changing the line to
noise = torch.randn(model_output.shape, dtype=model_output.dtype, generator=generator, device=device)

cc @patil-suraj

patil-suraj · 2022-11-03T16:52:46Z

Great catch @andrea-gatto !

geocine · 2022-11-03T16:58:22Z

Are you making a PR @andrea-gatto ? I also experienced this just now

patil-suraj · 2022-11-03T17:06:10Z

PR is here #1124

WASasquatch · 2022-11-04T03:24:40Z

@WASasquatch I'm able to get reproducible results on GPU. You need to set your generator generator = torch.Generator("cuda").manual_seed(seed) and pass it to the pipeline You'll get an error because here (and same for euler ancestral)

diffusers/src/diffusers/schedulers/scheduling_euler_discrete.py

Line 217 in 269109d

noise = torch.randn(model_output.shape, dtype=model_output.dtype, generator=generator).to(device)

torch.randn expects the generator to be on cpu. This can be fixed by changing the line to
noise = torch.randn(model_output.shape, dtype=model_output.dtype, generator=generator, device=device)
cc @patil-suraj

The issue isn't an error. It runs without error, the results for Euler_A are just random, while every other sampler works fine with torch generator. The code is fine, it's simple Euler_A isn't deterministic with a set seed like the rest. Though I did merge code hours before it was officially merged with a fork of Diffusers, as I was tired of waiting. ;) But I don't see any differences in files to current state on my end so not sure what it'd be with that specific sampler.

andrea-gatto · 2022-11-04T12:34:24Z

@WASasquatch I get deterministic results with Euler Ancestral on GPU following that fix, maybe the fork you were using wasn't exactly the same (which could explain why you didn't get the error mentioned above), give diffusers==0.7.0 a try :)

WASasquatch · 2022-11-04T17:08:52Z

Well, it couldn't possibly be not the same code base,it was installed from the PR, but possibly just outdated. Though seems you guys merged only few ours after I forked diffusers/PR.

…

On Fri, Nov 4, 2022 at 5:34 AM andrea-gatto ***@***.***> wrote: @WASasquatch <https://github.com/WASasquatch> I get deterministic results with Euler Ancestral on GPU following that fix, maybe the fork you were using wasn't exactly the same (which could explain why you didn't get the error mentioned above), give diffusers==0.7.0 a try :) — Reply to this email directly, view it on GitHub <#1019 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIZEZNVEXDFU2ANX4PQXB3WGT7FZANCNFSM6AAAAAARP6566U> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Sincerely, *Jordan S. C. Thompson *

… one config file (huggingface#1019)

* k-diffusion-euler * make style make quality * make fix-copies * fix tests for euler a * Update src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com> * Update src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com> * Update src/diffusers/schedulers/scheduling_euler_discrete.py Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com> * Update src/diffusers/schedulers/scheduling_euler_discrete.py Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com> * remove unused arg and method * update doc * quality * make flake happy * use logger instead of warn * raise error instead of deprication * don't require scipy * pass generator in step * fix tests * Apply suggestions from code review Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * Update tests/test_scheduler.py Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * remove unused generator * pass generator as extra_step_kwargs * update tests * pass generator as kwarg * pass generator as kwarg * quality * fix test for lms * fix tests Co-authored-by: patil-suraj <surajp815@gmail.com> Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com> Co-authored-by: Pedro Cuenca <pedro@huggingface.co> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

k-diffusion-euler

41e3bf2

hlky added 2 commits October 27, 2022 12:49

make style make quality

9032c0b

make fix-copies

5496649

hlky mentioned this pull request Oct 27, 2022

0.5.1 k schedulers #944

Closed

patil-suraj self-assigned this Oct 27, 2022

patil-suraj reviewed Oct 27, 2022

View reviewed changes

src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py Outdated Show resolved Hide resolved

tests/pipelines/stable_diffusion/test_stable_diffusion.py Show resolved Hide resolved

patil-suraj requested review from anton-l, patrickvonplaten and pcuenca October 27, 2022 14:00

anton-l approved these changes Oct 28, 2022

View reviewed changes

fix tests for euler a

6753644

hlky and others added 6 commits October 28, 2022 16:45

Update src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py

85ae890

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

Update src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py

bdc8334

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

Update src/diffusers/schedulers/scheduling_euler_discrete.py

003a61b

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

Update src/diffusers/schedulers/scheduling_euler_discrete.py

8a3b4a3

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

Merge branch 'k-diffusion-euler' of https://github.com/Sygil-Dev/diff…

07a99c7

…users into k-diffusion-euler

remove unused arg and method

44c0509

patil-suraj added 2 commits October 28, 2022 18:19

update doc

0126944

quality

deedc4e

patil-suraj reviewed Oct 28, 2022

View reviewed changes

src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py Outdated Show resolved Hide resolved

patil-suraj reviewed Oct 28, 2022

View reviewed changes

tests/test_scheduler.py Outdated Show resolved Hide resolved

make flake happy

dde3f8d

patrickvonplaten reviewed Oct 28, 2022

View reviewed changes

src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py Outdated Show resolved Hide resolved

patrickvonplaten reviewed Oct 28, 2022

View reviewed changes

src/diffusers/schedulers/scheduling_euler_ancestral_discrete.py Outdated Show resolved Hide resolved

Apply suggestions from code review

3198b77

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

patrickvonplaten reviewed Oct 31, 2022

View reviewed changes

tests/test_scheduler.py Show resolved Hide resolved

patrickvonplaten reviewed Oct 31, 2022

View reviewed changes

src/diffusers/schedulers/scheduling_ipndm.py Outdated Show resolved Hide resolved

anton-l approved these changes Oct 31, 2022

View reviewed changes

patil-suraj and others added 7 commits October 31, 2022 14:04

Update tests/test_scheduler.py

30db08a

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

remove unused generator

9cf1cf0

pass generator as extra_step_kwargs

b1324ca

update tests

c7fe0a0

pass generator as kwarg

c5e6aa5

pass generator as kwarg

d6daae7

quality

5993631

patil-suraj requested a review from patrickvonplaten October 31, 2022 13:28

patil-suraj added 2 commits October 31, 2022 14:33

fix test for lms

6d484c3

fix tests

207a5d2

patrickvonplaten approved these changes Oct 31, 2022

View reviewed changes

patrickvonplaten merged commit a1ea8c0 into huggingface:main Oct 31, 2022

patil-suraj mentioned this pull request Nov 3, 2022

handle device for randn in euler step #1124

Merged

patrickvonplaten mentioned this pull request Nov 16, 2022

[Design discussion] Diffusers schedulers/samplers & 2nd Order Schedulers #1308

Closed

PhaneeshB pushed a commit to nod-ai/diffusers that referenced this pull request Mar 1, 2023

[SD] Merge configs of different max lengthes from the same variant to…

2af1102

… one config file (huggingface#1019)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

k-diffusion-euler #1019

k-diffusion-euler #1019

hlky commented Oct 27, 2022

HuggingFaceDocBuilderDev commented Oct 27, 2022 •

edited

Loading

patil-suraj left a comment

patil-suraj commented Oct 28, 2022

hlky commented Oct 28, 2022

anton-l left a comment

patil-suraj commented Oct 28, 2022 •

edited

Loading

tonetechnician commented Oct 28, 2022 •

edited

Loading

patil-suraj commented Oct 28, 2022

anton-l left a comment

patrickvonplaten left a comment

patrickvonplaten commented Oct 31, 2022

patil-suraj commented Oct 31, 2022

WASasquatch commented Nov 2, 2022 •

edited

Loading

andrea-gatto commented Nov 3, 2022

patil-suraj commented Nov 3, 2022

geocine commented Nov 3, 2022

patil-suraj commented Nov 3, 2022

WASasquatch commented Nov 4, 2022 •

edited

Loading

andrea-gatto commented Nov 4, 2022

WASasquatch commented Nov 4, 2022 via email

k-diffusion-euler #1019

k-diffusion-euler #1019

Conversation

hlky commented Oct 27, 2022

HuggingFaceDocBuilderDev commented Oct 27, 2022 • edited Loading

patil-suraj left a comment

Choose a reason for hiding this comment

patil-suraj commented Oct 28, 2022

hlky commented Oct 28, 2022

anton-l left a comment

Choose a reason for hiding this comment

patil-suraj commented Oct 28, 2022 • edited Loading

tonetechnician commented Oct 28, 2022 • edited Loading

patil-suraj commented Oct 28, 2022

anton-l left a comment

Choose a reason for hiding this comment

patrickvonplaten left a comment

Choose a reason for hiding this comment

patrickvonplaten commented Oct 31, 2022

patil-suraj commented Oct 31, 2022

WASasquatch commented Nov 2, 2022 • edited Loading

andrea-gatto commented Nov 3, 2022

patil-suraj commented Nov 3, 2022

geocine commented Nov 3, 2022

patil-suraj commented Nov 3, 2022

WASasquatch commented Nov 4, 2022 • edited Loading

andrea-gatto commented Nov 4, 2022

WASasquatch commented Nov 4, 2022 via email

HuggingFaceDocBuilderDev commented Oct 27, 2022 •

edited

Loading

patil-suraj commented Oct 28, 2022 •

edited

Loading

tonetechnician commented Oct 28, 2022 •

edited

Loading

WASasquatch commented Nov 2, 2022 •

edited

Loading

WASasquatch commented Nov 4, 2022 •

edited

Loading