Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update live preview only after a deliberate change rather than mid-change #628

Open
lboklin opened this issue Apr 18, 2024 · 3 comments
Open

Comments

@lboklin
Copy link

lboklin commented Apr 18, 2024

Currently the live preview will generate new images constantly while I am in the process of making changes to the image or prompt, even mid-stroke/mid-typing. This is unnecessary and will only delay the final generation based on the change I mean to make, not to mention the constant output of images that will inevitably be replaced (it can be frustrating to see incidentally desirable results and not being able to apply or reproduce it).

For this, some basic heuristics would be useful, if possible, to hold off on generating previews until there is a pause in prompt editing or a stroke has been completed. Krita itself decides which actions should be undo-able, and that heuristic could be a good starting point at least as it pertains to image modifications.

@lboklin
Copy link
Author

lboklin commented Apr 18, 2024

In addition--and I don't know if this should be its own issue or not, or even if this is the already intended behaviour--if a new live preview is requested while one is already in the process of being generated, I would like for the currently processed one (and any others queued) to be immediately cancelled.

Currently (especially noticeable given the main issue above and my humble hardware), new requests appear to be appended to the queue or at least wait for the current one to finish before being processed. This leads to a series of fully generated intermediate results based on old or incomplete updates being displayed.

Edit: I don't know how this could be handled with respect to existing queues from before switching to live preview. It would be unfortunate if those queues were overriden.

@Acly
Copy link
Owner

Acly commented Apr 20, 2024

So the idea of live preview is that it's (more or less) "live", ie. it updates in less than a second. I know this is not achievable on a lot of hardware, but there is the "classic" generation mode for handling it as longer running background task. What you're suggesting make me thing that it's more suited for what you want...

To clarify some points:

  1. Live requests are never queued, a new one is only started when the previous one has finished. Work queued previous from other modes is completed first.
  2. Cancelling live requests does not make sense, the idea is that it updates while you draw/type/work. If it cancelled on every change, it would never generate an image at all in that case - not until you sit back and actively wait without touching anything. It would be very frustrating.
  3. Only updating for "deliberate changes" makes some sense, but is difficult to implement. Python plugins don't have access to Krita's undo stack, or even any events that indicate things have changed. It's currently implemented via periodic polling and full comparisons of the pixel data... not much to go on. For text changes it would be under control, but even that seems a lot of work for minor effect.

And maybe as a workaround: You can set a hotkey for "Generate" in Krita's keyboard settings. It also works in live mode when it's not active. So if you prefer the preview on the side, you can use it to manually trigger generation with 1 button press.

@lboklin
Copy link
Author

lboklin commented Apr 20, 2024

What you say makes sense and I agree that neither cancelling nor queueing make much sense. I would however consider (3) a significant improvement to the UX. Without access to the undo stack, is there perhaps a way to query activity in some other way to at least implement some sort of delay for when modifications are being actively made?

And maybe as a workaround: You can set a hotkey for "Generate" in Krita's keyboard settings. It also works in live mode when it's not active. So if you prefer the preview on the side, you can use it to manually trigger generation with 1 button press.

That's a good workaround, thanks for the tip!

modelflat added a commit to modelflat/krita-ai-diffusion that referenced this issue Nov 8, 2024
…on in live preview

* This will prevent some of the "useless" generations, e.g. from the very start of
  the brush stroke
* Period is configurable in settings; setting the default to 0 to
  preserve the existing behaviour

This at least partially addresses/follows the discussions in Acly#628 and Acly#1248
modelflat added a commit to modelflat/krita-ai-diffusion that referenced this issue Nov 26, 2024
…on in live preview

* This will prevent some of the "useless" generations, e.g. from the very start of
  the brush stroke
* Period is configurable in settings; setting the default to 0 to
  preserve the existing behaviour

This at least partially addresses/follows the discussions in Acly#628 and Acly#1248
modelflat added a commit to modelflat/krita-ai-diffusion that referenced this issue Nov 26, 2024
…on in live preview

* This will prevent some of the "useless" generations, e.g. from the very start of
  the brush stroke
* Period is configurable in settings; setting the default to 0 to
  preserve the existing behaviour

This at least partially addresses/follows the discussions in Acly#628 and Acly#1248
Acly pushed a commit that referenced this issue Nov 29, 2024
…on in live preview (#1412)

* Add a grace period between detecting a change and triggering generation in live preview
* This will prevent some of the "useless" generations, e.g. from the very start of
  the brush stroke
* Period is configurable in settings; setting the default to 0 to
  preserve the existing behaviour
This at least partially addresses/follows the discussions in #628 and #1248
* Don't exit live generation loop when switching documents
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants