-
-
Notifications
You must be signed in to change notification settings - Fork 303
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Transition to Vulkan #208
Comments
Some conditions that must be met:
|
One big difference between OpenGL and Vulkan is that OpenGL sort of implicitly queues commands for the GPU behind the scenes, whereas Vulkan is very explicit about when the GPU is ready to process the next image. Due to this OpenGL behaviour, Nannou's current behaviour is that when Changing the
|
The new Frame typeSeeing as the new |
Calls to
|
I don't fully understand this domain so my thoughts might be a bit misguided but I would imagine that you would not want to couple the update call to the refresh rate of the monitor. It would not be bad for example for a 90hz monitor to draw the same model twice if the scenario didn't have a reason to change the model. I think it would be good to have a model that all monitors get in their acquire_next_image() and this model is updated by some master app loop with its own rate |
The reason I think it might be important to provide this option is in order to allow for minimizing latency as much as possible - that is the difference between the state that gets displayed and what the state actually should be at that exact instant. For example, if we have an arbitrary I think this generally gets really important for interactive applications or games where you want your view to seem as synchronised as possible with your actions. E.g. imagine the case where the
Just to clarify, |
Ah I see what you mean. |
Like if one screen gets ahead of the other it will only get the latest available state of the model. |
I guess what I'm concerned about is, if you were to call
Yeah true, maybe it should be possible to be able to specify a "minimum latency" interval in order to avoid this issue? That way when |
I was more imagining that each screen would be "subscribed" to the parts of the model that are relevant to it. Some of those parts would be shared among other screens. Each screen has it's own update that pulls in the relevant events and updates the model that it holds. Not calling one master update twice. Or could it be more that an events are only applied once and the model is shared like
I mean events in a sense more then UI events. Like you might have your physics engine sending movement events at a certain rate |
Ok I think I get what you're saying by grouping all input and application events into a single "event". To clarify, the reason why I am singling out user input events in particular is that those are the events that affect the feeling of "responsiveness" and low-latency to the user, as it is the point at which the user interacts with the system. The other distinction that I think is important is between events that occur from the external world (via I/O like user input, cam input, audio I/O, clocks, etc) vs events like physics updates that are generated within the application itself that the user has more control over. I'm generally lumping all the application events (physics etc) into the I do like the idea of allowing for an |
For anyone else following along, this is the new /// The mode in which the **App** is currently running the event loop and emitting `Update` events.
#[derive(Clone, Debug, PartialEq)]
pub enum LoopMode {
/// Specifies that the application is continuously looping at a consistent rate.
///
/// An application running in the **Rate** loop mode will behave as follows:
///
/// 1. Poll for and collect all pending user input. `event` is then called with all application
/// events that have occurred.
///
/// 2. `event` is called with an `Update` event.
///
/// 3. Check the time and sleep for the remainder of the `update_interval` then go to 1.
///
/// `view` is called at an arbitraty rate by the vulkan swapchain for each window. It uses
/// whatever the state of the user's model happens to be at that moment in time.
Rate {
/// The minimum interval between emitted updates.
update_interval: Duration,
},
/// Waits for user input events to occur before calling `event` with an `Update` event.
///
/// This is particularly useful for low-energy GUIs that only need to update when some sort of
/// input has occurred. The benefit of using this mode is that you don't waste CPU cycles
/// looping or updating when you know nothing is changing in your model or view.
Wait {
/// The number of `update`s (and in turn `view` calls per window) that should occur since
/// the application last received a non-`Update` event.
updates_following_event: usize,
/// The minimum interval between emitted updates.
update_interval: Duration,
},
/// Synchronises `Update` events with requests for a new image by the swapchain for each
/// window in order to achieve minimal latency between the state of the model and what is
/// displayed on screen. This mode should be particularly useful for interactive applications
/// and games where minimal latency between user input and the display image is essential.
///
/// The result of using this loop mode is similar to using vsync in traditional applications.
/// E.g. if you have one window running on a monitor with a 60hz refresh rate, your update will
/// get called at a fairly consistent interval that is close to 60 times per second.
///
/// It is worth noting that, in the case that you have more than one window and they are
/// situated on different displays with different refresh rates, `update` will almost certainly
/// not be called at a consistent interval. Instead, it will be called as often as necessary -
/// if it has been longer than `minimum_latency_interval` or if some user input was received
/// since the last `Update`. That said, each `Update` event contains the duration since the
/// last `Update` occurred, so as long as all time-based state (like animations or physics
/// simulations) are driven by this, the `update` interval consistency should not cause issues.
///
/// ### The Swapchain
///
/// The purpose of the swapchain for each window is to synchronise the presentation of images
/// (calls to `view` in nannou) with the refresh rate of the screen. *You can learn more about
/// the swap chain
/// [here](https://vulkan-tutorial.com/Drawing_a_triangle/Presentation/Swap_chain).*
RefreshSync {
/// The minimum amount of latency that is allowed to occur between the moment at which
/// `event` was last called with an `Update` and the moment at which `view` is called by
/// a window's swapchain.
minimum_latency_interval: Duration,
/// The windows to which `Update` events should be synchronised.
///
/// If this is `Some`, an `Update` will only occur for those windows that are contained
/// within this set. This is particularly useful if you only want to synchronise your
/// updates with one or more "main" windows and you don't mind so much about the latency
/// for the rest.
///
/// If this is `None` (the default case), `Update` events will be synchronised with *all*
/// windows.
windows: Option<HashSet<window::Id>>,
},
} |
In an attempt to better understand the First I tried testing the time between frames on average. I tested using two different
This implies that I also found that the 16ms wait does occur during the call to |
So far I've been thinking that the only approach to ensuring that the I just remembered that |
The
|
This is some epic work mitch maestro! Getting excited for all this Vulkan goodness. |
OK, so although we don't need a thread to avoid blocking on If we are using only the main thread, how do we check whether a window's swapchain is ready for an image without continuously looping? If we sleep for even a short amount of time, we may miss the moment at which a swapchain becomes ready. Further, how do we check the window swapchains in a way that gives even priority? E.g. if while we are iterating, one window further down the list became ready before one earlier in the list, we have no way to know this and will end up prioritising the window that is earlier in the list. Threading Swapchain Image SelectionThis brings me back to thinking that it may be necessary to use a thread per window (whether a green thread or a native thread). I can think of two options:
While the native thread option would probably be fine for a couple of windows, using tokio's work-stealing runtime will probably be a safer, more scalable way of distributing the work. State SynchronisationWhile the solution appears to be threading the next-image-selection process, this also introduces some issues with synchronising the necessary state.
Further, the loop mode may change at any moment in time and it is worth considering how this would affect interaction between the main thread and the image selection thread. The run loop is already quite complex and I can imagine threading the image selection process won't improve things, so I'm keen to properly think this out to ensure there aren't any better options before going ahead with this. |
Main-thread Image SelectionThe previously mentioned issues with selecting the next image to process on the main thread are:
Possible solutions
Does syncing with refresh rate require blocking?All that said, I just realised that although we can now avoid blocking on |
The meaning of
|
Ok, think I've got most of the system laid out now! Currently just trying to start with a basic Currently attempting to clear the frame via
Seems to be a bug in vulkano: |
This switches the windowed-graphics API from OpenGL (via glium) to Vulkan (via vulkano). You can read more about the motivations, thoughts and discussion behind this switch at nannou-org#208 and nannou-org#108. There are still a few items left to complete before this PR is ready: - [ ] Fix bug where view seems to be vertically flipped (probably in vertex mapping process). - [ ] Fix bug where clearing an image that is then multisampled does not work. vulkano-rs/vulkano#1123 - [ ] Add depth attachment to draw renderpass. - [ ] Fix bug where alpha channels always seem opaque. - [ ] Update old loop modes for changes in the application loop. - [ ] Merge conrod vulkan backend and switch to it. - [ ] Merge and publish vulkano-rs/vulkano#1117 or related fix so we can switch to crates.io dep. Closes nannou-org#208. Closes nannou-org#108.
It's been a long journey, but all these boxes are ticked! There are still a few small bug fixes and some macos build automation to land but they are mostly upstream issues. Otherwise, v0.9 should be just about Vulkan ready! |
This switches the windowed-graphics API from OpenGL (via glium) to Vulkan (via vulkano). You can read more about the motivations, thoughts and discussion behind this switch at #208 and #108. There are still a few items left to complete before this PR is ready: - [ ] Fix bug where view seems to be vertically flipped (probably in vertex mapping process). - [ ] Fix bug where clearing an image that is then multisampled does not work. vulkano-rs/vulkano#1123 - [ ] Add depth attachment to draw renderpass. - [ ] Fix bug where alpha channels always seem opaque. - [ ] Update old loop modes for changes in the application loop. - [ ] Merge conrod vulkan backend and switch to it. - [ ] Merge and publish vulkano-rs/vulkano#1117 or related fix so we can switch to crates.io dep. Closes #208. Closes #108.
This switches the windowed-graphics API from OpenGL (via glium) to Vulkan (via vulkano). You can read more about the motivations, thoughts and discussion behind this switch at #208 and #108. There are still a few items left to complete before this PR is ready: - [ ] Fix bug where view seems to be vertically flipped (probably in vertex mapping process). - [ ] Fix bug where clearing an image that is then multisampled does not work. vulkano-rs/vulkano#1123 - [ ] Add depth attachment to draw renderpass. - [ ] Fix bug where alpha channels always seem opaque. - [ ] Update old loop modes for changes in the application loop. - [ ] Merge conrod vulkan backend and switch to it. - [ ] Merge and publish vulkano-rs/vulkano#1117 or related fix so we can switch to crates.io dep. Closes #208. Closes #108.
This is a follow up to #108 with some actionable steps!
The general plan is to depend on vulkano for its type-safe API and switch out every that we currently use glium for it.
This Vulkan Tutorial should be a useful reference.
vulkano::swapchain::Surface
methods fromWindow
.LoopMode
that onlyupdate
s when the next image is ready for a specific window.Should consider making this a builder method on a window so that a unique update may be used for each if desired?Can't do this as i would require adding type params to theApp
.Frame
type usingswapchain::acquire_next_image
future.LoopMode::RefreshSync
run loop working with a cleared image.LoopMode
s.LoopMode
switch.draw
backend to replace the glium one. Refer to triangle example for handling buffers.The text was updated successfully, but these errors were encountered: