Vulkan design tracking issue

[tomaka]

The general plan concerning Vulkan and drawing in general is:

• Applications can use the Vulkan API to draw windows through a vulkan interface.
• This vulkan interface consists of the serialized calls to Vulkan.
• In order for an application to create a Vulkan surface, the solution for now is to use another interface named window. One of its functions allows creating a Vulkan surface from the created windows.
• The handler of the window and vulkan interfaces would be the windows manager.
• The windows manager would implement this by communicating with the video driver.
• The video driver or the host would implement the Vulkan API with the VK_KHR_display extension to creates surfaces covering the whole screen.

[tomaka]

One of blocks concerning Vulkan is how to handle vkMapMemory across process boundaries or across the VM boundary.

The most straight-forward way would be to:

• Remove the HOST_COHERENT flag from all memory flags.
• Override vkFlushMemoryedMappedRanges/vkInvalidateMemoryMappedRanges to also transfer the content of the invalidated ranges to the other process or to the outside of the VM.

This, however, poses a problem. The Vulkan API mentions that at least one memory type with the HOST_COHERENT flag must exist in the list. And this is probably not something we can just ignore, as it is very likely that a lot of programs simply enumerate the list of memories until they find this one.

Another solution could be to add a new type of memory representing "memory inside of this VM". However this means that we also have to implement creating buffers and images on top of this memory, plus implement creating command buffers that copy from that memory to elsewhere. It also raises the question of what to do if the user wants to copy from that "memory inside of this VM" to a device-local-only memory.
This solution requires too much code to be written.

The next solution I can think of is to make the memory coherent. That is, to automatically detect changes that are made to the "mapped" memory locations and propagate them to the other process or outside of the VM. This raises a lot of other questions though, and can potentially make the design way more complicated. If this solution is chosen, I would lean towards having a "mem-map" interface (that's always implemented by the kernel) with messages that explicitly ask to flush/invalidate memory.

[tomaka]

Pragmatically speaking, the Vulkan/windowing roadmap would be:

• Make the vulkan-triangle example work (which requires resolving the vkMapMemory issue above and finishing Opening a Window now yields a Vulkan surface #16 ).
• Expand it to be a proper example (that draws more than a triangle) and see if it still works.
• Write a program that handles the vulkan and window interfaces and that, as a start, redirects all calls to the host's handler. The vulkan-triangle would call into that program.
• Modify the program so that when it handles a swapchain creation message, instead draws to an image of its own and then draws the image itself. Might be more difficult than it sounds.
• Expand on that in order to draw the "swapchains" (i.e. images) of multiple processes next to each other. We now have a windows manager.
• Modify the host so that it provides the VK_KHR_display extension and remove its implementation of the window interface. New interfaces for the mouse and keyboard input need to be created as replacements.

[tomaka]

Thinking about this, I think we might have to implement the encoding/decoding code manually, rather than automatically generating it.

[tomaka]

Closing for #187

Vulkan is too much efforts to make work properly.