New Feature Development RoadMap for future VulkanSceneGraph-1.2 release and beyond

[robertosfield]

I have put down my thoughts on maintenance of the VulkanSceneGraph-1.0 series in Discussion thread #596. and in this thread will put down my thoughts on what would be useful to tackle for new feature development in the VulkanSceneGraph and companion projects. Feel free to add in your thoughts on what would be useful, but please be prepared to roll your sleeves up and help develop those feature, or if you work for a company that needs these features perhaps funding some open source development work, by myself or others, might be a good way to make things happen.

In no order, these as my possible items that I think would be useful to be tackled for 1.2 or perhaps later:

VulkanSceneGraph

1. Support for shadows and environment maps in the Phong and PBR ShaderSets, and helpers functions/classes for creating scene graphs that render shadow maps.
2. Character animation/skinning in standard.vert shaders used by Phong and ShaderSets?
3. Rewrite RayTracing classes to modernize them and bring them more inline with other core VSG classes.
4. Rewrite MeshShader to use Khronos Mesh Shaders functions.

osg2vsg

1. Complete rewrite of the osg2vsg library to use modern VSG capabilities like ShaderSet/GraphicsPipelineConfiguration.
2. Development of ShaderSet that mirrors OpenGL fixed function pipeline.
3. Implementation of OSG/VSG interfaces that use OpenGL/Vulkan extensions for passing buffers/images/frame buffers between OpenGL/OSG and Vulkan/VulkanSceneGraph application. Some experimental work was done on this a few years back but we didn't spend long enough on it to figure out all the details of how to get things working cross platform.
4. Make osg2vsg-1.0 release.

vsgXhange

1. Add a range of optimization utilities for removing redundant nodes, combining/flattening static transforms, inserting cull nodes. Note, currently vsgXchange::assimp doesn't do any optimization, and doesn't insert any CullNode/CullGroup so user applications currently have to do this - which is fine for power users, but for other users they might assume all this is done for them already.
2. Get the compressed ktx image features working.
3. Develop dedicated loaders for common formats if Assimp doesn't load them, or doesn't map the features as well as if we could map them directly rather than via the Assimp graph.

vsgImGui

1. Update to the latest ImGui release.
2. Review/Merge the ImPlot submission.
3. Make 1.0.0 release

vsgQt

1. Fix handling of key mapping
2. Make 1.0.0. release

vsgUnity

1. Find a engineer to take over management of this repo as I don't have much knowledge of Unity, or any projects that require it.
2. Make 1.0.0 release?

vsgPoints

1. Further features dependent on user base needs.
2. Make 1.0.0 release?

vsgExamples

1. Review all examples with a view to tailoring them more for educational purpose rather than just test beds for feature development as they have been up to VSG-1.0.
2. Create a scheme for testing examples in some form of standard unit test. This might be best done as a entirely separate Vulkan/VSG unit test scheme, that happens to use vsgExamples within it's set of tests.

Treat all these as ideas of what I/others can potentially tackle in the next 3-6 months. I'm sure the community will have ideas of what they'd like to see and help develop, these might be within the umbrella of things that I'm personally managing but also could be companion projects that are managed entirely separately.

As discussions progress I aim to distill out an update of the VulkanSceneGrah/ROADMAP.md.

[robertosfield]

Other items that would be useful to have:

VulkansSceneGraph

5. Acceleration structures for CPU based geometry operations i.e. KdTree or similar to speed up intersection testing etc.
6. Multiview extension implementation

[timoore]

Standardizing lighting and material models within core VSG would be great. I guess shaders that are bundled with vsgXchange have become the de facto standard shaders in VSG, but this is poorly documented, not to mention included in an obfuscated way right now. I would like to see (and maybe even implement) a standard PBR rendering mode with some sort of global illumination support, and tone mapping of the result. The community may not be ready for this to be the default, so an OpenGL-like backup is probably necessary too. I don't know how this interacts with model loading, where shaders should live in the scene graph, where descriptor sets should be bound, etc.

[robertosfield]

On Sun, 20 Nov 2022 at 15:20, timoore ***@***.***> wrote: Standardizing lighting and material models within core VSG would be great. I guess shaders that are bundled with vsgXchange have become the de facto standard shaders in VSG, but this is poorly documented, not to mention included in an obfuscated way right now.

Much of the work I did this Autumn was standardizing the lighting and shaders using ShaderSet as the framework for this. vsgXchange no longer provides any shaders, they are all provided by the core VSG with the FlatShader, Phong and PBR ShaderSets. I wrote this work uip as it progressed. https://github.com/vsg-dev/VulkanSceneGraph/blob/master/include/vsg/utils/ShaderSet.h#L137 I would like to have resolved the disparate material structs used for osg2vsg shaders, Phong and PBR so there would be one material type but they were a bit too different to easily merge so can to delay this refactor. The osg2vsg rewrite will likely help refine exactly what will need to be in a unified material struct if it's possible.

I would like to see (and maybe even implement) a standard PBR rendering mode with some sort of global illumination support, and tone mapping of the result. The community may not be ready for this to be the default, so an OpenGL-like backup is probably necessary too. I don't know how this interacts with model loading, where shaders should live in the scene graph, where descriptor sets should be bound, etc.

The vsgXchange::Assimp uses the PBR ShaderSet for all the models I've used in testing, in theory it has a code path that uses the Phong ShaderSet but so ar haven't come across a Assimp material setting that selects it over PBR. Expanding/refining the PBR shaders/ShaderSet is something I have in mind for 1.2. The vsg::Options class has a shaderSets member that is a std::mapt[std::string,. ref_ptr<ShaderSet>> for the purpose of overriding the default ShaderSet used by the loaders, so if you want to override the PBR shader used by the vsgXhcnage::assimp loader you do something like: auto options = vsg::Options::create(); options->add(vsgXchange::all); options->shaderSet["pbr"] = myCustomPbrShaderSet; auto model = vsg::read("model.gltf", options); This allows you to use entirely different shaders or simply different defaults for the various elements of the graphics pipeline settings. The vsgtext example illustrates this in action when customizing the settings in setting up text by creating a text ShaderSet then changing the defaults. https://github.com/vsg-dev/vsgExamples/blob/master/examples/text/vsgtext/vsgtext.cpp#L202 All of these features I've written up extensively over the past years on vsg-users. It's not formal documentation but as the features were evolving as I figured out how to design/implement them. Now that 1.0.0 is out and the API is more set, documentation becomes more achievable. It would be good for companies wanting to use the VSG to help fund this effort as I can't indefinitely pour my time into the "greater good" - it's already self funded myself for 3/4'rs of the past year getting VSG to 1.0.0. Cheers, Robert. Message ID: <vsg-dev/VulkanSceneGraph/repo-discussions/600/comments/4188827@

…

github.com>

[robertosfield]

In response to discussion thread "PBR Specular missing" #608 I'm adding another feature item for VulkanSceneGraph/vsgXchange:

6. Add enviromental texture support to built in PBR ShaderSet and vsgXchange for environment textures.

[vsg-dev]

One of the items I've had on my own personal wish list, but missed adding it to the wish list is noise. The OSG has perlin noise, but there are others out there such as simplex and worley noise, and now another contender - octavia noise:

https://github.com/lorenSchmidt/fractal_cell_noise

Adding to the growing list for VulkanSceneGraph:

1. Support for shadows and environment maps in the Phong and PBR ShaderSets, and helpers functions/classes for creating scene graphs that render shadow maps.
2. Character animation/skinning in standard.vert shaders used by Phong and ShaderSets?
3. Rewrite RayTracing classes to modernize them and bring them more inline with other core VSG classes.
4. Rewrite MeshShader to use Khronos Mesh Shaders functions.
5. Acceleration structures for CPU based geometry operations i.e. KdTree or similar to speed up intersection testing etc.
6. Multiview extension implementation
7. Noise function, CPU or GPU or both.