Shader compilation in bgfx

[q4a]

There are 3 options for shader compilation in bgfx:

1. Real time shader compiler
   Pros and cons:
   ++ This is already working in the bgfx branch.
   + No need to store binary shaders and recompile them when changing shaders code.
   + There will be a single place in the code for each bgfx::RendererType: https://bkaradzic.github.io/bgfx/bgfx.html#structbgfx_1_1_renderer_type
   +- We will use undocumented feature ?, described: Compile shader at runtime bkaradzic/bgfx#405
   - I'm a little afraid of whether there will be performance problems.

2. Pre-compilation of shaders into separate folders/files. A simple technique that is used in many of the bgfx examples.
   Pros and cons:
   + Definitely no performance issues.
   - Need to store binary shaders and recompile them when changing shaders code.
   - I don't like the idea of ​​having code like this (even in one place):
   https://github.com/bkaradzic/bgfx/blob/master/examples/common/bgfx_utils.cpp#L105

switch (bgfx::getRendererType() )
{
	case bgfx::RendererType::Noop:
	case bgfx::RendererType::Direct3D9:  shaderPath = "shaders/dx9/";   break;
	case bgfx::RendererType::Direct3D11:
	case bgfx::RendererType::Direct3D12: shaderPath = "shaders/dx11/";  break;
	case bgfx::RendererType::Metal:      shaderPath = "shaders/metal/"; break;
	case bgfx::RendererType::OpenGL:     shaderPath = "shaders/glsl/";  break;
	case bgfx::RendererType::OpenGLES:   shaderPath = "shaders/essl/";  break;
	case bgfx::RendererType::Vulkan:     shaderPath = "shaders/spirv/"; break;
	case bgfx::RendererType::WebGPU:     shaderPath = "shaders/spirv/"; break;

3. Pre-compilation of shaders into a single file. It is used less often, but there are few examples:

• Bad example, but it's real render: https://github.com/jpcy/ioq3-renderer-bgfx/blob/f14eee0491640b398cf1d2347079a3ba468af2a1/shaders_bin/Shader.cpp
• Better example: https://github.com/bkaradzic/bgfx/blob/master/examples/02-metaballs/metaballs.cpp#L530

#include <bgfx/embedded_shader.h>
#include "vs_metaballs.bin.h" // precompiled all in one shader bin header

static const bgfx::EmbeddedShader s_embeddedShaders[] =
{
	BGFX_EMBEDDED_SHADER(vs_metaballs),
	BGFX_EMBEDDED_SHADER(fs_metaballs),

	BGFX_EMBEDDED_SHADER_END()
};

//// ... some code here ...

bgfx::ShaderHandle vsh = bgfx::createEmbeddedShader(s_embeddedShaders, type, "vs_metaballs");

Pros and cons:
+ No performance issues.
+- No switch-case in our C++ code or separated separate folders/files for each bgfx::RendererType. There is still same separation, but already in bgfx header <bgfx/embedded_shader.h>, but it's not our area of responsibility.
- Need to store binary shaders and recompile them when changing shaders code.

============================

Anyway, I want that you also think and say which option do you like best? And what performance problems can you see in different options?

[np-c0mp1ete]

Thanks for bringing this up.
Usually games precompile shaders for release builds. In debug it's better to have them compiled at runtime to make shader development faster.
For us at this point I would prefer the runtime compilation. We can always optimize it in the future when bgfx is fully integrated.

[Hammie]

Consider the games using this engine are often heavily modded, the more things we can update at runtime, the better.

I do think that in time, it may be useful to have a caching mechanism so shaders only have to be compiled once (unless changed) to speed up startup times. But then I would also opt for a runtime caching system.