Runtime VertexAttributes

[Elzair]

Can you create a GraphicsPipeline or a Buffer when you only know the vertex attributes at runtime? I see there is an example for loading shaders at runtime, but the Vertex data is still specified in the code.

Say someone wants to write a viewer for GLTF 2.0 scenes. The spec says meshes can have any of eight different vertex attributes (POSITION, NORMAL, TANGENT, TEXCOORD_0, TEXCOORD_1, COLOR_0, JOINTS_0, WEIGHTS_0) in any order (for now, let's assume they are all interleaved in one buffer). Accounting for all arrangements would require defining floor(8! * e) - 1 = 109,600 structs. There HAS to be a better way to do this!

[tomaka]

For the graphics pipeline it is possible by writing your own implementation of the VertexDefinition trait. This trait is a bit crappy though.

For the buffer, if everything is a float you can simply create a buffer of type [f32]. Otherwise I guess you would have to use [u8] or [u32], and either transmutes or the to_bits functions that will be stable in Rust 1.20. This is not really something that is specific to vulkano I guess. If you want to create a box whose content type is only known at runtime, you have the same problem.

Also note that for the moment the content of the vertex buffer is not checked. In other words you can use any buffer of any content in combination with any vertex definition and will not get an error. That eliminates a potential problem.

[tomaka]

Slightly off-topic but I wanted to write a glTF loader as well, so I'll probably encounter the same problems.

[trevex]

Let me piggy back this issue:

I was looking for a nice way to leverage runtime vertex attributes as well.
While I am new to Rust and Vulkan(o), I believe Vulkano changed a lot since the inception of this issue.

When I tried to implement runtime vertex attributes, I ended up with a Mesh structure roughly as follows:

#[derive(Hash, Eq, PartialEq, Debug)]
pub struct VertexAttribute {
    pub name: Cow<'static, str>,
    pub format: Format,
}

pub struct Mesh {
    attributes: HashMap<VertexAttribute, Arc<dyn BufferAccess>>,
    length: u64,
}

This is similar to how other higher level graphics APIs I know handled it, basically attributes are declared with their name and format, e.g.:

const ATTRIBUTE_POSITION: VertexAttribute = VertexAttribute::new("position", Format::R32G32_SFLOAT);
const ATTRIBUTE_COLOR: VertexAttribute = VertexAttribute::new("color", Format::R32G32B32_SFLOAT);

And can then be used to construct a mesh using a builder pattern, something akin to this:

let mesh = MeshBuilder::new(&memory_allocator)
        .add(
            ATTRIBUTE_POSITION,
            vec![
                Vec2::new(-0.5, -0.25),
                Vec2::new(0.0, 0.5),
                Vec2::new(0.25, -0.1),
            ],
        )
        .build();

As only the BufferAccess trait is required in the HashMap, a builder can theoretically take care of all the different allocation/transfer needs and an additional trait on Vec<Vec2> can ensure that the format is compatible with the given attribute.

As we have non-interleaved buffers, we can now use a "generalized" VertexDefintion, such as:

pub struct NonInterleavedInput {}

impl NonInterleavedInput {
    pub fn new() -> Self {
        Self {}
    }
    // TODO: allow marking of bindings as per-instance
}

unsafe impl VertexDefinition for NonInterleavedInput {
    #[inline]
    fn definition(
        &self,
        interface: &ShaderInterface,
    ) -> Result<VertexInputState, IncompatibleVertexDefinitionError> {
        let mut attributes: Vec<(u32, VertexInputAttributeDescription)> = Vec::new();
        let mut bindings: HashMap<u32, VertexInputBindingDescription> = HashMap::new();

        for element in interface.elements() {
            tracing::debug!("creating binding and attribute for element: {:?}", element,);
            bindings.insert(
                element.location,
                VertexInputBindingDescription {
                    stride: element.ty.num_components * element.ty.num_elements * 4,
                    input_rate: VertexInputRate::Vertex,
                },
            );
            attributes.push((
                element.location,
                VertexInputAttributeDescription {
                    binding: element.location,
                    format: to_format(&element.ty), // to_format helper function left out for brevity
                    offset: 0_u32,
                },
            ));
        }

        tracing::debug!(
            "resulting vertex input state has bindings: {:?} and attributes: {:?}",
            bindings,
            attributes
        );
        Ok(VertexInputState::new()
            .bindings(bindings)
            .attributes(attributes))
    }
}

However we require the ShaderInterface when binding the vertex buffer (or "specializing" them before). An excerpt of my draft implementation looks as follows:

impl Mesh {
    pub fn buffers(&self, interface: &ShaderInterface) -> Vec<Arc<dyn BufferAccess>> {
        let bufs_len = interface.elements().len();
        let mut bufs: Vec<Option<Arc<dyn BufferAccess>>> = Vec::with_capacity(bufs_len);
        bufs.resize(bufs_len, None);

        for element in interface.elements() {
            let name = element.name.as_ref().unwrap();
            if let Some(buf) = self.attributes.get(&VertexAttribute {
                name: Cow::Owned(name.clone().into_owned()),
                format: to_format(&element.ty),
            }) {
                bufs[element.location as usize] = Some(buf.clone());
            }
        }

        bufs.iter()
            .map(|maybe_buf| maybe_buf.clone().unwrap())
            .collect() // TODO: check if all required bindings are available
    }

    pub fn len(&self) -> u64 {
        self.length
    }
}

It can be used in a pipeline as follows:

    .bind_pipeline_graphics(pipeline.clone())
    .bind_vertex_buffers(0, mesh.buffers(vs.entry_point().input_interface()))
    .draw(mesh.len() as u32, 1, 0, 0)

The above approach is quite hacky and I am not certain about the safety, so some feedback and review of the above approach would be highly appreciated.

This leads me to a second point, that would make the above nicer to implement. Currently there is no way to retrieve the ShaderInterface from a GraphicsPipeline. So for the above implementation the interface has to be retrieved from the shader entrypoint separately to the pipeline.
While browsing the documentation and code to find a way to get the required information from the pipeline I stumbled upon the following TODO:
https://github.com/vulkano-rs/vulkano/blob/master/vulkano/src/pipeline/graphics/mod.rs#L107
https://github.com/vulkano-rs/vulkano/blob/master/vulkano/src/pipeline/graphics/mod.rs#L164

I would propose to store the shader's EntryPoint in the HashMap to be able to retrieve shader information at a later date.
An alternative would be to create a "ShaderDescriptor"-struct which tracks input and output ShaderInterfaces.
As this sounds like a straightforward first contribution, I could create a PR for this.

P.S. any feedback welcome :)