Add option to bake a mesh from blend shape mix

[smix8]

Adds option to bake a mesh from a blend shape mix.

Implements proposal godotengine/godot-proposals#24 and godotengine/godot-proposals#8122.

Pairs with #85018 to bake the animated mesh.

This option is especially useful for games with a lot of character customization that require the use of blend shapes to customize character models but do not require to change the blend shape weights constantly.

E.g. load and use the heavy mesh with all the blend shapes inside the character editor menu. When finished designing the character model bake the resulting mesh for gameplay and unload the heavy mesh with all the blend shapes.

This can help to avoid all the performance implications of having to load and update a heavy mesh with tons of blend shapes at runtime as you can unload the heavy mesh when no longer needed, e.g. when exiting a character editor.

Even if performance would be a negligible reason for this pr with more and more rendering optimizations in Godot 4 this pr also makes it possible to use the mesh or mesh data for other purposes in the engine that require a baked mesh. Currently if a model has blend shapes that changes the model boundaries drastically it becomes unusable for such purposes.

Production edit: closes godotengine/godot-roadmap#58

[fire]

<3 I'll try to review this as soon as I can.

[lyuma]

Seems like a good start. Reminds me of unity's SkinnedMeshRenderer.BakeMesh() API which is useful.

It would also be nice to have a similar function for baking a skinned mesh / skeleton. (this isn't a requirement to address now, just an observation / musing. feel free to ignore this)

It might be nice to have a version of this which passes in an existing mesh reference, rather than being forced to allocate a new one. Examples of uses for passing in a mesh include cases where a mesh is reused in many places in a scene, or in the above-mentioned skinned mesh case, (so that the skin can be more easily baked after blend shapes are baked, since it must be done in this order).

The only nuance for reusing an ArrayMesh is that it either needs to disallow baking a mesh to itself; or it needs to buffer the list of surfaces, surface names and materials in a LocalVector<Array> / LocalVector<Material>, then clear the surfaces, then add them back.

[smix8]

The only nuance for reusing an ArrayMesh is that it either needs to disallow baking a mesh to itself; or it needs to buffer the list of surfaces, surface names and materials in a LocalVector<Array> / LocalVector<Material>, then clear the surfaces, then add them back.

@lyuma Disallow is better as the LocalVector cache for the mesh arrays and materials seem to be unable to prevent mesh corruption. As soon as the clear_surfaces() function is called on the existing mesh the data breaks for whatever reason while it works just fine with other meshes. I am not feeling it to do a full data copy of the arrays to support such edge uses.

no longer relevant

[lyuma]

I apologize for the request changes review. I dismissed the review comment because the comment is no longer relevant. I'll try to avoid using that in the future.

Overall, the code looks reasonable. I think there are always concerns left and right about GPU memory copies, inefficient Mesh APIs, ArrayMesh vs ImporterMesh vs SurfaceTool. Either the state of Godot's mesh APIs is in dire need of rethinking, or we need to implement a way to have a CPU-backed ArrayMesh.

Mathematically the code is fine. And usability-wise, it does look like a function that games could find useful and could be a bit slow in terms of CPU perf to recreate this in GDScript, as long as the inherently slow design involving a GPU-to-CPU copy is documented.

I would be ok if this gets merged. However, I'm not involved enough with the rendering team to make that sort of call.

[akien-mga]

Approving to keep track of it, and it's a self-contained change.

But an assessment of the feature and implementation from @godotengine/rendering before merging would still be welcome.

[smix8]

Changed so that the baked mesh is always returned, and if an optional existing mesh param is provided it updates this mesh RID instead of creating a new mesh as this helps with updating instances that already use that mesh.

[akien-mga]

Thanks!

[RavinMaddHatter]

Does this actually work? I am trying to use it, but posed or animated body doesn't seem to generate a different mesh after calling in godot 4.3.

Looking for an example but the docs are bare on this one. even some sort of a unit test would be something i would like to take a peak at for how it is supposed to work.

[RavinMaddHatter]

Lookin at this in 4.4. Dev2 i dont see the mesh changing. Is there a test case or a test i can confirm before filing a bug?

[RavinMaddHatter]

I have tried many things I am using a model with and without animation. Minipulating the bones via the script. Alot of stuff i have yet to get a model that differs from the original model. I am happy to be wrong. but i cant, with the documentation i have get it to work. There is about a 80% chance i am doing something wrong, but that means the documentation is incomplete.

[fire]

Are you sure you're only baking blend shapes into meshes?

[RavinMaddHatter]

As sure as i can be without and example project that shows the function working. Here is a snip of the code i am using. The character has the bones posed earlier in the code. Body is a meshInstance3d with a skeleton. it is posed several different ways. character is a child of Collection, when i add the new meshInstance3d with the new mesh, it is in the default pose regardless of the pose character was in when calling the method. I have tried in all modes (mobile, compatibility, and Forward+) i have exported the mesh as an obj and it is the same. i have tweaked other aspects of the mesh and those carry (this isnt very useful), but not the skeleton manipulations.

character.get_node("Body").bake_mesh_from_current_blend_shape_mix(baked_char)
var mesh=MeshInstance3D.new()
collection.add_child(mesh)
mesh.set_mesh(baked_char)

In the game i can see posed vs unposed.

[fire]

#85018 is the pr that bakes the skeleton. This one bakes the blend shapes.

[RavinMaddHatter]

So the comment on that pr is correct. Ignore my comment here then

…

On Wed, Sep 11, 2024, 5:35 PM K. S. Ernest (iFire) Lee < ***@***.***> wrote: #85018 <#85018> is the pr that bakes the skeleton. This one bakes the blend shapes. — Reply to this email directly, view it on GitHub <#76725 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABBUC5FCKAL26HJDXUGDLRDZWDOVPAVCNFSM6AAAAAAXVKNRACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBVGAZDQNJZGE> . You are receiving this because you commented.Message ID: ***@***.***>