Import animations in bone space (no longer relative to bone rest)

[reduz]

Describe the project you are working on

Godot

Describe the problem or limitation you are having in your project

Godot since its creation loads animations by computing the Rest pose of the skeleton (generally the T-Pose), and
then converting animation transforms for each bone as relative to the local bone rests.

This approach has some advantages:

• Reuse of animations in slightly different models is possible.
• In animation blending, additive blending works relatively well (used generally to make some parts of a character turn while not affecting the current animation).

Still, it also introduces disadvantages:

• Non uniform scaling in animations does not properly work with this approach, because this information is lost when converting to quaternion/scale relative to the rest pose.
• Obtaining a proper Rest pose is not always possible in animations that come from 3DS Max or Maya.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

To overcome the problems described, the idea is to move to animations storing transforms relative to the bone (no longer relative to the rest).

To do this, several things need to be modified in the import process.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The proposed solution consists on the following changes to how animations are imported and processed:

• Animations would no longer import relative to the rest, they would contain a bone pose to override the one in the skeleton.
• For additive animations, these should be marked as such during import, and those will be made relative to the rest. This is now possible thanks to the advanced import options, which allow per animation options.
• As a Rest animation is still needed for some things, so we can continue trying to generate it as always, or we can allow the user to supply one. This could be customizable in the scene import settings as an option called "Rest computation mode": [Bind Matrices / Default Pose / Custom Animation]. For the later you supply the name and the default is REST.
• For animation reuse among different models, we can use an alternative approach. Animation tracks can be marked as not containing translation, and this translation is obtained from the bone rest. When animation player process bones, they can simply request and cache the bone rest origin and use it as translation during animation processing. Operating this way should be an import option that is turned off by default.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Animation is core.

Is there a reason why this should be core and not an add-on in the asset library?

Animation is core.

[michael-nischt]

Disclaimer, I haven't worked with Godot much. But I tend to prefer relative joint animation to a neutral/rest pose of the skeleton.

Hope you don't mind a few thoughts / questions. Please ignore if there are not relevant or failed to understand things in my quick read.

Non-relative bone animations are larger.

Most bones have a fixed length. Hence relative joint animations don't need the translation / position channels. Scaling is a rare case too. Usually bones have scale 1 and rarely animate it. Scaling prop joints for items is an notable exception. E.g. scale zero to hide objects. But that is for very few bones only.

The vast default is that relative joint animations only have a rotation.

One hybrid solution is to use relative compression. With the first key in bone space and the rest relative to it. That for most parts should solve the size problem. But hybrid solution often add much complexity.

Relative Scaling is weird.

Non uniform scaling in animations does not properly work with this approach, because this information is lost when converting to quaternion/scale relative to the rest pose.

I'm not 100% sure on this. Tbh relative scale is super unintuitive to me. Compared to translation and rotation. But I never had the problem you mentioned. Do you mind elaborating what gets lost when applying relative non-uniform scale?

Is this also the case for relative matrices?

I often compute the relative matrix of the current frame to rest posse and decompose it into T, R. S.
(e.g as in this blender exporter).
Multiplying the rest joint matrix with TRS matrix constructed from the animation sample at runtime.

I suppose the problem is if you can't use matrices but need T,R,S components again?

Sadly, matrix de-construction is a bit tricky for animations which are non-uniformly sampled. Possible but a disadvantage.

Non-existing Rest-Pose

Obtaining a proper Rest pose is not always possible in animations that come from 3DS Max or Maya.

That's interesting. I remember Max & Max having explicit rest / binding poses. But don't know that Blender has that concept. Hence, I used to define the current (or first) frame in Blender as the "neutral" (rest) pose.

Anyway, what I'm asking is what the exact disadvantage here is?

The biggest problem I can see are animations split across files. Each needs to contain the (same) rest pose. To compute the relative transformation for the export. Is that it?

It is an annoyance if you have hundreds of animations. Then you want more than one Blender/Maya/Max file to avoid merge conflicts.

[reduz]

@michael-nischt In an ideal world you get everything you need, but unfortunately it is not the case when you export from Max and Maya. All you get is IBMs for the bones affecting a mesh (which means not even all of them have it, if a bone has no effect on a mesh, the exporters omit the IBM in the skin) and they can differ from object to object. Blender is vastly superior in this aspect because it has the actual concept of a skeleton and a rest pose for it. Likewise, you can get files that have messed up scaling so non-uniform animations will not always work.

I do agree that storing animations local to a rest is the superior approach, but unfortunately the type of content you receive from DCCs goes against using this approach if you want to achieve maximum compatibility.

Blender spoiled us too much in this regard because it uses a far superior approach to skinning than Autodesk products, but reality is hard when you look beyond it.

[michael-nischt]

@reduz All good. It seems you are set on the change already due to the compatibly issues. I understand that this is the highest priority. Rightly so I guess!

As mentioned just a few notes on a chance anything might be helpful.

Admittedly I don't really understand the non-uniform scaling issues. E.g. if those are more related to formats or specific exporters, .. than the general concept. But doesn't matter if it's causing headaches and will go away.. 🙂

[reduz]

@michael-nischt The problem with non uniform scaling is that sometimes you get terrible models with bones scaled in a way where skew is generated. If you use TRS normally, it does not matter so much because its just a bone upper in the hierarchy with a weird scale, but if you create a rest matrix, then the rotation relative to it needs to have skew to compensate, which is not possible with TRS.

Again you get only this is very terrible non-production quality models, but given Godot is such a general purpose game engine, it needs to prioritize compatibility above all else. If a file does not open properly, it's a bug no matter how much this file sucks.

[michael-nischt]

@reduz Hm, trying wrap my head around this: The skew generated by non-uniform scale across bones shouldn't be a problem. Since the animation is relative to the local bone space. But I remember Maya has a pre & post transform per joint. This could generate a skew after combing all 3 to the local bone space if 2 have a non-uniform scale. And cause what you describe.. but then getting rid if relative animations doesn't solve the issue, right?

I agree 100% that compatibility has the highest priority for a general purpose game engine like Godot. Sorry for derailing the proposal with my notes / thoughts.

[avedar]

Lowering the quality of the workflow and output for everyone just to enable usage of "very terrible non-production quality models" sounds very extreme to me. Giving some kind of warning/fix-it message for models that have those properties sounds like a better solution. If this were also just an import option that would make more sense as well.

[reduz]

@avedar I think you are misunderstanding this. This is mostly about making the workflow less Blender centric. Maya and Max work different and we have very poor support for this. The comment I made about non-production quality models is related to non uniform scaling, which gets fixed as a side effect of this change, but the main aim of this is to improve compatibility with scenes exported from Autodesk products.

[avedar]

It's possible I'm misunderstanding but I know that Maya/Max are more than capable of producing models that operate within industry standard workflows. So I don't see why they would require a novel approach if the goal is for compatibility and not to just allow garbage input/output - which just leads to poor user experience in the long run.

[reduz]

@avedar You are 100% and fully misunderstanding here, forget about what you read on bad quality models as this was not related to the main issue discussed in here. Improved compatibility with crap models is a side effect of this proposal, the real reason this proposal is needed is because of proper compatibility with Autodesk products, which can not be realized with the approach described by @michael-nischt (which is pretty much how Godot works right now) due to how skins work in those products.

The real issue is that, to do rest-related animation you need a rest matrix to begin with, but this is not ensured to be available with models exported from Autodesk products because they don't use the concept of "rest". Only Blender does. Autodesk products use something called IBM (Inverse Bind Matrix) which generally only exists per model and only for the vertices affected by a skeleton, and can even be different per model for the same bone. Expecting a rest matrix to exist like in Blender is something Godot currently does, which should not do and, as a result, compatibility with Autodesk products is currently low.

[reduz]

Closing, as this was implemented by godotengine/godot#53765