Add ability to drive full-body avatars using OpenXRHand

[Malcolmnixon]

This PR allows the OpenXRHand to drive:

• OpenXR rigged hand skeletons located under the OpenXRHand node
• Godot Humanoid rigged hand skeletons located under the OpenXRHand node
• OpenXR rigged avatar skeletons located separately in the scene-tree
• Godot Humanoid avatar skeletons located separately in the scene-tree

The change to support both OpenXR and Godots Humanoid skeletons is achieved by a new Skeleton Rig enum property allowing the developer to select "OpenXR" (default) and "Humanoid".

OpenXR provides bones oriented as defined in the OpenXR specification with Z+ pointing back along the bones to the wrist, and Y+ pointing out the back of the hand. The Godot "SkeletonProfileHumanoid" skeleton uses bones oriented with Y- pointing back along the bones to the wrist, and Z- pointing out the back of the hand. The correction is performed by multiplying the incoming OpenXR quaternion with a fixed [0, $-\sqrt{1/2} $, $\sqrt{1/2}$, 0] quaternion.

The change to allow driving avatars involves correctly handling when and how bones are positioned:

Condition	Control
OpenXR joint has no associated skeletal-bone	No bone driven
OpenXR joint refers to skeletal-bone Skeletal-bone has no parent-bone	Bone is driven relative to OpenXR root palm bone
OpenXR joint refers to skeletal-bone Skeletal-bone has parent-bone Parent-bone has no associated OpenXR joint	Bone is a child of larger skeleton skip driving so code/IK can position via skeleton
OpenXR joint refers to skeletal-bone Skeletal-bone has parent-bone Parent-bone driven by OpenXR joint	Bone is driven relative to parent OpenXR joint

Using these rules the OpenXRHand is capable of driving hand skeletons that start at "Palm", "Wrist", or even individual fingers, with the "root" bones being correctly driven to the OpenXR tracked positions. Additionally the OpenXRHand is capable of driving hands of avatars, where the Palm/Wrist/Hand has a parent Radius/Ulna/Lower-Arm bone not driven/known by OpenXR by skipping the Palm/Wrist/Hand bone (driven by the SkeletonIK3D or code) and instead just driving the finger bones.

The following video demonstrates the results. Node that some models aren't perfect as I'm not overly proficient at blender and bone-weighting.
https://github.com/godotengine/godot/assets/1863707/4c1f6f95-07a9-41ec-8c97-a033dd74e4c2

A demo project exists at https://github.com/Malcolmnixon/OpenXRHandTestSuite which was used to create the video.

[fire]

If you wish, I can merge the tests on https://github.com/godotengine/godot-tests.

[BastiaanOlij]

Thanks @Malcolmnixon , all looks perfect, just some nitpicking things on my end to future proof it a bit. I think the description for the docs could do some work to be more clear of the change we're doing.

All in all this is a great improvement, thanks for all the work you've done looking into the skeleton and avatar logic!

[Malcolmnixon]

If you wish, I can merge the tests on https://github.com/godotengine/godot-tests.

I have concerns with including the current rigged meshes into the official godot test suites. They're quite large (37MB) and I had to edit some of the rigs in blender to add/remove/rotate bones to match the OpenXR or Humanoid standards, but my blender-fu is weak and I'm not practiced in the art of weight-painting so the meshes don't bend very convincingly.

I'll look around for smaller low-poly alternatives which will be more suitable for inclusion in a test suite without burdening the repo.

[TokageItLab]

I have not checked it deeply, but as far as I can read in the overview, I think it is fine way.

In order to neatly apply motion to the bones of a 3D model's hand, it is important to assign roles to the XYZ axes of the bones of the fingers of the model's hand.

In other words, if you are applying an animation that has a role for the XYZ axis of the bones, such as Bend or Roll, you should define the role of the XYZ axis on the model side as well.

GodotSkeletonHumanoid has the following roles defined for the XYZ axes.

• Directs the +Y axis from the parent joint to the child joint
• +X rotation bends the joint like a muscle contracting

https://docs.godotengine.org/en/latest/tutorials/assets_pipeline/retargeting_3d_skeletons.html#rest-fixer

So, as long as the roles are defined on the OpenXR Hand's XYZ axes, they should be perfectly convertible by a 90 degree * n rotation as the description mentioned.

At this time, finger bones, unlike body bones, do not require retargeting like Rest overrides.

For example, if an athlete's idle pose is applied to a crooked old man without a Rest override, the old man's spines are suddenly straightened and the model's physique/identity is lost.

However, the fingers should have no concept of physique/identity, unlike the spines, so it should look best if the animation is transferred without such retargeting, only converting the role of the axes.

FYI, I have written a Japanese article that summarizes this theme before https://qiita.com/TokageItLab/items/aa6c113dcf9e713f0f74#%E6%8C%87%E3%81%AB%E3%81%A4%E3%81%84%E3%81%A6%E3%81%AF-absoluteanimation-%E3%81%8C%E4%BD%BF%E3%81%88%E3%82%8B%E3%81%AE%E3%81%A7%E3%81%AF%E3%81%AA%E3%81%84%E3%81%8B

See also https://github.com/TokageItLab/realtime_retarget?tab=readme-ov-file#how-to-use

[DigitalN8m4r3]

tested with character creator 4 and mixamo avatar models.
testing went well, no issues encountered.
thx @Malcolmnixon for the work on this!

[akien-mga]

Thanks!