New and improved IK in Skeleton3D

[TwistedTwigleg]

This pull request lays the groundwork for the new and improved IK in Godot. This includes behind the scene changes, as well as the IK code itself.

• Note: See this branch for the verbose commit log for commits 1-7: https://github.com/TwistedTwigleg/godot/tree/GSOC_2020_Working_Branch_IK_VERBOSE_ARCHIVE
  The first 7 commits are squashed changes from the verbose log, bringing the commit count from 91 to just 7. Commits after commit 7 are new changes added on top of the changes in the verbose archive linked above.

Below is a list of everything that has been added, removed, or otherwise changed. The list is in no particular order.

---

Changes:

• Added a SkeletonModifier3D class
  • This class, which extends Resource, will be the basis of the new IK solvers in Godot, and potentially even the PhysicalBone3D. This class is a virtual class that is intended to be extended.
  • Because the SkeletonModifier3D class extends Resource, it should be able to be saved to a file. This should make it easier for imported skeletons.
  • All of the new IK solvers will use this class, allowing IK to be nicely managed by the Skeleton3D node.
  • The Skeleton3D class has new functions to allow for adding, removing, and changing SkeletonModifier3D resources.
  • The Skeleton3D has a single slider that controls the interpolation for the modifications attached to it. This could change in the future.
  • In theory, it should also be extendable through GDScript, however this is not required to use the SkeletonModification3D classes.
    • I can now confirm, this indeed does work! This should make it possible for Godot users to make custom SkeletonModification3Ds that work alongside the existing, built-in solutions.
  • SkeletonModification3D resources can be applied in the Editor, through newly exposed properties. The process should be easy and straightforward.
  • Ideally, all SkeletonModification3D resources are compatible with each other, allowing for configurable, fully-body IK solutions.
  • Has a clamp_angle function, taken from the 2D IK PR. This is used by CCDIK to constrain the angles. Potentially, it may be possible to use with other modifications.
  • The modifications can be configured to run in either _process or _physics_process. Some modifications have options that only are available in _physics_process.
    • This was previously a modification stack setting, but now its been moved to each individual modification. This allows for mixing and matching modification execution modes.

|

• Added a SkeletonModifierStack3D class
  • This class, which extends Resource, holds all of the SkeletonModification3D resources that are going to be applied to a Skeleton3D.
  • Contains the global settings, like the strength the modifications are applied at, whether all of the modification are enabled, etc.
  • Because this class extends Resource, it can be saved to a file! This allows for easily saving SkeletonModification3D data, which is especially important for imported Skeletons3D nodes.
  • When duplicated, the modification stack will be made unique and so will all of its modifications. This is so IK and modifications work with instanced scenes, as each modification stack needs to only execute on a single Skeleton3D node in order to work.
    • This is a limitation of using Resource as the base class for the modifications and modification stack. However, in game development projects using IK, you often want it to be unique to each instance anyway, unless you are looking to mirror the animation across a bunch of different meshes (a very niche use case).

|

• Added a SkeletonModifier3DLookAt class
  • The simplest of the bone modifiers but can be very helpful. This modifier rotates a bone to look at a target.
  • Because of its relatively simplicity, it has been extremely helpful for debugging and implementation purposes.
  • Has options for applying additional rotation, or ignoring rotation on certain axes.

|

• Added a SkeletonModification3DCCDIK.
  • Uses the CCDIK algorithm to solve the IK chain. CCDIK is ideal for joints that are generally more robotic or have relatively mechanical motion requirements.
  • The SkeletonModification3D_CCDIK operates as a chain of bones, rather than each individual modification being for a single bone. Each CCDIK joint in the chain, where each joint is a single bone in the Skeleton3D, has options that can be fully configured in the Godot editor.
  • Has fully functioning hinge-constraints, which have configurable angle settings. The constraints can also be inverted, allowing for full, 360 degree control over the hinge constraint and how it restricts the CCDIK joints.
  • Has an option to enable or disable high quality mode. This is enabled by default.
    • High quality mode iterates over each joint in the chain, solving it and all it's children joints before moving onto the next. This gives a nicer, more natural looking result, at the cost of being more computationally expensive. Because of this, it can be disabled to save performance, though the results may not look as good.
  • I've rewritten the solver three times! Yikes! I'm hoping, fingers crossed, that the current implementation stays stable and continues to work moving forward. CCDIK is kinda finicky to implement, but I think I've finally nailed it. The results look very similar to my GDScript-powered prototype now and the code is similar to the working 2D CCDIK solver.

|

• Added SkeletonModification3DFABRIK.
  • Uses the FABRIK algorithm to solve the IK chain. FABRIK is ideal for all sorts of situations, and is the same algorithm as the one used in the SkeletonIK node (though I implemented it slightly differently).
  • The joints in the FABRIK chain are easily configurable, with sane defaults to make getting FABRIK running quick and easy.
    • For all but the leaf/tip bones in the chain, the modification can auto calculate the bone length using the information provided from the Skeleton. For leaf/tip bones, the length can either be entered manually or a "tip" node (like a BoneAttachement3D) can be used to calculate the length.
    • The chain tolerance and max iterations are configurable from the editor, with defaults that resemble those seen in SkeletonIK. This allows for lowering the iterations and raising the threshold for projects that do not require precise IK solving, improving performance.
  • Has configurable magnet positions for all but the origin bone in the FABRIK chain.
  • The final joint in the chain can use the target's Basis rather than have FABRIK calculate it, similar to the SkeletonIK node.
  • Exposes settings for adjusting the roll (or twist) of the bones, since FABRIK does not take the existing bone roll/twist into account when solving. This allows the user to control the roll/twist used in the final result.

|

• Added SkeletonModification3DJiggle
  • Uses jiggle physics to move the joint relative to a target. Helpful for long hair, edges of clothing, etc. Is primarily used for complementary, dynamic animations.
  • Works in a chain, just like FABRIK and CCDIK. Has default options that are applied to each joint in the chain, though these can be overridden on a per joint basis.
  • Has physics collider/shape support! It will not rotate into physics objects.
    • Uses raycasting to check for colliders/shapes. Exposes a collision mask to allow for defining which colliders/shapes are taken into account.
  • Exposes settings for adjusting the roll (or twist) of the bones, since the Jiggle modifier does not take the existing bone roll/twist into account when solving. This allows the user to control the roll/twist used in the final result.

|

• Added SkeletonModification3DTwoBoneIK
  • A simpler, general more stable solver that only operates on two bones using the law of cosigns to solve.
  • Is great for arms, legs, and any other joint that is usually only composed of two joints.
  • Joint lengths can be auto calculated or manually inputted, making it easy to get it setup and working.
  • Has an option for setting a node as a pole vector, controlling the bend direction.
    • This should, when finished, make the TwoBoneIK solver much more usable for arms, legs, etc.
  • Exposes settings for adjusting the roll (or twist) of the bones, since TwoBoneIK does not take the existing bone roll/twist into account when solving. This allows the user to control the roll/twist used in the final result.

|

• Added SkeletonModification3DStackHolder
  • This is a simple modification that allows users to use more than one modification stack with a single skeleton. Can be helpful for organization and modification reuse.
  • Known limitation: A stack holder modification can only execution modifications in the stack it's holding if the modification execution mode lines up with the stack holder modification's mode.
  • I'm not 100% positive how much use it will get though, but I can see how if you need it for your project, it would be helpful to have. The code is also pretty simple and straightforward, so adding and maintaining it shouldn't be difficult. It did create the necessity to have per-modification execution modes though.

|

• The Bone struct now includes a local_pose_override.
  • The new local_pose_override transform works just like the global_pose_override transform, but the transform is assumed to be a transform relative to the parent rather than a global position.
    • This makes it much easier to rotate bones with IK, but still have them properly attached (no dislocated bones!)
  • Functions have be added to manipulate this transform, including getting the transform. There is also now a new function for getting the transform used in global_pose_override, for consistency.
  • Functions have been added to convert a local bone pose (like pose, rest, and local_pose_override) to a global bone pose, and vice versa. These functions make working with the bone transforms much easier, especially when used in conjunction with the helper functions added in Skeleton and Skeleton inspector low-level changes #39126.
  • Initially, I was going to use a special transform akin to the global_pose_override, called modification_pose, but I found this led to dislocated bones that floated in the air. Additionally, it basically was just another global_pose_override. Using the new local_pose_override fixed the dislocation issue and the code is just as usable.
  • Functions very similarly to the global_pose_override, just uses a local bone pose instead of a global one. Has the same functions and capabilities.

|

• The Bone struct now keeps track of its children bones, if it has any.
  • This is a rather large underlying change that required a fair amount of code refactoring, but in my testing, everything works exactly as before. However, this PR should probably be consider as compatibility breaking until heavily tested due to this change.
  • Functions have been added to take advantage of this change. In addition to there being functions for adding, removing, and retrieving bone children, there are also functions for updating the skeleton transforms from that bone down.
    • The function for updating a bone's transform and all of the children bones and their transforms is required for decent performance with iterative IK solvers, as the alternative would be to go through the entire skeleton each time.
  • This change removes the need to have a process list, as now the skeleton can be updated via going down the skeleton tree from a root, parent-less bone.
  • This change should also make generating the process order/list much more efficient, as it no longer needs to use a bubble sort. The generation and child assignment process only visits each bone once.

|

• The process_list functions and related code have been removed.
  • They can be added back as deprecated functions though, if needed. Ideally, moving forward, code should take advantage of the bones being aware of their children.
  • The Skeleton3D editor gizmo that draws the bones in the editor still works as expected and now takes advantage of the parent-child information. Additionally, the Skeleton3D inspector also now correctly uses the parent-child information stored in the bones.

|

• Refactored the BoneAttachment3D node.
  • Now Skeleton3D is not responsible for updating the BoneAttachment3D nodes! Once the PhysicalBone3D changes are also done, this will make the Skeleton3D class only responsible for being a Skeleton3D. The BoneAttachment3D node now handles updating itself to the correct bone positions.
  • The BoneAttachment3D node now updates based on a new signal, bone_pose_changed, which is emitted when the bone changes. From a end-user perspective, the node should function exactly the same as before.
    • The function that keeps the transform updated, on_bone_pose_update, is virtual. This allows GDScript users to extend it if needed. The reason behind this is that it allows users to offset BoneAttachment3D nodes via GDScript, something I have seen request a few times.
  • The BoneAttachment3D node now exposes the bone index, along with the bone name. This allows for setting the bone either through the dropdown, or by manually inputting the bone index.
  • The functions for binding nodes to bones, like bind_child_node_to_bone, in Skeleton3D have been removed! This is because they are no longer needed, were already marked as deprecated, and were not really used anywhere in the code base. For projects that use or need these functions, the new signal and a tiny bit of code should be usable to replicate the functionality. Technically this is compatibility breaking, but I do not believe very many projects used the binding functions directly.
  • BoneAttachment3D nodes can now override bone poses! This allows for modifying the bone poses from the node in real time.
    • This property is disabled by default, so any projects using BoneAttachment3D do not have to change anything.
    • This property may help eliminate confusion, as now it is obvious that when the property is disabled, it will NOT override the bone pose. I have seen confusion in beginners with the BoneAttachment3D node and modifying it, so hopefully adding this functionality and exposing it as a property will help.
    • Additionally, this makes it easier to move bones for those who do not want and/or understand the transform conversion system. It also gives a runtime option for manipulating bones that does not require the Godot editor (unlike bone gizmos or the Skeleton3D inspector, both which require the Godot editor to function)
    • The modes currently implemented are global_pose_override, local_pose_override, and custom_pose. The remaining poses could be added in the future if wanted/needed with relative ease.
    • An unintended feature: When set to local_pose_override, you can actually directly modify the results of IK in the editor! This gives an interesting level of control over how IK operates that was entirely accidental, but its a cool added bonus feature.
    • To keep overridden bones in the proper positions, overriding BoneAttachment3D nodes should be children of a BoneAttachment3D node that is following (not overriding) the parent bone. See scene nine in the test project for a working scene example.
  • The BoneAttachment3D node can now use external skeletons, no longer requiring it to be a direct child of a Skeleton3D node.
    • This is helpful for instanced scenes from 3D files, as now the BoneAttachement3D nodes can externally linked so they are not lost on re-import.
    • This property is disabled by default, so newly created BoneAttachement3D nodes expect to be children of a Skeleton3D node and function like they did previously.
    • This also allows BoneAttachment3D nodes to be children of other BoneAttachment3D nodes.
    • When the external skeleton property is enabled, if the BoneAttachment3D node is a child of another BoneAttachment3D node and a NodePath to a Skeleton3D has not already been assigned, it will automatically use the parent BoneAttachment3D's parent Skeleton3D node.
  • BoneAttachment3D now shows a configuration warning when it is not properly setup.

|

• Added functions to Skeleton3D for getting the forward vector using the information stored in the rest pose for the bones. Also added a function for taking a Z-forward basis and converting it to the correct forward direction, allowing LookAt to be used with bones more easily.
  • Also added a signal that emits when a bone pose changes. The signal passes the index of the bone that changed along with it. This is a major part of how I was able to decouple the BoneAttachment3D node from Skeleton3D. This also has the benefit of allowing GDScript users to use the signal if needed, which can be helpful for Skeleton3D plugin creation (for example).

|

• Added a function called rotate_to_align to Basis
  • This function works just like the rotate_from_vector_to_vector function I previously added in Quat, but uses a Basis instead.
  • This function can, and should probably, be optimized, as noted in the GitHub gist that the function is based on.
  • Exposed to GDScript and class reference for the function included.

|

• Everything added, changed, or otherwise touched by this PR should be also be documented in the class reference.

|

• Probably other little changes I have forgot! This is a rather big PR with lots of changes, and it is getting difficult to manage this post. I will try, however, to keep this list as accurate as possible.

Todo:

• Convert the SkeletonIK node's code to a SkeletonModification3D Not needed! The SkeletonIK node will likely need to stick around for awhile so projects can be migrated.
  • A thought, and I have no idea how hard this would be: to make migration easier, have the SkeletonIK node work as a holder for a single SkeletonModification3D, kinda like a little independent-ish ModificationStack. Not sure how feasible or useful this would be though. Something to think about I suppose.
• Investigate converting the PhysicalBone3D so it can function as a SkeletonModification3D (Paused for now)
  • This will involve making PhysicalBone3D a fully functioning, standalone node that works with the Skeleton3D node. See the 2D IK PR for a 2D version of what I plan to do for the PhysicalBone3D node.
  • Write a modification that mirrors the existing API for physical bones currently in the Skeleton3D node.
  • Add additional features!

Note the list above is in no particular order.

Known bugs/errors/problems:

• Saving a scene with SkeletonModification3D resources attached to a Skeleton cause them to no longer work in the editor. Functionality returns upon reloading the scene, and this only occurs in the editor. Fixed/Workaround-found!
  • Possibly related to Custom Resource does not update in editor if changed/saved from script #24646. This PR, even with the fix/workaround, does not solve the problem in the linked issue, but rather it solves the problem just for the SkeletonModificationStack3D.
• GDScript modifications do not work with @tool mode in the Godot editor, as you cannot assign them. If you remove @tool, you can assign it correctly but it will not work in the editor until you add @tool, in which case it will work from that point on.
  • I think this is a GDScript issue, but I'm not 100% certain.

---

The test project I am using can be found here. This project will evolve as I continue to work.

Disclaimer: This work is part of the GSOC 2020 program.

[TwistedTwigleg]

Reserved, in case the opening post gets too big

[TwistedTwigleg]

Updated to the latest version of the master branch, hence the force push

Edit 06/11/2020 - Updated branch to use the latest changes in #39126 (requiring a force push) and pushed initial CCDIK implementation

[JFonS]

I haven't looked deeply into style but the changes look good overall. We need Juan's input on a couple of things, namely the new Quat methods and the fact that skeleton modifications are a Resource.

I think the Resource usage is fine, but better ask now than have the work rejected at the end ;)

[JFonS]

This needs Juan's approval. While the functions are useful, they may be not general enough to be added in Quat itself. If that's the case we will have to move them to where they are used (local "hacks" have priority over global bloat).

[fire]

I've been using these definitions from my peer. Hope it's helpful.

Vector<IKQuat> IKQuat::get_swing_twist(Vector3 p_axis) {
	Vector3 euler = get_euler();
	const real_t d = Vector3(
			euler.x,
			euler.y,
			euler.z)
							 .dot(Vector3(p_axis.x, p_axis.y, p_axis.z));
	set(p_axis.x * d, p_axis.y * d, p_axis.z * d, w);
	normalize();
	if (d < 0) {
		operator*(-1.0f);
	}

	IKQuat swing = IKQuat(-x, -y, -z, w);
	swing = operator*(swing);

	Vector<IKQuat> result;
	result.resize(2);
	result.write[0] = swing;
	result.write[1] = IKQuat(x, y, z, w);
	return result;
}

void IKQuat::clamp_to_quadrance_angle(real_t p_cos_half_angle) {
	real_t new_coeff = 1.0f - (p_cos_half_angle * p_cos_half_angle);
	real_t current_coeff = y * y + z * z + w * w;
	if (new_coeff > current_coeff) {
		return;
	} else {
		x = x < 0.0 ? -p_cos_half_angle : p_cos_half_angle;
		real_t composite_coeff = Math::sqrt(new_coeff / current_coeff);
		y *= composite_coeff;
		z *= composite_coeff;
		w *= composite_coeff;
	}
}

void IKQuat::clamp_to_angle(real_t p_angle) {
	real_t cos_half_angle = Math::cos(0.5f * p_angle);
	clamp_to_quadrance_angle(cos_half_angle);
}

[TwistedTwigleg]

Thanks @fire!

[JFonS]

There is a shared commit between this PR and #39126. I don't know if that will cause merging issues but it's something to be aware of. Maybe @akien-mga knows if git will play along nice with that or not.

[TwistedTwigleg]

I'll rebase and remove the commit from this PR once #39126 is merged, so there will be no conflicts. I just wanted it in this PR so I could use the helper functions.

[JFonS]

As a convention, properties use lower case snake_case.

[JFonS]

Same here, properties should be lower case.

[JFonS]

May be worth adding an ERR_FAIL_COND_MSG() explaining what is going on. Unless it's expected to happen often and it's not really an error. Same with all the state checks in this file.

[JFonS]

CCIDK

[TwistedTwigleg]

Whoops! I'll fix that 😅

[reduz]

I guess this makes sense, I don't see why this should be an error

[fire]

Useful to be StringName for performance reasons of a pointer compare vs string compare?

[TwistedTwigleg]

bone_name is just used to get the bone index using the find_bone function in Skeleton3D, and in a couple conditions. The function returns a String, so I figured I should store it as such. Would storing it as a StringName be more performant?
I tested with switching it to a StringName and it compiled okay, but it wasn't properly getting the bone index from the find_bone function. To be fair though, I only changed the type to a StringName and fixed a couple errors, so it is possible that something needs adjusting for it to work as a StringName and I missed that.

[fire]

Do you need to compare the whole thing or character by character?

For comparison of whole strings, a StringName is faster.

You probably need to cast back to a string.

[TwistedTwigleg]

Looking at the code, it looks like the only comparison is for checking if it is an empty string (if bone_name == "" or equivalent), and this is only in one of the two modifications (LookAt).

Thinking about it, I can probably remove the condition altogether from the modifications, making it just for storage and use in the find_bone function.

edit: I removed the condition and everything seems to be working fine. Now bone_name is just being used for data storage and for the find_bone function.

[fire]

Not variable constructor initialized which is the new Godot Engine master style.

[fire]

Typo. Quaternion.

I agree that this (get_twist_quat and get_swing_quat) should be in the quaternion class and not as a subclass.

[fire]

Spaces are different in documentation. I think this causes an extra space. Please check documentation text.

[TwistedTwigleg]

This is an outdated bit from #39126. Once said PR is pulled in, I'll rebase this PR so that commit is no longer there. This should make the documentation the same as #39126.

I do need to document what I have added/changed via this PR though...

[TwistedTwigleg]

Rebased with the latest version of master, which removes the overlap with #39126.

[fire]

// TODO: for now, fail silently because otherwise, the log is spammed with this error when saving the resource.
// however, in the future, see if there is a way to work around this.

You can use ERR_PRINT_ONCE.

[fire]

Godot's PI constant is Math_PI in math_defs.h.

Edited:

Had to replace PI and PI_2 with Math_PI and Math_PI / 2.0.

[fire]

Godot Engine coding style uses PascalCase.

• SkeletonModification3D_CCDIK
• SkeletonModification3D_LookAt
• SkeletonModification3D_FABRIK

There's an extra underscore.

[TwistedTwigleg]

@fire said:
You can use ERR_PRINT_ONCE.

Yeah, that might not be a bad idea for now. Ideally, I just need to fix the issue that causes that condition to fail, but until then, having an error print couldn't hurt.

Godot's PI constant is Math_PI in math_defs.h.

Awesome! That should fix AppVeyor not building then. I'll make the necessary adjustments.

Godot Engine coding style uses PascalCase.

Yeah. The only issue I have with using PascalCase with the modifications is that it makes it hard to read. For example:

• SkeletonModification3D_FABRIK becomes SkeletonModification3DFABRIK

That said, I probably should change it, to stay consistent. I might leave it for a bit, just to make sure I do not make too many big (potentially breaking) changes at once. I will add renaming the classes to PascalCase to the TODO list though!

[fire]

Missed one :D .\scene/resources/skeleton_modification_3d.h(190): error C2065: 'M_PI': undeclared identifier

[TwistedTwigleg]

Good eye! I'll fix it in the next commit :)
Edit: Fixed 👍

[fire]

Good news, only formatting changes! Not errors. CICD green go-go.

[TwistedTwigleg]

Great! Now I just need to get FABRIK fixed (right now it's not working), implement two bone IK, fix the naming issue, and then I can open this PR up and start fixing bugs 😄

Edit: FABRIK should be fixed now! The jiggle modification was also broken, and now it should also be fixed. I'm not entirely sure what broke or why, but undoing a few things and implementing (and using) get_bone_axis_forward (now called get_bone_axis_forward_vector) fixed the issue.

[fire]

I've been using the term effector to mean tip.

Rotation in a 360 fashion is named rotation

While rotation on the axis is called twist.

Naming things gets confusing with all the terms.

[TwistedTwigleg]

Thanks @fire! The PR has been rebased (CI builds pending) and I adjusted the code based on the review feedback 👍

Edit: Reran doctool to fix documentation issues and adjust the new documentation to current styles.

[fire]

I'm having a hard time finding problems, this is good!

1. needs to be merged into one pr
2. For all of the todos, can you do them? You can also remove them and leave the work for a enhancement proposal? Wanted to resolve them before merging.

I'll run the code through some of the uses and report back. I don't think any demos work anymore.

[TwistedTwigleg]

Thanks. For the single PR, I’ll look at trying to get it into a squashed PR next weekend.

For the TODOs, I’d need to look and see, but probably I won’t be able to tackle them and they’ll need to be handled at a later date and/or by someone else. I can at very least pull them out, remove them from the PR and make an enhancement proposal. I’ll look at trying to get this processing going next weekend as well.

[TwistedTwigleg]

Closing this PR, as I made a squashed version here: #51368
The squashed PR is still in draft form as I need to still go through and cull out the TODOs, but once I have that done I'll change its status.

Edit: I culled out the TODOs, updated the proposal including the enhancements, and opened the PR for potential merging.