[GodotPhysics] CharacterBody3D interaction with large StaticBody3D producing too many collisions

[Dabnabbit]

Godot version

4.0.beta7

System information

Windows 11, RTX 3070TI, Vulcan

Issue description

[UPDATE 3
After more specifically recreating the minimal reproduction projects to utilize a vertex shader displaced plane mesh, with run-time height-map sampled and generated HeightMapShape3D collision mesh data, it seems that the issue is much more specific to Godot4.0 than Godot Physics itself... I've reuploaded both 3.5 minimal reproduction (which seems to work fine in Bullet and Godot Physics) and the direct translation of that project to 4.0 (where the error occurs); Unfortunately this is a pretty major problem for ANY 3D game that intends to use Godot beyond basic planar/cube collision shapes in small indoor environments.
https://youtu.be/pU8mz7SlkzE
/UPDATE]

[UPDATE 2
I've uploaded a video demonstrating the issue a bit more clearly, although not fully demonstrating just how destructive the effect is, it at least demonstrates some of the incorrect calculations being performed.
https://youtu.be/Kw0ThRnUPDs
/UPDATE]

[UPDATE 1
Update after more research and testing on the issue:

It appears that on top of generating too many collisions between only two objects (player and terrain) almost always hitting the hard-coded maximum collision count of 6, the Normals calculated from collisions also stand to be wildly inaccurate occasionally at collision mesh edges, they are capable of returning perpendicular normal angles (parallel to the terrain surface) which all affect the resultant velocity causing some really... bad motion

/UPDATE]

In Godot 4.0.beta5, beta6, and beta7 (and likely all 4.0) CharecterBody3D collisions utilizing any collision shape (most extensively tested with capsule mesh/collider) is producing undesirable and incorrect physics interactions by triggering too numerous collision events per tic with the same object. This is most exaggerated in the example of large terrain colliders, it would appear that even soft/subtle terrain gradient changes can trigger extremely large numbers of collisions (though gdscript only seems to report a maximum of 6 collisions per move_and_slide()?) that are causing incorrect and undesirable outcomes.

The perceived effect is that the player object is "snagged" and appears to almost snap to vertex and mesh edges along the collider when moving, and can completely negate all velocity while moving across a relatively flat collision mesh, sometimes also causing a sling-shot effect when the player object escapes a snag.

In attempts to trouble-shoot this matter, I discovered that printing/producing the entire list of collisions while the player is moving indicates that any trouble-interactions are indicated by far more collisions with the StaticBody than normal.

In Godot 3.5, the KinematicBody and StaticBody interaction produces typically 1 collision per physics tic, which is the expected result, and occasionally produces 2 collisions when moving up steeper collisions, which is also to be expected and behaves correctly.

(Output of all instances where more than one collision occurs simultaneously per tic; notice the infrequency and collision count never exceeding two (index of 1))

In Godot 4.0.b7, the CharecterBody3D and StaticBody3D interaction produces between 2-6 collisions per physics tic, which is NOT the expected result between only two colliders, and does not appear to report on collision counts past 6 (index 5). This directly seems to affect the move_and_slide() behavior very negatively and incorrectly.

(Output of all instances where more than one collision occurs simultaneously per tic; notice the massive increase in frequency of occurrence, and the number/volume of collisions per tic is 6 (possibly more but not reported?)

I have produced two minimal projects to demonstrate this contrasting issue in 3.5 and 4.0.b7, however due to size of the terrain mesh object, the file-size exceeds GitHub upload limitations, so I'm trying to figure out how to upload/share them... They were produced using the exact same high resolution terrain mesh in GLB format automatically generating a collision mesh through Godot's import process, and the only thing added to the scene other than the terrain is a very basic player controller, utilizing a KinematicBody/CharecterBody3D with a Capsule CollisionShape and Capsule MeshInstance, with a child camera, and very basic identical (other than translation) input controls for movement and utilizing "move_and_slide"...

(Godot 4 snapshot of Terrainmesh+Collider)

(Godot 3.5 snapshot of Terrainmesh+Collider)

Steps to reproduce

Create large size terrain mesh in GLB format, automatically generating a collision mesh through Godot's import process, and add basic player controller, utilizing a KinematicBody/CharecterBody3D with a Capsule CollisionShape and Capsule MeshInstance, with a child camera, and very basic identical (other than translation) input controls for movement and utilizing "move_and_slide"

Minimal reproductio

HeightTest_3_5.zip
n project
HeightTest_3_5_to_4_0.zip

[Calinou]

Can you test this after switching the 3D physics engine to GodotPhysics in the Project Settings in 3.5? Godot 4.0 only features GodotPhysics, not Bullet.

[Dabnabbit]

Hey good call, I swapped the 3D physics engine in the project settings in 3.5 to Bullet, tested it out as a control, and it was behaving normally, I swapped the 3D physics engine to Godot, and sure enough the issue showed up in 3.5:

It's worth mentioning it doesn't seem QUITE as bad as it does in 4.0, and is only showing 4 collisions per tic instead of the 6 I'm seeing in 4.0+ (This appears to be because the hardcoded collision count changed from 4 to 6 in Godot4.0), but it definitely still has the snagging and the jankiness while moving along smooth and low gradient surfaces.

[Dabnabbit]

Update after more research and testing on the issue:

It appears that on top of generating too many collisions between only two objects (player and terrain) almost always hitting the hard-coded maximum collision count of 6, the Normals calculated from collisions also stand to be wildly inaccurate occasionally at collision mesh edges, they are capable of returning perpendicular normal angles (parallel to the terrain surface) which all affect the resultant velocity causing some really... bad motion

[Salvakiya]

Just leaving this here. I believe it could be related to this function which is long.

godot/servers/physics_3d/godot_space_3d.cpp

Lines 646 to 1023 in 2f0d3d5

	bool GodotSpace3D::test_body_motion(GodotBody3D *p_body, const PhysicsServer3D::MotionParameters &p_parameters, PhysicsServer3D::MotionResult *r_result) {
	//give me back regular physics engine logic
	//this is madness
	//and most people using this function will think
	//what it does is simpler than using physics
	//this took about a week to get right..
	//but is it right? who knows at this point..
	ERR_FAIL_INDEX_V(p_parameters.max_collisions, PhysicsServer3D::MotionResult::MAX_COLLISIONS, false);
	if (r_result) {
	*r_result = PhysicsServer3D::MotionResult();
	}
	AABB body_aabb;
	bool shapes_found = false;
	for (int i = 0; i < p_body->get_shape_count(); i++) {
	if (p_body->is_shape_disabled(i)) {
	continue;
	}
	if (!shapes_found) {
	body_aabb = p_body->get_shape_aabb(i);
	shapes_found = true;
	} else {
	body_aabb = body_aabb.merge(p_body->get_shape_aabb(i));
	}
	}
	if (!shapes_found) {
	if (r_result) {
	r_result->travel = p_parameters.motion;
	}
	return false;
	}
	real_t margin = MAX(p_parameters.margin, TEST_MOTION_MARGIN_MIN_VALUE);
	// Undo the currently transform the physics server is aware of and apply the provided one
	body_aabb = p_parameters.from.xform(p_body->get_inv_transform().xform(body_aabb));
	body_aabb = body_aabb.grow(margin);
	real_t min_contact_depth = margin * TEST_MOTION_MIN_CONTACT_DEPTH_FACTOR;
	real_t motion_length = p_parameters.motion.length();
	Vector3 motion_normal = p_parameters.motion / motion_length;
	Transform3D body_transform = p_parameters.from;
	bool recovered = false;
	{
	//STEP 1, FREE BODY IF STUCK
	const int max_results = 32;
	int recover_attempts = 4;
	Vector3 sr[max_results * 2];
	real_t priorities[max_results];
	do {
	GodotPhysicsServer3D::CollCbkData cbk;
	cbk.max = max_results;
	cbk.amount = 0;
	cbk.ptr = sr;
	GodotPhysicsServer3D::CollCbkData *cbkptr = &cbk;
	GodotCollisionSolver3D::CallbackResult cbkres = GodotPhysicsServer3D::_shape_col_cbk;
	int priority_amount = 0;
	bool collided = false;
	int amount = _cull_aabb_for_body(p_body, body_aabb);
	for (int j = 0; j < p_body->get_shape_count(); j++) {
	if (p_body->is_shape_disabled(j)) {
	continue;
	}
	Transform3D body_shape_xform = body_transform * p_body->get_shape_transform(j);
	GodotShape3D *body_shape = p_body->get_shape(j);
	for (int i = 0; i < amount; i++) {
	const GodotCollisionObject3D *col_obj = intersection_query_results[i];
	if (p_parameters.exclude_bodies.has(col_obj->get_self())) {
	continue;
	}
	if (p_parameters.exclude_objects.has(col_obj->get_instance_id())) {
	continue;
	}
	int shape_idx = intersection_query_subindex_results[i];
	if (GodotCollisionSolver3D::solve_static(body_shape, body_shape_xform, col_obj->get_shape(shape_idx), col_obj->get_transform() * col_obj->get_shape_transform(shape_idx), cbkres, cbkptr, nullptr, margin)) {
	collided = cbk.amount > 0;
	}
	while (cbk.amount > priority_amount) {
	priorities[priority_amount] = col_obj->get_collision_priority();
	priority_amount++;
	}
	}
	}
	if (!collided) {
	break;
	}
	real_t inv_total_weight = 0.0;
	for (int i = 0; i < cbk.amount; i++) {
	inv_total_weight += priorities[i];
	}
	inv_total_weight = Math::is_zero_approx(inv_total_weight) ? 1.0 : (real_t)cbk.amount / inv_total_weight;
	recovered = true;
	Vector3 recover_motion;
	for (int i = 0; i < cbk.amount; i++) {
	Vector3 a = sr[i * 2 + 0];
	Vector3 b = sr[i * 2 + 1];
	// Compute plane on b towards a.
	Vector3 n = (a - b).normalized();
	real_t d = n.dot(b);
	// Compute depth on recovered motion.
	real_t depth = n.dot(a + recover_motion) - d;
	if (depth > min_contact_depth + CMP_EPSILON) {
	// Only recover if there is penetration.
	recover_motion -= n * (depth - min_contact_depth) * 0.4 * priorities[i] * inv_total_weight;
	}
	}
	if (recover_motion == Vector3()) {
	collided = false;
	break;
	}
	body_transform.origin += recover_motion;
	body_aabb.position += recover_motion;
	recover_attempts--;
	} while (recover_attempts);
	}
	real_t safe = 1.0;
	real_t unsafe = 1.0;
	int best_shape = -1;
	{
	// STEP 2 ATTEMPT MOTION
	AABB motion_aabb = body_aabb;
	motion_aabb.position += p_parameters.motion;
	motion_aabb = motion_aabb.merge(body_aabb);
	int amount = _cull_aabb_for_body(p_body, motion_aabb);
	for (int j = 0; j < p_body->get_shape_count(); j++) {
	if (p_body->is_shape_disabled(j)) {
	continue;
	}
	GodotShape3D *body_shape = p_body->get_shape(j);
	// Colliding separation rays allows to properly snap to the ground,
	// otherwise it's not needed in regular motion.
	if (!p_parameters.collide_separation_ray && (body_shape->get_type() == PhysicsServer3D::SHAPE_SEPARATION_RAY)) {
	// When slide on slope is on, separation ray shape acts like a regular shape.
	if (!static_cast<GodotSeparationRayShape3D *>(body_shape)->get_slide_on_slope()) {
	continue;
	}
	}
	Transform3D body_shape_xform = body_transform * p_body->get_shape_transform(j);
	Transform3D body_shape_xform_inv = body_shape_xform.affine_inverse();
	GodotMotionShape3D mshape;
	mshape.shape = body_shape;
	mshape.motion = body_shape_xform_inv.basis.xform(p_parameters.motion);
	bool stuck = false;
	real_t best_safe = 1;
	real_t best_unsafe = 1;
	for (int i = 0; i < amount; i++) {
	const GodotCollisionObject3D *col_obj = intersection_query_results[i];
	if (p_parameters.exclude_bodies.has(col_obj->get_self())) {
	continue;
	}
	if (p_parameters.exclude_objects.has(col_obj->get_instance_id())) {
	continue;
	}
	int shape_idx = intersection_query_subindex_results[i];
	//test initial overlap, does it collide if going all the way?
	Vector3 point_A, point_B;
	Vector3 sep_axis = motion_normal;
	Transform3D col_obj_xform = col_obj->get_transform() * col_obj->get_shape_transform(shape_idx);
	//test initial overlap, does it collide if going all the way?
	if (GodotCollisionSolver3D::solve_distance(&mshape, body_shape_xform, col_obj->get_shape(shape_idx), col_obj_xform, point_A, point_B, motion_aabb, &sep_axis)) {
	continue;
	}
	sep_axis = motion_normal;
	if (!GodotCollisionSolver3D::solve_distance(body_shape, body_shape_xform, col_obj->get_shape(shape_idx), col_obj_xform, point_A, point_B, motion_aabb, &sep_axis)) {
	stuck = true;
	break;
	}
	//just do kinematic solving
	real_t low = 0.0;
	real_t hi = 1.0;
	real_t fraction_coeff = 0.5;
	for (int k = 0; k < 8; k++) { //steps should be customizable..
	real_t fraction = low + (hi - low) * fraction_coeff;
	mshape.motion = body_shape_xform_inv.basis.xform(p_parameters.motion * fraction);
	Vector3 lA, lB;
	Vector3 sep = motion_normal; //important optimization for this to work fast enough
	bool collided = !GodotCollisionSolver3D::solve_distance(&mshape, body_shape_xform, col_obj->get_shape(shape_idx), col_obj_xform, lA, lB, motion_aabb, &sep);
	if (collided) {
	hi = fraction;
	if ((k == 0) || (low > 0.0)) { // Did it not collide before?
	// When alternating or first iteration, use dichotomy.
	fraction_coeff = 0.5;
	} else {
	// When colliding again, converge faster towards low fraction
	// for more accurate results with long motions that collide near the start.
	fraction_coeff = 0.25;
	}
	} else {
	point_A = lA;
	point_B = lB;
	low = fraction;
	if ((k == 0) || (hi < 1.0)) { // Did it collide before?
	// When alternating or first iteration, use dichotomy.
	fraction_coeff = 0.5;
	} else {
	// When not colliding again, converge faster towards high fraction
	// for more accurate results with long motions that collide near the end.
	fraction_coeff = 0.75;
	}
	}
	}
	if (low < best_safe) {
	best_safe = low;
	best_unsafe = hi;
	}
	}
	if (stuck) {
	safe = 0;
	unsafe = 0;
	best_shape = j; //sadly it's the best
	break;
	}
	if (best_safe == 1.0) {
	continue;
	}
	if (best_safe < safe) {
	safe = best_safe;
	unsafe = best_unsafe;
	best_shape = j;
	}
	}
	}
	bool collided = false;
	if ((p_parameters.recovery_as_collision && recovered) || (safe < 1)) {
	if (safe >= 1) {
	best_shape = -1; //no best shape with cast, reset to -1
	}
	//it collided, let's get the rest info in unsafe advance
	Transform3D ugt = body_transform;
	ugt.origin += p_parameters.motion * unsafe;
	_RestResultData results[PhysicsServer3D::MotionResult::MAX_COLLISIONS];
	_RestCallbackData rcd;
	if (p_parameters.max_collisions > 1) {
	rcd.max_results = p_parameters.max_collisions;
	rcd.other_results = results;
	}
	// Allowed depth can't be lower than motion length, in order to handle contacts at low speed.
	rcd.min_allowed_depth = MIN(motion_length, min_contact_depth);
	body_aabb.position += p_parameters.motion * unsafe;
	int amount = _cull_aabb_for_body(p_body, body_aabb);
	int from_shape = best_shape != -1 ? best_shape : 0;
	int to_shape = best_shape != -1 ? best_shape + 1 : p_body->get_shape_count();
	for (int j = from_shape; j < to_shape; j++) {
	if (p_body->is_shape_disabled(j)) {
	continue;
	}
	Transform3D body_shape_xform = ugt * p_body->get_shape_transform(j);
	GodotShape3D *body_shape = p_body->get_shape(j);
	for (int i = 0; i < amount; i++) {
	const GodotCollisionObject3D *col_obj = intersection_query_results[i];
	if (p_parameters.exclude_bodies.has(col_obj->get_self())) {
	continue;
	}
	if (p_parameters.exclude_objects.has(col_obj->get_instance_id())) {
	continue;
	}
	int shape_idx = intersection_query_subindex_results[i];
	rcd.object = col_obj;
	rcd.shape = shape_idx;
	bool sc = GodotCollisionSolver3D::solve_static(body_shape, body_shape_xform, col_obj->get_shape(shape_idx), col_obj->get_transform() * col_obj->get_shape_transform(shape_idx), _rest_cbk_result, &rcd, nullptr, margin);
	if (!sc) {
	continue;
	}
	}
	}
	if (rcd.result_count > 0) {
	if (r_result) {
	for (int collision_index = 0; collision_index < rcd.result_count; ++collision_index) {
	const _RestResultData &result = (collision_index > 0) ? rcd.other_results[collision_index - 1] : rcd.best_result;
	PhysicsServer3D::MotionCollision &collision = r_result->collisions[collision_index];
	collision.collider = result.object->get_self();
	collision.collider_id = result.object->get_instance_id();
	collision.collider_shape = result.shape;
	collision.local_shape = result.local_shape;
	collision.normal = result.normal;
	collision.position = result.contact;
	collision.depth = result.len;
	const GodotBody3D *body = static_cast<const GodotBody3D *>(result.object);
	Vector3 rel_vec = result.contact - (body->get_transform().origin + body->get_center_of_mass());
	collision.collider_velocity = body->get_linear_velocity() + (body->get_angular_velocity()).cross(rel_vec);
	}
	r_result->travel = safe * p_parameters.motion;
	r_result->remainder = p_parameters.motion - safe * p_parameters.motion;
	r_result->travel += (body_transform.get_origin() - p_parameters.from.get_origin());
	r_result->collision_safe_fraction = safe;
	r_result->collision_unsafe_fraction = unsafe;
	r_result->collision_count = rcd.result_count;
	r_result->collision_depth = rcd.best_result.len;
	}
	collided = true;
	}
	}
	if (!collided && r_result) {
	r_result->travel = p_parameters.motion;
	r_result->remainder = Vector3();
	r_result->travel += (body_transform.get_origin() - p_parameters.from.get_origin());
	r_result->collision_safe_fraction = 1.0;
	r_result->collision_unsafe_fraction = 1.0;
	r_result->collision_depth = 0.0;
	}
	return collided;
	}

I would honestly really like to see an implementation of Jolt https://github.com/jrouwe/JoltPhysics

[Salvakiya]

Possibly related #69203, #66917

[Zireael07]

Confirming that the normals can get very weird in Godot physics

[rburing]

The worst glitches here are a result of the static body effectively being scaled 2048 x 1 x 2048, which is not well supported in Godot Physics. The scale comes from the mesh being scaled this way in the .glb file. Using e.g. Blender to apply the scale (or export as .obj) gives a mesh for which collision can be properly generated. Please report what issues remain after doing this.

Also be aware that getting stuck on an edge (after taking the above into account) can be due to the character body's floor_max_angle property, and "hopping" when going down slopes can be remedied by increasing floor_snap_length.

It is an editor issue that we don't generate colliders for scaled meshes correctly at the moment.

[Dabnabbit]

I usually double check to make sure all transforms are applied in Blender before exporting any meshes but I'll triple check that for the sake of the minimal reproduction project, however, it's very critical to note that the primary issue involves a dynamic run-time generated collision shape that does not undergo any import process and is not scaled, it is simply utilizing the HeightMapShape3D mesh shape.

It is also worth noting that there are use-cases where the HeightMapShape3D MUST be scaled due to the fact it does not contain separate properties for size and index count, unfortunately they are directly linked through the only three properties of the shape that are exposed through the API. That said I've done extensive testing mostly ensuring a scale of 1:1 is maintained on the collision shape, and that this behavior seems to persist across different collision shapes, appearing to impact convex, concave, and heightmapshape collider shapes at the very least.

I will update soon after some additional testing I thought about performing last night and even re-exporting the minimal reproduction project mesh from blender (which uses a concave collider on import I believe?) to provide any additional findings

[Dabnabbit]

The worst glitches here are a result of the static body effectively being scaled 2048 x 1 x 2048, which is not well supported in Godot Physics. The scale comes from the mesh being scaled this way in the .glb file. Using e.g. Blender to apply the scale (or export as .obj) gives a mesh for which collision can be properly generated. Please report what issues remain after doing this.

Also be aware that getting stuck on an edge (after taking the above into account) can be due to the character body's floor_max_angle property, and "hopping" when going down slopes can be remedied by increasing floor_snap_length.

It is an editor issue that we don't generate colliders for scaled meshes correctly at the moment.

Wellp, after double checking my minimal projects, apparently when I re-exported the blender mesh after trimming it (it was originally too large in filesize for GIT) the scale was not applied anymore, so it WAS being imported scaled instead of fully normalized. This definitely can contribute to the results in the minimal projects making them unreliable for verification of the issue.

That said, the actual issue is not from interaction from the imported mesh colliders, but with dynamically generated collision meshes of the HeightMapShape3D variety with no scaling applied, HOWEVER, this does give me more stuff to double check and verify to see if this is specifically a HeightMapShape3D difference...

[rburing]

Thanks for checking. A reproduction project using a heightmap would be much appreciated (preferably including the cool debug drawing of the normals).

[Dabnabbit]

Thanks for checking. A reproduction project using a heightmap would be much appreciated (preferably including the cool debug drawing of the normals).

As a quick update I've very painstaking rewritten and isolated much of the code to more closely replicate the actual issue initially suffered in 4.0 into 3.5 (backporting and chopping up code a lot, and backporting the debugger visualizers) and this is where things get weird: the run-time heightmap collider doesn't seem to be experiencing the same issues in 3.5 as in 4.0 even using Godot physics... so there's something extra weird going on here... even though the code is almost identical, minus a lot of the other parts that it doesn't need (the entire clipmap system for the visual mesh components)...

For now I've removed the old minimal projects, and reuploaded the heightmap generated collision 3.5 project now, which can be switched between Godot and Bullet physics (but apparently both behave properly as of right now?)

Gonna re-rewrite the same minimal project in 4.0 again, based on the 3.5 project... tediously annoying, but hoping to isolate when exactly the issue starts happening...
[UPDATE]
Fortunately the conversion back from 3.5 to 4.0 was pretty quick for the most part, and it would appear that this in fact a difference between 3.5 and 4.0, not just Godot Physics... gonna close all projects out and reload them side by side again and make another recording, and upload the minimal projects after confirming.

[rburing]

I confirm the issue.

The move_and_slide method eventually calls GodotSpace3D::test_body_motion, and to resolve collisions it uses GodotCollisionSolver3D::solve_static, which takes the capsule and heightmap shapes and their transforms etc. This then calls the special case GodotCollisionSolver3D::solve_concave (since GodotHeightMapShape3D derives from GodotConcaveShape3D), which calls GodotHeightMapShape3D::cull with GodotCollisionSolver3D::concave_callback. This leads to sat_calculate_penetration being called with a GodotCapsuleShape3D and a GodotFaceShape3D from the heightmap, hence the special case _collision_capsule_face is called, where a best separating axis is determined (mostly vertical, so far so good), and contacts are generated in SeparatorAxisTest::generate_contacts. To generate contacts, first the supports of both shapes are determined, and then _generate_contacts_from_supports generates the contacts from there (calling e.g. _generate_contacts_point_face or _generate_contacts_point_edge).

Now, from the capsule being on top of a relatively flat part of the heightmap, an expected type of collision is that of a point of the capsule with a face of the heightmap, and indeed these collisions are occurring with nice normals. Another type of collision is that of a point of the capsule with an edge of a face of the heightmap. These collisions are also occurring, and often with mostly vertical normals, but sometimes not! The problem is that even though the best separation axis is always mostly vertical, it can happen that the point is very close to the edge, and vertically closer than horizontally, which causes the normal to be more horizontal.

That's what I've found so far anyway. I'm not sure how this should be fixed.

[Salvakiya]

I confirm the issue.

The move_and_slide method eventually calls GodotSpace3D::test_body_motion, and to resolve collisions it uses GodotCollisionSolver3D::solve_static, which takes the capsule and heightmap shapes and their transforms etc. This then calls the special case GodotCollisionSolver3D::solve_concave (since GodotHeightMapShape3D derives from GodotConcaveShape3D), which calls GodotHeightMapShape3D::cull with GodotCollisionSolver3D::concave_callback. This leads to sat_calculate_penetration being called with a GodotCapsuleShape3D and a GodotFaceShape3D from the heightmap, hence the special case _collision_capsule_face is called, where a best separating axis is determined (mostly vertical, so far so good), and contacts are generated in SeparatorAxisTest::generate_contacts. To generate contacts, first the supports of both shapes are determined, and then _generate_contacts_from_supports generates the contacts from there (calling e.g. _generate_contacts_point_face or _generate_contacts_point_edge).

Now, from the capsule being on top of a relatively flat part of the heightmap, an expected type of collision is that of a point of the capsule with a face of the heightmap, and indeed these collisions are occurring with nice normals. Another type of collision is that of a point of the capsule with an edge of a face of the heightmap. These collisions are also occurring, and often with mostly vertical normals, but sometimes not! The problem is that even though the best separation axis is always mostly vertical, it can happen that the point is very close to the edge, and vertically closer than horizontally, which causes the normal to be more horizontal.

That's what I've found so far anyway. I'm not sure how this should be fixed.

That is interesting. I think there is another part too, Is there any findings as to why it generates so many collisions per tick? As mentioned earlier in 4.0 it generates 2-6 per tick vs 3.5's generating 1-2 depending on the situation.

[Dabnabbit]

I confirm the issue.

The move_and_slide method eventually calls GodotSpace3D::test_body_motion, and to resolve collisions it uses GodotCollisionSolver3D::solve_static, which takes the capsule and heightmap shapes and their transforms etc. This then calls the special case GodotCollisionSolver3D::solve_concave (since GodotHeightMapShape3D derives from GodotConcaveShape3D), which calls GodotHeightMapShape3D::cull with GodotCollisionSolver3D::concave_callback. This leads to sat_calculate_penetration being called with a GodotCapsuleShape3D and a GodotFaceShape3D from the heightmap, hence the special case _collision_capsule_face is called, where a best separating axis is determined (mostly vertical, so far so good), and contacts are generated in SeparatorAxisTest::generate_contacts. To generate contacts, first the supports of both shapes are determined, and then _generate_contacts_from_supports generates the contacts from there (calling e.g. _generate_contacts_point_face or _generate_contacts_point_edge).

Now, from the capsule being on top of a relatively flat part of the heightmap, an expected type of collision is that of a point of the capsule with a face of the heightmap, and indeed these collisions are occurring with nice normals. Another type of collision is that of a point of the capsule with an edge of a face of the heightmap. These collisions are also occurring, and often with mostly vertical normals, but sometimes not! The problem is that even though the best separation axis is always mostly vertical, it can happen that the point is very close to the edge, and vertically closer than horizontally, which causes the normal to be more horizontal.

That's what I've found so far anyway. I'm not sure how this should be fixed.

This sounds very much exactly to be what's happening, and explains almost all of the symptoms and why it's doing what it is doing. It is exclusively generating bad normal calculations along face edges, and far too often generating them at horizontal as opposed to vertical axis. This ALSO explains why it exhibits the same behavior using the HeightMapShape AND the auto-generated collisions that Godot creates when importing meshes for collision, which utilize Concave mesh shapes.

This sounds like Concave collision in Godot (although concave is always the hardest) needs to be immediately flagged as unsupported and dysfunctional.

Is it possible to implement any of the following:

• Limit the number of interactions per tic, right now part of the issue is that the system is allowing the same two bodies to collide up to the maximum number of hard-coded iterations even without any additional external interactions applied. If there are only two bodies involved in the interaction, there should not be more than 1-2 collisions occurring PER axis, and right now it will spend 6 collisions on just the floor/terrain interaction, and the more collisions it performs the higher the chance of generating bad normals, which cause bad interactions and incorrect flagging of "floor" and "wall" also creating cascading issues across physics AND gameplay logic.
• Perform some kind of more "fixed" vector orientation for the separation axis? Can we provide a known UP vector of sorts to help mitigate this being miscalculated as non-vertical, especially for the same of heightmaps that will almost exclusively always use the same upward vector? (Very bandaid)
• Finally, would it be possible to implement some kind of anomalous result detection in the collision iterations, telling the engine to discard any wildly different results? I know this one is a bit of a stretch, but would it be possible to compare results of collisions/iterations and discard any extreme deviance? It should be very very unlikely to have normals vary over 45 degrees at the same point of contact, whether calculated by edge, face, or vertex, if the normals are coming back at extreme differences, perhaps discard the collisions and perform some kind of interpolation instead?

Overall this is a very unfortunate situation, as this affects the entire 3D engine as a whole, and has far deeper reaching implications than it seems at surface value. This should be marked as highly critical for all 3D development on the engine.

Thanks for your expedient follow-up rburing, I know it's some very very tedious source code to traverse (we already did some of our own) and a lot of it relatively unchanged from up to NINE (9) years ago, and I appreciate your help. Hopefully someone has a potential solution in mind, or this helps increase the prioritization of engine physics corrections and improvements.

[Calinou]

• Limit the number of interactions per tic

As I understand it, this approach has undesired side effects: #35945 (comment)

[rburing]

The three suggestions above don't sound like the correct solution to this deeper issue. You can try the first and third idea by implementing your own character controller using move_and_collide, but I think there must be something better we can do or change internally to solve this issue.

I've posted the theoretical question arising from this issue on Game Development Stack Exchange: https://gamedev.stackexchange.com/questions/203635/collision-normal-for-sphere-near-edge-of-triangle-face

[Dabnabbit]

The three suggestions above don't sound like the correct solution to this deeper issue. You can try the first and third idea by implementing your own character controller using move_and_collide, but I think there must be something better we can do or change internally to solve this issue.

I've posted the theoretical question arising from this issue on Game Development Stack Exchange: https://gamedev.stackexchange.com/questions/203635/collision-normal-for-sphere-near-edge-of-triangle-face

I think you are 100% correct, I'm going to move forward writing my own custom raycast-centric player controller, however, everything I've suggested is a bandaid fix and will not be something I'd expect from 95% of the Godot developer community and is a critical issue for the current and future viability of Godot 4.0+ 3D development.

I'm just a bit concerned that some of these issues have been brought up several times over the years and not fully addressed, as 4.0 was a major selling point for myself and other long-term experienced developers to consider Godot as a viable platform for 3D development, it's a bit disheartening to see such a core component in it's current state, as I've otherwise fallen in love with the engine. Thanks for taking this on Rburing, let me know if there is anything else that can be done to help troubleshoot and debug matters.

[reduz]

AFAIK this algorithm should work properly even with trimesh, and even if there is a tiny separation normal. I think here it would be worth investigating where the values differ in 3.5 and 4.0 when feeding the same input, as it may just be a recently introduced bug.

[Salvakiya]

although above demonstrates the issue persists in 3.5 (GDPhysics) and in 4.0. It was just mentioned that it seemed to be worse in 4.0. I think something is just fundamentally broken in the code. Might just be me but everyone I know using 3.x chose bullet physics over GDPhysics. Finding and solving the issue will probably be quite an undertaking as one of the functions is 350+ lines of code. It will be a very necessary thing to tackle very soon though.

[rburing]

Here is a more minimal reproduction project:

triangle_edge_not_normal.zip

triangle_edge_not_normal3.zip

[rburing]

In GodotSpace3D::test_body_motion, after kinematic solving we move the body into the unsafe position:

godot/servers/physics_3d/godot_space_3d.cpp

Lines 927 to 929 in 0bb94df

	//it collided, let's get the rest info in unsafe advance
	Transform3D ugt = body_transform;
	ugt.origin += p_parameters.motion * unsafe;

and we generate collision normals from there.

Now observe that if p_parameters.recovery_as_collision == true and p_parameters.motion == Vector(0, 0, 0), the resulting position is not exactly "unsafe", but rather the position where the body ended up after recovery:

godot/servers/physics_3d/godot_space_3d.cpp

Line 784 in 0bb94df

body_transform.origin += recover_motion;

The use of min_contact_depth in the calculation of recover_motion is probably supposed to ensure that we can get a decent contact normal, but this seems to be somehow failing for the capsule shape. Namely, it seems that the contacts are getting too close, which leads to numerical error in the contact normal, which causes it to become more horizontal.

(I've confirmed CharacterBody3D::_move_and_slide_grounded does end up calling test_body_motion this way.)

The strange thing is that a capsule with height == 2.0 * radius (i.e. a capsule which degenerates to a sphere) does not seem to have this problem, but non-degenerate capsules with height > 2.0 * radius (even with radius = 0.5 and height = 1.001) do have this problem. This made me suspicious of GodotCapsuleShape3D::project_range and GodotCapsuleShape3D::get_support(s), but I don't see any mistake in them. The problem is also present in 3.x though slightly less bad (I added a 3.5 version of the MRP above), and cylinders also have the same problem of bad normals on trimesh edges. @reduz do you know why else the contacts could be getting too close?

[GuyCooke]

I believe I have encountered this same issue, but not through terrain. My steps:

New project, new 3D scene.
Make a CharacterBody3D "character" with CollisionBody3D (capsule shape) and CSGCylinder3D children.
Make a StaticBody3D "ground" with CollisionBody3D (box shape) and CSGBox3D children.
Make a RigidBody3D "box" with CollisionBody3D (box shape) and CSGBox3D children. Rotate the box at a jaunty angle so it is not upright and not square to the coordinate system. I discovered the issue originally by experimenting with having the box tumble into place, but cannot reproduce it if the box is only rotated around Y.
Add script to the CharacterBody3D. Choose the Character Movement template.
Add a camera.

Run the scene. Use the directional keys to rub up against the box (slide) with the character. These issues occur:

The character snags on the box corners (vertical edges of the box) when moving along/against the box wall and cannot pass the box corner without leaving the box and going around the corner manually.

The character sometimes snags on the box and cannot be moved except by jumping or random movement after which they may break free and move again.

The character sometimes snags permanently on the box and cannot be moved even by jumping.

Issue remains after changing the character collision shape to a cylinder instead of a capsule, though the cylinder slides less well along the box (sometimes shuddering movement).

I have logged the velocity, remainder and travel after the call to move_and_slide in the template. The travel is the zero vector when snagged, the remainder is a very small vector and the velocity is the zero vector unless you jump while snagged in which case the y component of the vector stays 4.5 per the template. If you break free from the snag, the y component is applied as expected, but remains 4.5 forever while snagged.

While in the snagged state, y component either 0 or 4.5, on_wall is always false and on_floor is always true after calling move_and_slide.

If you move directly toward the box while snagged, on_wall remains false after move_and_slide, even when you move toward the box, but it toggles between true/false as expected if not snagged. Logging and noting this is how I have come to recognise the "permanently snagged" state.

[Swarkin]

Is my problem related, or should I create a new issue about this;
Bad/Jittery collision of CharacterBody3D with ConvexPolygonShape, regular collision with BoxShape and similar simple shapes

convexpolygon.mp4

[Dabnabbit]

Is my problem related, or should I create a new issue about this; Bad/Jittery collision of CharacterBody3D with ConvexPolygonShape, regular collision with BoxShape and similar simple shapes

convexpolygon.mp4

@Swarkin Oh hey I saw your project on Discord I think, love the aesthetic!

It's hard to tell if this is exactly what's causing your issue, but know that there is a very large number of closely related collision solver issues tied to this issue. Fortunately they ARE working investigated and worked on by a few folk in the community, the best place to see recent progress in this regard is in this issue:
https://github.com/godotengine/godot/issues/70615#issuecomment-1368377145

I wouldn't be at all surprised if it's the same issue, I just haven't experienced it from the top like you are doing here in your video clip.

[rburing]

To give an update on this:

• the badness of the normals seems to be fixed by recent PRs improving the collision normal reporting,
• but multiple collisions being reported on internal edges of trimesh/heightmap shapes is still an issue.

I propose to implement section 4 of https://www.codercorner.com/MeshContacts.pdf, for which #72868 is a first step (for trimesh shapes), but it will take some work to fit the rest of the solution into the existing Godot Physics.

[Dabnabbit]

Thanks for the update @rburing. I wish I could say I still had the same amount of time to dedicate to trouble-shooting this that I had before, but that hasn't been the case. Glad to see some progress has been made, but I really hope this is on the radar for everyone on the team as a major road-block for Godot4 in 3D. It was enough for me to abandon some mature projects in Godot and migrate them to Unreal for the time being, though I would prefer working in Godot if these things were ever resolved.

[AttackButton]

I think the problem is with the value retrived by the _RestCallbackData2D and_RestCallbackData (the 3D version). I think they are new in Godot 4.

Some of the functions related were implemented, but I'm not sure if they were used directly or indirectly within body_test_motion as they should. Some of the related functions with _RestCallbackData are _rest_cbk_result and GodotPhysicsDirectSpaceState.

From body_test_motion 2D result:

	r_result->collision_normal = rcd.best_normal;
	r_result->collision_point = rcd.best_contact;
	r_result->collision_depth = rcd.best_len;

That's not to say that body_test_motion (2D and 3D) doesn't have its own problems. Which look pretty manageable.

[AttackButton]

This could probably be the same problem I had in #79518. It's just the default values of safe_margin and collision_priority that need to be changed to make it work. Which does not exclude some issues with body_test_motion.

@Dabnabbit When you can, try to set the collision_priority of the StaticBodies 10, 100 or even more times higher than the PLayer (priority 1 is ok) and other objects. This prevents the Player from passing through walls and platforms. And the safe_margin of the objects at the minimum possible. I think it is 0.001 at the moment.

In you script (test_collider.gd):

	var static_body := StaticBody3D.new()
	static_body.collision_priority = 100
	static_body.set_name("ClipMapStaticBody")
	add_child(static_body)

	var height_map_shape := HeightMapShape3D.new()

Normally, walls and floors need to have a collision_priority 10 or 100 times higher than other objects. This way you will have space to configure the priority of smaller and faster objects if you want. The faster and small the object's, the lower the priority needs to be.