Spine Two-Color Tint Bug

[TyelorD]

• axmol version: 2.1.2
• devices test on: M1 Macbook Pro
• developing environments
  • Xcode version: 15.2
  • cmake version: 3.29.0

Steps to Reproduce:

1. After getting the Axmol project building and running on M1 Macs, update the Spine runtimes to the latest version off GitHub.
2. Add the Spine Cocos2d-x runtimes examples into the new Axmol project, and use the Spine Cocos2d-x runtimes AppDelegate.h instead of the one packaged with Axmol.
3. Change line # 60 from auto glview = director->getOpenGLView(); to auto glview = director->getGLView(); and line # 65 from director->setOpenGLView(glview); to director->setGLView(glview);
4. To better see this issue change line # 107 of AppDelegate.cpp from director->setAnimationInterval(1.0f / 60); to director->setAnimationInterval(1.0f); (This gives us longer to see the first frame failing to render properly.)
5. In AppDelegate.cpp include "RaptorExample.h", un-comment line # 113 and then comment out line # 114, this way the AppDelegate loads up directly into the example that uses two-color tinting.
6. Add the following lines to the RaptorExample.cpp before it's added as a child (e.g. between lines # 53 and # 54): skeletonNode->setDebugBonesEnabled(true); skeletonNode->setDebugMeshesEnabled(true); (This shows that the skeleton is loaded and should be rendered on the first frame, but isn't while two-color tint is enabled.)
7. Run the project on your arm64 Mac, and observe that the Spine-boy Raptor skeleton doesn't render anything except the debug bones and meshes for the first frame, and then renders correctly after that.
8. For posterity, comment out line # 52 of the RaptorExample.cpp class to disable two-color tinting here, and then re-build and re-run the project, observe that this time the Spine-boy Raptor skeleton properly renders on the first frame, as well as subsequent frames.

This bug also exists in Cocos2d-x v4.0, but not Cocos2d-x v3.17.2, likely due to all the Renderer, GL, and Shader changes made to support Metal on Apple devices.

I've submitted a bug report to the Esoteric Software team as well, but it's very possible that their fix will need to be back ported to Axmol from Cocos2d-x (it'd be nice if they started supporting Axmol officially since Cocos2d-x is dead, but I don't see that happening anytime soon).

I will mention that in my use case it sometimes takes 24 frames or more before the skeleton renders for the first time, and not a single frame like the repro steps I provided.

I've played around with trying to update the skeleton, and the SkeletonTwoColorBatch with deltaTime = 0.0f, I've tried to force flush for the lastTwoColorTrianglesCommand on each draw, I've tried disabling culling, setting the skeleton to its setup pose, and updating its world transforms, but nothing solves the issue.

This does bring up two additional Spine related bugs:

1. The version of Spine packaged with Axmol contains the following bug, and should be updated the latest source code from GitHub to fix this bug. This is a critical issue that causes the game to crash when attempting to load a skeleton that contains mesh deformations (which was a headache I had to deal with last week).
   (That one was my fault due to getting the Axmol 2.1.2 release and not the latest code from GitHub.)
2. CC_USE_CULLING is used in the SkeletonRenderer.cpp code for the Spine Cocos2d-x runtimes, and thus the Axmol runtimes without modifications, but CC_USE_CULLING is never defined to be an alias for AX_USE_CULLING. This causes the Spine runtimes to ignore the Axmol AX_USE_CULLING config option in the Configs.h file.
   (With the merge of Add CC_USE_CULLING Alias to cocos2d.h #1825, this issue is now fixed.)

[halx99]

You means update to 4.1 branch: https://github.com/EsotericSoftware/spine-runtimes/tree/4.1

EDIT: seems no changes since 4.1-54fac9d, so I assume need update to spine-runtimes latest 4.2 branch?

[TyelorD]

so I assume need update to spine-runtimes latest 4.2 branch?

No, not 4.2: I believe those runtimes are still in beta right now.
Edit: Nope, I guess Spine 4.2 was just release today...

Each major Spine runtime is incompatible with the previous and post versions. So the 4.1 runtimes aren't compatible with 4.2, 4.0, etc.

I just doubled checked though, and you're right, the 4.1 runtimes are up-to-date with the Spine GitHub: so I'm not really sure why the crash I was experiencing stopped once I updated the runtimes to the latest source off GitHub. It certainly seemed like it was because of this bug, but after double checking it does appear that the code fix for that bug is implemented into the Axmol spine runtimes.

Okay I just triple checked and in Axmol the commit that updates the Spine runtimes to 4.1-54fac9d was done on February 26th of this year, and the 2.1.2 Axmol Release was cut on February 24th of this year. I got Axmol from the 2.1.2 release: so that explains why I had the bug until I updated the runtimes.

Sorry, that one was my fault (though since it does fix a Spine related crash, perhaps a 2.1.3 is in-order soon?) Thanks for the correction.

[halx99]

Why reopen, does this issue still require any changes?

[TyelorD]

Why reopen, does this issue still require any changes?

I only closed it by accident. The main issue for this still exists, and that is that the two-color tint bug with Spine Skeletons that I outlined with the majority of this bug report.

[TyelorD]

An update on this bug report.

The team behind Spine, Esoteric Software investigated the issue and are pushing the fix back to the maintainers of Cocos2d-x (and/or it's forks, i.e. Axmol). They say that the Spine runtimes are working properly based on their findings and that the issue is with the Cocos2d-x two-color tint renderer itself.

I've copy and pasted badlogic's response below for brevity:

Two-color tinting is implemented with a custom batcher (SkeletonTwoColorBatch.cpp) which submits custom commands (TwoColorTrianglesCommand) to the Cocos2d-x renderer.

Following the repro steps for the RaptorExample, I've confirmed that on the first frame, the SkeletonRenderer actually submits render commands to the batcher, which submits them to the Cocos2d-x Renderer, which you can see here:

This happens in CCRenderer.cpp:371. These submitted commands are ultimately processed in Renderer::processRenderCommand()

I can also confirm that the commands are submitted to the GPU:

In subsequent frames, the exact same code paths are executed, but result in the commands actually being rendered.

Another indicator that we do everything correctly is that in our example, the Cocos2d-x backend reports the number of vertices and draw calls correctly for the first frame, despite nothing being rendered.

The second issue you described might well be a "follow-on" error from the original issue that causes the first frame not to be drawn.

I'm afraid I have no idea how this can be. On our end, we are not doing anything different between the first and subsequent frames. My guess is that this is a bug in Cocos2d-x with regards to custom rendering commands. As such, we might have to loop in the Cocos2d-X folks. (or fork maintainers) to fix this. I'm afraid I'm not familiar enough with Cocos2d-X internals to fix this myself.

So, ultimately, this needs to be passed to whoever is now maintaining Cocos2d-X (or a fork). I'm afraid on our end there is not much we can do, other than recommend to not use the two-color tinting renderer.

Hopefully this information is helpful to fix the issue. I attempted for about a day and a half and made no progress on this issue, I ended up disabling two-color tinting on our skeletons in the meantime. However this really sucks, since we use the two-color tinting system in order to telegraph specific attack types to our players...

[rh101]

@TyelorD Just for clarification. does this issue only occur when using the Metal renderer backend? The reason I ask is that I can't seem to reproduce this issue when using the OpenGL renderer.

Never mind! I forgot to turn on the debug bones and meshes to see them popup before the first frame of the textures. I see the issue now.

[rh101]

If you look at the spine two color tine fragment shader in Axmol, you see this:

#version 310 es

precision highp float;

layout(binding = 0) uniform sampler2D u_tex0;

layout(location = COLOR0) in vec4 v_light;
layout(location = COLOR1) in vec4 v_dark;
layout(location = TEXCOORD0) in vec2 v_texCoord;

layout(location = SV_Target0) out vec4 FragColor;

void main() {
    vec4 texColor = texture(u_tex0, v_texCoord);
    float alpha = texColor.a * v_light.a;
    FragColor.a = alpha;
    FragColor.rgb = ((texColor.a - 1.0) * v_dark.a + 1.0 - texColor.rgb) * v_dark.rgb + texColor.rgb * v_light.rgb;
}

Change the main() method to this, which ignores the v_light values:

void main() {
    vec4 texColor = texture(u_tex0, v_texCoord);
    float alpha = texColor.a;
    FragColor.a = alpha;
    FragColor.rgb = ((texColor.a - 1.0) * v_dark.a + 1.0 - texColor.rgb) * v_dark.rgb + texColor.rgb;
}

You should notice that the texture appears correctly on the first frame. The v_light and v_dark values don't seem to be valid for a certain period of time, so when v_light.a is multiplied in float alpha = texColor.a * v_light.a;, it ends up as 0, which is completely transparent. You can verify this by using allowing the v_light.rgb to be multiplied, but still ignoring the v_light.a:

void main() {
    vec4 texColor = texture(u_tex0, v_texCoord);
    float alpha = texColor.a;
    FragColor.a = alpha;
    FragColor.rgb = ((texColor.a - 1.0) * v_dark.a + 1.0 - texColor.rgb) * v_dark.rgb + texColor.rgb * v_light.rgb;
}

On the first frame, this appears:

You can see that v_light.rgb contains 0,0,0, and also that the texture is actually valid.

The color is set in SkeletonRenderer::draw(), so that may be the source of the problem, and not the SkeletonTwoColorBatch implementation.

EDIT: Looking into SkeletonRenderer::draw(), it does seem to be setting a color that isn't 0,0,0,0 on the first frame, being in this section of code (color = light, color2 = dark):

TwoColorTrianglesCommand *batchedTriangles = lastTwoColorTrianglesCommand = twoColorBatch->addCommand(renderer, _globalZOrder, texture, _programState, blendFunc, trianglesTwoColor, transform, transformFlags);

V3F_C4B_C4B_T2F *vertex = batchedTriangles->getTriangles().verts;
for (int v = 0, vn = batchedTriangles->getTriangles().vertCount; v < vn; ++v, ++vertex) {
    vertex->color = color4B;
    vertex->color2 = darkColor4B;
}

So it appears that the command is being updated correctly.

[rh101]

@TyelorD If you have a bit of time, can you please try this:

In SkeletonRenderer.cpp, around line 461, find this section of code for the two color tint:

} else {
   TwoColorTrianglesCommand *batchedTriangles = lastTwoColorTrianglesCommand = twoColorBatch->addCommand(renderer, _globalZOrder, texture, _programState, blendFunc, trianglesTwoColor, transform, transformFlags);

    V3F_C4B_C4B_T2F *vertex = batchedTriangles->getTriangles().verts;
    for (int v = 0, vn = batchedTriangles->getTriangles().vertCount; v < vn; ++v, ++vertex) {
        vertex->color = color4B;
        vertex->color2 = darkColor4B;
    }
}

Change it to this:

} else {
    V3F_C4B_C4B_T2F* vertex = trianglesTwoColor.verts;
    for (int v = 0, vn = trianglesTwoColor.vertCount; v < vn; ++v, ++vertex)
    {
        vertex->color  = color4B;
        vertex->color2 = darkColor4B;
    }
    lastTwoColorTrianglesCommand = twoColorBatch->addCommand(renderer, _globalZOrder, texture, _programState, blendFunc, trianglesTwoColor, transform, transformFlags);
}

Check if that fixes the issue for you. I can't tell if this affects the two color tinting though, since the Raptor example still looks the same, and I'm not even sure if the Raptor example has two color tinting on any of the slots.

[rh101]

Tested the tank Spine sample (tank-pro.json), which has 2 color tint enabled on the smoke for the "shoot" animation, and the two color tint is still working with the modification in my previous post.

[TyelorD]

Check if that fixes the issue for you. I can't tell if this affects the two color tinting though, since the Raptor example still looks the same, and I'm not even sure if the Raptor example has two color tinting on any of the slots.

Based on a quick smoke test, this solution does indeed appear to fix the issue I was experiencing. Our skeletons which use two-color tinting are also still tinting properly, while also rendering correctly on the first frame.

Thanks a ton @rh101, I assumed we'd have to engineer a different solution to the effect we were getting by using two-color tinting, but you seem to have saved the day!

I recommend submitting your solution as a PR as well :)

[rh101]

Based on a quick smoke test, this solution does indeed appear to fix the issue I was experiencing. Our skeletons which use two-color tinting are also still tinting properly, while also rendering correctly on the first frame.

Thanks a ton @rh101, I assumed we'd have to engineer a different solution to the effect we were getting by using two-color tinting, but you seem to have saved the day!

I recommend submitting your solution as a PR as well :)

Excellent! One thing to note though, this may not just be an issue with two color tints, since the code for the single color also has the same problem, but it may not have been obvious that the initial frame is incorrect, since the shader operates differently. The non-two color tint shouldn't be affected, even thought it does update the data after the render command is added to the queue, because the vertex data seems to be used at a different point in the rendering process. In the #1875 the code has been changed to change the behaviour for both normal and two color tint, just in case.

I noticed the post on the Spine repo, so I assume they'll be fixing it there themselves. The Axmol runtimes are here, and this issue is for all versions in that repo. @halx99 I'll create PR in Axmol to fix this, but do you want me to generate a PR for each of the different versions in the axmolengine/spine-axmol repo, or will you merge the changes when they fix them in the official run-time?

[halx99]

Based on a quick smoke test, this solution does indeed appear to fix the issue I was experiencing. Our skeletons which use two-color tinting are also still tinting properly, while also rendering correctly on the first frame.
Thanks a ton @rh101, I assumed we'd have to engineer a different solution to the effect we were getting by using two-color tinting, but you seem to have saved the day!
I recommend submitting your solution as a PR as well :)

Excellent! One thing to note though, this may not just be an issue with two color tints, since the code for the single color also has the same problem, but it may not have been obvious that the initial frame is incorrect, since the shader operates differently. The non-two color tint shouldn't be affected, even thought it does update the data after the render command is added to the queue, because the vertex data seems to be used at a different point in the rendering process. In the #1875 the code has been changed to change the behaviour for both normal and two color tint, just in case.

I noticed the post on the Spine repo, so I assume they'll be fixing it there themselves. The Axmol runtimes are here, and this issue is for all versions in that repo. @halx99 I'll create PR in Axmol to fix this, but do you want me to generate a PR for each of the different versions in the axmolengine/spine-axmol repo, or will you merge the changes when they fix them in the official run-time?

@rh101 I invite you as collaborator of repo: axmolengine/spine-axmol , you can push to fix all branches directly without PR.

[rh101]

@rh101 I invite you as collaborator of repo: axmolengine/spine-axmol , you can push to fix all branches directly without PR.

No worries, I'll update all the runtimes today.