[Superseded by #77154] Rework Particle Turbulence

[KdotJPG]

After encountering the latest Alpha snapshot and discovering @RPicster's great efforts on the Particle Turbulence PR, I found some areas for improvement and rework, to hopefully start this feature off on the best footing possible for the upcoming 4.0 release. (Updated description here following discussions further down)

The current implementation merged in #55387 uses directionally-scrolling 3D noise to create an evolving pattern in 3D. This is problematic because it forces the consideration of a scrolling direction, which creates unavoidable and often unfitting bias in resulting patterns. The included speed-random parameter is poised as a means to conceal these effects, but it's only a moderately-effective patch on top of a problem that doesn't need to exist in the first place. This PR replaces the scrolling 3D noise with 4D noise which varies in place over time.

Further, the included formula for curl noise is highly complex and is based on axis-biased arithmetic of uncertain mathematical origin. This PR replaces it with a more concise solution that uses fewer noise calls while also producing more natural effects.

These changes do break turbulence configs made using the existing dev snapshots!

[NEW] Full change set:

• Replaced 3D noise with 4D noise to enable in-place pattern variation.
• Changed turbulence_noise_speed_random so that it controls the noise evolution instead (pending feedback on desired implementation).

[OLD] Full change set:

• Exchanged scrolling 3D noise pattern variation approach with 4D noise that evolves in place, producing more convincing effects through more intuitive configuration. Replaced turbulence_noise_speed (vec3) with turbulence_noise_evolution_speed (float). 2D particles use evolving 3D noise.
  • I can also re-add turbulence_noise_speed (vec3) if it is decided that the scrolling is useful -- perhaps as turbulence_noise_directional_trend to be more explicit about its purpose and effect.
• Replaced the then-merged 3D curl formula with a simple and orientation-invariant cross(noiseGradientA, noiseGradientB) which produces a proper divergence-free flow field with many fewer steps.
  -Added a separate 2D case which uses an even simpler rotate90(noiseGradient) formula (paraphrased code).
• Replaced displaced-sampling-based noise gradient (partial derivative vector) computation with single-evaluation analytic derivative sampling.
• Replaced Perlin noise algorithm with more-directionally-unbiased-looking Simplex noise. Simplex is also easier to calculate the analytic derivatives for.
  • I used two octaves of noise per each gradient computation to remove / vary up the straight line segments some of the particles were following in the single layers. It also increases variation.
• Adjusted the curve of turbulence_noise_scale as it's passed to the shader, to better reflect the description in the documentation. shader_turbulence_noise_scale = 1.0 / turbulence_noise_scale;
• Removed turbulence_noise_speed_random since it no longer has its necessary use case.
• Removed turbulence_noise_strength since it had no effect on the particle update step, and its effect during the initialization step could be accomplished just the same by increasing turbulence_initial_displacement_min and turbulence_initial_displacement_max. Determined that it does produce an effect and re-added corresponding functionality though a turbulence_sharpness parameter.
  • It was also previously used in the definition of `adj_contrast, which no longer plays a role now that the partial derivative vector is computed analytically.
• Changed existing formula for turbulence_influence (from values sampled according to turbulence_influence_min and turbulence_influence_max), which biases the result towards more influence, to do so more cleanly. old, new https://www.desmos.com/calculator/hmsaly3f4b
  • I also adjusted the way the normalization happens so that the new vector is always the same length as the old.
• Fixed the distance-from-center selection in the existing EMISSION_SHAPE_SPHERE code -- it needs to be passed through a cube-root for a proper uniform distribution that doesn't cluster at the center.
  • This technically does change existing patterns, but I wouldn't generally expect it to break any. If I should revert it or implement it differently, just give me the word.

Motion to also remove Initial Displacement

Initial Displacement seems well-intentioned, but I haven't found any behavior in it that I couldn't replicate with the Emission Shape features. If there is consensus, I will update this PR with its removal. (@RPicster feel free to fill me in if I'm missing something.)

---

---

All code added is either MIT-licensed or self-authored.

[RPicster]

I'm in vacation for two weeks and won't have the time to properly review this PR.

I saw that there are a lot of changes and if it's ok I would like to review it in practice before big changes like this are getting merged.

There are a couple of things I found on the first look where I have some "feelings". But I like to properly review it and test it before saying anything.
I think I would rather have easier to maintain code instead of optimizing too much - talking about separate noise functions for 2D and 3D here. Let's please keep it 3D focused.
Especially because I have some things in the works to "fake" some 3D behaviour in 2D in the future and there, the Z translation would be necessary in 2D too.

One thing that I can tell already is, that the changes to the emission shapes should go into a separate PR.

And, please also keep in mind the necessary license changes in this PR. Where are the functions from? What is necessary for Godot and Godot users to respect the license?

[KdotJPG]

No worries. Thanks for taking the time to give the initial comment!

I went ahead and removed the 2D-specific code, as well as the changes to emission shape. I'll set up a second emission shape PR at some point.

All the code is either MIT or my own. To break it down:

• The base for Simplex noise 4D comes from the following file in its MIT-licensed repository: noise4D.glsl and its output partial derivative/gradient vector code is adapted from this other file there: noise3Dgrad.glsl
• The hash functions which replace the mod289/permute randomness come from this Shadertoy here, which declares itself with the MIT-license (webarchive link for ongoing verifiability). (white noise audio warning)

The new vertex gradient system between the hash and the noise code itself is self-authored. The 4D noise in the repository seemed to produce some diagonal patterns that changed when I rendered slices of the noise with different coordinate permutations, meaning the gradient directions were not well-distributed. The replacement here first distributes the points on the surface of a tesseract, then uses a two-stages of polynomial approximation to get something like the result of the 4D version of this. The 'magic numbers' come from integral regression solving in Desmos.

I also realized a chance to re-tune the turbulence_noise_scale curve to be more in line with the description, and perhaps what users might expect from the parameter. I replaced it with simple division, making the term 'scale' define a number proportional to the average distance between bumps and valleys in the noise.

Feel free to let me know of any more questions / concerns as they come up.

[clayjohn]

In general the changes look good to me. I have two primary concerns:

1. I am concerned that the API changes may be less-intuitive for artists. The original implementation was designed with artists in mind. That being said I am not in a position to speak about what is intuitive for others, the changes you mention make sense to me, so lets wait and see what @RPicster and @QbieShay think based on trying it out.
2. I share RPictster's concern that the simplex noise implementation may be overly complex. While I grant it is quite efficient and used widely, it is very hard for non-math people to understand. That being said, there are not many good 4D noise functions with analytic derivatives floating around, so we may not have much of a choice.

[clayjohn]

I am very sorry about that close/reopen. It was not intentional!

[KdotJPG]

All good! Figured it was that the moment I saw the cut-off comment.

Re: noise code complexity, I see where you're coming from, and definitely agree there's a learning curve to understanding the internals of this type of noise. What I do see as helping its cause in that regard is that it's fairly self-contained. While it takes space as a block of code that needs study to completely understand, in general you don't need to fully grasp it to write good code around it. I wouldn't consider it worth settling for a quality downgrade for a more-immediately-intuitive function here -- or really in most cases when paired with decent encapsulation principles, but I would certainly agree the code's readability as it is can be improved.

I've gone ahead and revised the noise code with clearer variable names and more verbose comments.

[Zireael07]

@clayjohn: Noise, in my experience, is the sort of thing you implement once and don't care afterwards. It's better that it's more performant even at the cost of readability. If you really want people to understand how noise is done, you could link in a comment to a more readable, unoptimized implementation "if you want to understand noise look it up at blah blah"

[RPicster]

@clayjohn: Noise, in my experience, is the sort of thing you implement once and don't care afterwards. It's better that it's more performant even at the cost of readability. If you really want people to understand how noise is done, you could link in a comment to a more readable, unoptimized implementation "if you want to understand noise look it up at blah blah"

By the way, can you provide real life comparisons of particle systems with a performance gain that can give a good reason to change the simple functions with the more complex ones?

You pointed out that my original approach uses more calls, but how do the two approaches compare in terms of real world performance? It would be very interesting to see what differences we're talking about here.

[Zireael07]

Unsure if you're asking me or clayjohn, as I haven't used the turbulence (yet) but I plan on using it to make my tire smoke less uniform and performance will definitely be a concern.

From my experience testing/looking up shaders on Shadertoy, the number of noise lookups can make or break a shader. For instance, if you look up 'gas giant' or 'planet', many of those have very many lookups even to the point of freezing/crashing webgl. When implementing my own gas giant shader, I went for less 'realistic' but less lookups because I fully expect star systems with multiple giants and I don't want my performance to suffer (it's already suffering because in 3.5 there are no instance uniforms/parameters, so to have different individual planets I need to make them unique/local to scene so no instancing)

[RPicster]

I just want to avoid changing things when there is no problem ( https://docs.godotengine.org/en/stable/community/contributing/best_practices_for_engine_contributors.html ).

Me and several other people tested the turbulence with millions of particles and never had any performance problems - so it would be great to see the real benefit before changing things "just because".

[KdotJPG]

Performance isn't a huge concern, I agree. I ran similar tests with the largest value suggested by the slider and found that rendering was far more of a bottleneck than the particle updates were. There was no perceptible difference in framerate in either system when zoomed out far enough.

The main importance here would be the 4D noise and the evolution speed parameter. Scrolled 3D noise limits the space of parameters that produce patterns with more desirable properties and fewer undesirable ones -- namely a visible scrolling direction, which won't always be part of the artist's vision. The problem with the current solution is that it forces artists to consider a scrolling direction, when they shouldn't have to given the PCG tools we have to address it -- i.e. 4D noise. Scrolling same-dimensional noise can make sense where, for example, you're limited to textures and don't have the headroom to compute noise in real time. But here that isn't a constraint, and doing so only limits artists. Pattern evolution, if you accept these points, really should use 4D noise.

From there we'd need to debate the specifics of the implementation. We could simply upgrade the existing noise function to 4D, add the evolution parameter, and make as few changes to the code otherwise as possible. That would work, but I wouldn't say the rest of the changes in this PR are without their own rhymes and reasons.

The existing curl formula is more complex and less readable than it needs to be to achieve results of quality at least as good. From what I can gather, it effectively takes the gradients of three noise maps, rotates them each 90 degrees in different directions, projects them each to one of the three cardinal planes, and adds the results together. The rotations and cardinal plane projections seem arbitrary if there aren't any underlying principles I'm overlooking, and I'm not sure this merged solution is necessarily better for the purpose than the solution I propose. This cross-product formula here looks good, feels more intuitive, and has a solid basis in mathematics. I also find that it's more flexible for creating a variety of different patterns. It creates stronger flow-lines at high influence values, and patterns more similar to the current system at lower influence values. There is a subtle difference in character, I will grant, but my experience tells me that it's more technically sound and will offer artists more freedom.

Re: the noise algorithm conversation, the existing function could be extended to 4D. But if we do proceed with 4D noise, I would hold a Simplex noise solution to be the way to go for the reasons covered thus far. It's easier to compute a derivative for, it's more directionally-even to the eye, and has widely-used MIT implementations available that can be molded into exactly what we need. The code itself is more complex, but as @Zireael07 notes we can link an external resource -- for example Simplex Noise Demystified.

I agree that it's not sustainable to accept arbitrary changes. But the changes here do solve problems that users of this system will encounter through normal use -- especially pattern evolution. And it's better, the way I see it, to address these before release than after.

[RPicster]

Thanks for the great answer! I understand it much better now! I will test this PR extensively from the artist point of view when I return from my vacation! 👍

[KdotJPG]

Just rebased on upstream/master, and addressed a method call documentation alphabetization order that seemed as though it may have been related to the workflow failure, so it should just need your review and verification when that time comes.

Secondly, I have created an alternative solution in this branch here that uses FastNoiseLite-style domain warp simplex noise to create the vector field instead. It creates a more ribbon-like appearance in the flows. If you choose to test both and prefer this other solution, just give me the word and I'll rebase it into the branch for this PR.

[Calinou]

@Calinou Are there any known FOSS tools that can generate vector field textures?

Any 3D texture can be used, but dedicated tools for generating those are rare. You can import a 3D texture from any 2D image in Godot using layers (each row/column image is a different layer).

[RPicster]

Alright! So I gave the PR a lot of testing. Went in pretty deep with the two versions side by side.

I have to say that the PR is a definitive improvement, especially the coherence when the turbulence is animated!

@KdotJPG great work ♥

Sorry that it took that long 🙈 But I really wanted to make sure this is an improvement and that people / artists can create anything they can imagine. I think it's very powerful now.

By the way - in the current state of the PR I think it's safe to remove the breaks-compat tag.

[KdotJPG]

@RPicster Sounds great! Thanks for your in-depth review!

I've gone ahead and fixed the lingering spacing issues + done a final method naming tweak. Should we keep turbulence_noise_speed_random as the name of the 4D evolving parameter?

And regarding the removal of breaks-compat, would we be ok "sneaking" this in to 4.0 now? That would be the best in my eyes, powers willing, but if we need to wait for 4.1 then that's how it is.

[akien-mga]

4.0 is at release candidate stage and fully feature frozen, so no, this wouldn't be merged right now. But we can merge soon after the release to get it well tested for 4.1.

[RPicster]

As long as no names are changed, this PR will not break compat, so I think we should keep names as is.
To be honest, I think most people won't notice the change - and if they do, it will be a positive suprise that evolution of patterns is more consistent and better looking.

[Calinou]

Does someone have a testing project with particle turbulence setups, so we can compare performance between master and this PR? See #76501.

[Calinou]

Testing project: test_particle_turbulence.zip

I've tested this PR rebased against master in a fully optimized editor build (optimize=speed lto=full). Overall, this PR seems to have no noticeable performance impact – there are minor differences here and there, but they're within margin of error. Very high particle counts (1 million) seem to perform very slightly better, but this may also be due to testing variance.

Benchmark

OS: Fedora 37
CPU: Intel Core i9-13900K
GPU: GeForce RTX 4090 (NVIDIA 530.41.03)

1152×648

Amount	Turbulence OFF	Turbulence ON (master)	Turbulence ON (this PR)
10	7962 FPS (0.13 mspf)	5210 FPS (0.19 mspf)	5232 FPS (0.19 mspf)
100	7855 FPS (0.13 mspf)	5210 FPS (0.19 mspf)	5208 FPS (0.19 mspf)
1,000	7836 FPS (0.13 mspf)	5197 FPS (0.20 mspf)	5189 FPS (0.20 mspf)
10,000	7475 FPS (0.13 mspf)	4999 FPS (0.20 mspf)	4994 FPS (0.20 mspf)
100,000	4882 FPS (0.20 mspf)	3649 FPS (0.27 mspf)	3757 FPS (0.27 mspf)
1,000,000	950 FPS (1.05 mspf)	889 FPS (1.13 mspf)	920 FPS (1.09 mspf)

3840×2160

Amount	Turbulence OFF	Turbulence ON (master)	Turbulence ON (this PR)
10	5458 FPS (0.18 mspf)	1397 FPS (0.72 mspf)	1401 FPS (0.71 mspf)
100	5440 FPS (0.18 mspf)	1383 FPS (0.73 mspf)	1392 FPS (0.72 mspf)
1,000	5437 FPS (0.18 mspf)	1380 FPS (0.73 mspf)	1392 FPS (0.72 mspf)
10,000	5254 FPS (0.19 mspf)	1375 FPS (0.73 mspf)	1379 FPS (0.73 mspf)
100,000	3664 FPS (0.28 mspf)	1239 FPS (0.80 mspf)	1244 FPS (0.80 mspf)
1,000,000	844 FPS (1.18 mspf)	575 FPS (1.74 mspf)	591 FPS (1.69 mspf)

Regardless of this PR, it's interesting to see the mspf difference between turbulence disabled and enabled becomes much greater at higher resolutions. I thought particle shaders' performance weren't influenced by viewport resolution, since they run once for every particle and not for each pixel. Besides, in the testing project, each particle is only a few pixels tall at 3840×2160.

[QbieShay]

That's indeed bizarre @Calinou , @clayjohn do you have any insight on this?

[QbieShay]

This is good to go for me , just needs a rebase on latest master

[akien-mga]

Superseded by #77154. Thanks for your contribution!