Implement semi-fixed timestep / manual stepping

[lawnjelly]

Describe the problem or limitation you are having in your project:
Godot currently only supports fixed timestep. While this is my preferred method, in most cases it requires the use of fixed timestep interpolation in order to prevent jitter due to aliasing between physics ticks and frames.

This interpolation is now supported via godotengine/godot#30226 ,
example addon here: https://github.com/lawnjelly/smoothing-addon

During the development of the above, a simpler alternative strategy was also discussed (used by default in some engines, e.g. Unreal) to overcome this same problem of physics ticks / frame synchronisation - the use of semi-fixed timestep. This can be simpler to work with, particularly for beginners and game jams, and can provide a more responsive input experience in certain circumstances.

On the other hand, semi-fixed can suffer from lack of deterministic behaviour. This can make debugging, testing and QA difficult in some types of game (hence why I personally prefer using fixed timestep interpolation). This is a trade off.

Anyway in the interests of a rounded approach to the problem I investigated semi-fixed as well as fixed.

Describe how this feature / enhancement will help you overcome this problem or limitation:
Semi-fixed time step overcomes the need to use fixed timestep interpolation. Semi-fixed timestep can be used to limit the problem of physics 'explosion' due to too high deltas, when stepping physics by frame deltas, and can also be used to lock physics ticks to frames, or the frame rate.

Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:
I've already implemented semi-fixed using a hard coded path.

Semi-fixed logic is something like this:

float MAX_PHYSICS_DELTA_MS = 16.0f;

main::frame_iteration(float deltaMS)
{
// how many whole ticks
int nPhysicsTicks = floor(deltaMS / MAX_PHYSICS_DELTA_MS);
for (int n=0; n<nPhysicsTicks; n++)
    _physics_process(MAX_PHYSICS_DELTA_MS);

// always do a fractional physics tick
float leftMS = deltaMS - (nPhysicsTicks * MAX_PHYSICS_DELTA_MS);
_physics_process(leftMS);

// frame update
_process(deltaMS);
}

This is the semi-fixed timestep selectable in project settings (note that delta smoothing is not part of this PR):

If we do decide to add semi-fixed, it is notable that it can either be hard coded (as I have done already), or implemented as a customizable callback in e.g. gdscript.

func iteration(frame_delta):
    for i in range (4):
        Engine.step_physics(0.02)
 
    Engine.step_frame(frame_delta)

A customizable approach also has the potential for a tie in to solve the issue of the desire to manually step the physics in network games, both at the server and the client:
godotengine/godot#25068

Describe implementation detail for your proposal (in code), if possible:
I've already implemented semi-fixed timestep, as a hard coded solution, selectable from project settings:
godotengine/godot#30798

Alternatively custom manual stepping can be implemented as a callback (I've already done this in another area for delta smoothing), which also has the potential to provide a mechanism to allow manual stepping for multiplayer. However this would require some investigation because although running the main iteration from a callback is feasible, multiplayer may better be accomplished by allowing stepping from within _process during the frame update, which may or may not be feasible.

If this enhancement will not be used often, can it be worked around with a few lines of script?:
No, in both cases this would need core support.

Is there a reason why this should be core and not an add-on in the asset library?:
It cannot be implemented as an add-on.

Extra
I originally wrote the PR before godot proposals, but it seems a good idea to discuss the whole area here, as there must be overall support of the idea if we are to go ahead with it (or similar).

There are 3 possible options here:

1. Continue to only support fixed timestep
2. Add semi-fixed hard coded as an option in addition to fixed
3. Add customizable timestepping (possibly with a tie in for the multiplayer issue)

Probably strangely for a 'proposer', I am equally happy with any of these. It really boils down to the Godot mission statement, where we want to go with the engine - become highly focused for making single player games via a common method, or make it more adaptable. This involves trade offs, more options can bring in greater complexity and surface for bugs.

Realistically, if we did add semi-fixed I would tend towards the KISS principle, keep it simple stupid and go for the hard coded approach. I think most people for whom semi-fixed would be useful would be far more likely to use it if they simply had to switch it on in project settings, then forget about it, rather than write or copy some custom scripts.

Addendum

Just to add a little as we may get to discuss this soon:

Fixed to refresh rate

Another additional option that reduz is keen on, which is changing the fixed tick rate at runtime to match the refresh rate of the monitor.

This has some advantages - it is simple to use and does not require interpolation. On the downside, it means that game behaviour will be different on different machines, and may not play nicely with variable refresh rate monitors.

Delta smoothing

For best results with interpolation and semi fixed it can be a good idea to consider delta smoothing as an additional step. This is an attempt to compensate for the sources of error in making delta measurements to drive timesteps. This is fairly easy to implement and I got this working while I was working on the timestepping last year (both hard coded and with custom script interface), and is fairly simple to add, I didn't make a PR because I was waiting for decisions on timestepping.

Fixed timestep without interpolation offers some insulation against this problem. There are also some newer APIs in vulkan and android for improving frame timing information.

[realkotob]

Note that Unreal engine uses semi-fixed timestep and the default is to not have fixed timestep i.e. smooth physics is default, probably for the concerns mentioned above. "Physics Sub-step" has to be enabled from the settings (and it makes code a lot harder to work with unfortunately).

It's worth having and enabling by default in godot, but I'd still like the alternative to be as easy to implement/work with as it is now.

Look at this article for the devs perspective, specifically the "Why doesn’t UE4 use a fixed timestep?" section:

We actually had a debate about these two techniques and eventually decided on semi-fixed, and here’s why:

If you use Free the physics you have to use a timebank. If you want to tick physics at 60fps and you have a frame that took a little bit more than 1⁄60 you will need to have some left over time. Then you will tick the physics engine with the perfect 1⁄60 delta time, leaving the remainder for the next frame. The problem is that the rest of the engine still uses the original delta time. This means that things like blueprint will be given a delta time of 1⁄60 + a bit.

You can imagine a case where a user would want to use delta time in blueprint to determine the location of an object that’s moving at some constant speed by saying “new position = old position + speed * delta time” In such a case the user will expect the object to move speed * delta time. However, since delta time is longer the the time we tick a single physics frame, we would need to either brake the movement into two physics ticks to maintain the right speed, or we would have to place the object in the new expected position at the end of the frame, but increase speed. Either way the result isn’t what the user would expect.

Getting the rest of the engine to use this new delta time would affect many systems, and so we ultimately decided to go with semi fixed.

I personally fundamentally dislike how they architected the fixed timestep interface, but they have some valid points about why semi-fixed timestep should be an option.

[lawnjelly]

It seems like the Unreal guys may have made a historical architectural mistake by assuming semi-fixed, which made it more difficult to retrofit fixed timestep.

Godot and Unity are both far more agnostic in this respect and allow you to sensibly process things in either the fixed or frame update.

Luckily for us, implementing semi-fixed AFTER fixed makes things a lot easier. The only gotcha with the move to semi-fixed is that you have to start using the delta in the _physics_process function, if you weren't using it before:

i.e. you have to use:

pos += velocity * delta

rather than

pos += velocity

On the other hand, we missed a trick by not having built in support for fixed timestep interpolation from the get go. My smoothing addon works to allow this through an extra node, but it would have been neater and easier to use (particularly for beginners) if it was a built-in feature of Spatial. This would have made us a class leader in terms of timestepping. I actually did investigate this, and got it working, however it made the code too spaghetti because Spatial etc had not been designed from the start for this kind of use.

[realkotob]

@lawnjelly It would be worth breaking compat in 4.0 (or 4.1 so majority of people can get Vulkan without breaking physics) if it means we can get this working natively without any messy hacks.

Physics is one of those areas where you really don't want to do a lot of hacking to rewrite yourself, and is one of the few things I actively hate about other engines.

I think pos += velocity is fundamentally silly anyway and I can live with forcing new users to learn how to use delta, it's good practice and delta is needed in many other areas of game programming as well. Using delta is not limited to godot obviously, it's even in the pygame tutorials so even many beginners will already be familiar with it.

[lawnjelly]

@lawnjelly It would be worth breaking compat in 4.0 (or 4.1 so majority of people can get Vulkan without breaking physics) if it means we can get this working natively without any messy hacks.

The semi-fixed PR as is works seamlessly, is backward / forward compatible and there are no hacks needed, you merely change the option from Fixed to Semi-Fixed in project settings, that's it.

Physics is one of those areas where you really don't want to do a lot of hacking to rewrite yourself, and is one of the few things I actively hate about other engines.

Using fixed timestep interpolation on the other hand is a little more involved than it could be, as it has to work within the existing framework. I tried to make this as easy to use as possible with the addon. Last time it was discussed, reduz was against having fixed timestep interpolation in core.

None of the timestepping various options should have to break compatibility, afaik, I had them all working in 3.1 months ago.

[starry-abyss]

have to start using the delta in the _physics_process function

As far I understand, you should use delta anyway, otherwise changing tick rate breaks the code (it can be different per project settings even if constant in the same run)

[AndreaCatania]

Fixed physics delta time is really useful for physics stability and for networking (since it's the first thing to have to get a deterministic time step).

All games with such requirements can't benefit of the semi-fixed approach and so I've proposed the object interpolation approach, that solves the position problem while keeps physics delta time fixed: #671

[dacioromero]

I'm interested in manual stepping for a multiplayer physics based game that I'm working on. From some GDC talks that I've watched it seems like this would be very useful for that scenario.

[The-Randalorian]

Just stating this is something I would find extremely useful, particularly for networked physics. It would be a great help for building multiplayer games.

Definitely should be on the roadmap at some point. (4.1 seems a little far, but maybe not when compatibility is concerned.)

[lawnjelly]

@fakhraldin I think this is referring to frame pacing, which is a related, but orthogonal issue.

EDIT:
Please ask on rocket chat, this is going off topic.