AudioServer::get_output_latency() and audio "output latency" setting are inaccurate

[benjarmstrong]

Godot version: 4.0, 3.X (possibly 2.X, haven't checked)
OS/device including version: All
Issue description:
AudioServer::init() contains this line of code:
buffer_size = 1024; //hardcoded for now
As a result mixing is always performed at this fixed buffer size. This is problematic because it enforces a minimum of 1024 frames of audio delay. At the default engine settings (15 MS audio output latency and 44100Hz mix rate) this is approximately 23 MS (1000 * 1024 / 44100).

On a related note I believe this would adversely affect the performance of the audio server when the audio driver is running at any buffer size that isn't 1024, since audio callbacks need to remain relatively predictable in their performance to remain avoid buffer underruns and remain glitch-free. For example at a driver buffer length of 256:

• AudioDriver outputs 256 frames; AudioServer mixes 1024 frames and gives 256 to AudioDriver
• AudioDriver outputs 256 from last remaining AudioServer mix frames
• AudioDriver outputs 256 from last remaining AudioServer mix frames
• AudioDriver outputs 256 from last remaining AudioServer mix frames
• AudioDriver outputs 256 frames; AudioServer mixes 1024 frames and gives 256 to AudioDriver
• AudioDriver outputs 256 from last remaining AudioServer mix frames
• AudioDriver outputs 256 from last remaining AudioServer mix frames
• AudioDriver outputs 256 from last remaining AudioServer mix frames
• etc.

As a result glitches are much more likely on the frames that mixing occurs rather than the frames being distributed evenly among every driver output.

Steps to reproduce:
Testing inaccuracies in audio delay can be difficult for small values but be easily observed at larger values. The issue can be observed easily by changing a line of code in AudioServer::init() from
buffer_size = 1024; //hardcoded for now
to
buffer_size = 44100; //hardcoded for now
and compiling. At a mix rate of 44100 (the default) there will be about 1 second of audio output latency. Likewise this number can be lowered and latency is reduced. AudioServer::get_output_latency() will return the same value in all cases (unless the audio driver configuration has changed).

Minimal reproduction project:
Follow the steps to reproduce, create a new project and add this script to an AudioStreamPlayer:
extends AudioStreamPlayer func _ready(): print("Output latency = ", AudioServer.get_output_latency()) func _input(event): if event.is_action_pressed("ui_accept"): playing = true
Run the project and the delay can be audibly observed by pressing space.

Potential fix:
Rather than hardcoding a buffer size of 1024, this could be implemented as a virtual AudioDriver::get_buffer_size() method to inform the audio server on how big to make the buffers. This would also keep the driver and server in sync potentially leading to better performance, but would require changes to audio drivers to implement it. A fallback mixing buffer size of 1024 could be implemented for the AudioServer in the event that the active driver doesn't implement this method, allowing it to be introduced incrementally to audio drivers.

I currently require low latency audio for a project I'm working on and will most likely be implementing this in the near future. I'll update this issue with a PR when I'm done.

[Calinou]

This makes sense. I've been able to use a buffer length of 768 or even 512 in some games to decrease sound latency, and I'd like to do the same in Godot. Thanks for carrying out this thorough research 🙂

Also, should we add a command-line argument to change the audio buffer length like --audio-buffer-length <samples>? This way, you can change it in Godot games that don't expose a setting to do so (like most gamejam games).

[benjarmstrong]

@Calinou

This makes sense. I've been able to use a buffer length of 768 or even 512 in some games to decrease sound latency, and I'd like to do the same in Godot. Thanks for carrying out this thorough research 🙂

Not a problem 👍 I'm working on a project that is heavily based on rhythm and player-generated sound, so I need to figure out how to make the audio as responsive as possible.

Also, should we add a command-line argument to change the audio buffer length like --audio-buffer-length <samples>? This way, you can change it in Godot games that don't expose a setting to do so (like most gamejam games).

While I personally like the idea I don't know if the --audio-buffer-length argument would be considered a feature Godot needs based on 'best practices for engine contributors' guide. It could be argued that it doesn't need to be a core feature since it could be implemented in GDScript via OS.get_cmdline_args(), setting the audio output latency and restarting the game, and this could be in the asset lib. If it were implemented then --audio-output-latency might be a better name to be consistent with the name in the project settings.

At any rate I think modifying the audio server internals would provide a huge boost in responsiveness to the point where audio latency isn't much of an issue moving forward. The default engine setting of 15ms of audio latency (662 samples at 44100Hz) is sufficient for most games, and far less noticeable than the 23ms minimum it is achieving at the moment.

[Birdulon]

Re: Inaccuracy of AudioServer::get_output_latency() - a miserable implementation detail for audio_driver_pulseaudio is that it caches the first valid response and then never updates it again. This makes it nearly useless for debugging because the underlying PA call actually does change considerably, as evidenced by playing around in a non-trivial project with the if (latency == 0) { and matching brace commented out from https://github.com/godotengine/godot/blob/3.2/drivers/pulseaudio/audio_driver_pulseaudio.cpp#L303-L322 .

Prior to investigating this, I constantly saw my latency figure hover around 50ms unless I had breakpoints in my initialisation, which resulted in ~30ms like that screenshot. I had been considering refactoring my init code to see if it prevented what I interpreted as the game receiving a bad PulseAudio latency, based on this lie from a function that would be better served with a documentation note saying "avoid calling this too often for performance reasons" rather than a silent "optimisation" that totally poisons a performance metric. (And yes it is a critical performance metric for me as I am writing a rhythm game, and it's not solely debug as reliable figures from this would allow better input compensation)

[Calinou]

@Birdulon Check if this is fixed by #38280. If not, feel free to open a pull request for this 🙂

[Birdulon]

The specific function appears untouched in that PR, haven't looked into the additional API provided in it though.

[Crystalwarrior]

Any updates on this? We ran into this issue with just trying to replicate Ace Attorney blips which we expected to be extremely basic but is giving us hell. The reference video we're using for replicating that is https://www.youtube.com/watch?v=Min0hkwO43g
And in Godot the audio playback ended up being completely random and hard to figure out the culprit other than determining it must be audio buffer and inconsistent latency. We also encountered the issue of being unable to use the provided functions to introduce delay which further makes it impossible to develop any kind of sound-precision on even the basic audio presentation level.