Skip to content

Commit

Permalink
Support for audio output using WASAPI (#14697)
Browse files Browse the repository at this point in the history
NVDA's existing audio output code (nvwave) is largely very old and uses WinMM, a very old legacy Windows audio API. It is also written in pure Python, contains quite a few threading locks necessitated by WinMM, and parts of it have become rather difficult to reason about. There are several known stability and audio glitching issues that are difficult to solve with the existing code.

Description of user facing changes
At the very least, this fixes audio glitches at the end of some utterances as described in #10185 and #11061.
I haven't noticed a significant improvement in responsiveness on my system, but my system is also very powerful. It's hard to know whether the stability issues (e.g. #11169) are fixed or not. Time will tell as I run with this more.

Description of development approach
1. The bulk of the WASAPI implementation is written in C++. The WASAPI interfaces are easy to access in C++ and difficult to access in Python. In addition, this allows for the best possible performance, given that we regularly and continually stream audio data.
2. The WinMM code fired callbacks by waiting for the previous chunk to finish playing before sending the next chunk, which could result in buffer underruns (glitches) if callbacks were close together (Python 3 versions of NVDA produce a scratch in the speech when finishing the end of a line #10185 and Texts with multiple line spacings are voiced with NVDA + down arrow and voices crack #11061). In contrast, the WASAPI code uses the audio playback clock to fire callbacks independent of data buffering, eliminating glitches caused by callbacks.
3. The WinMM WavePlayer class is renamed to WinmmWavePlayer. The WASAPI version is called WasapiWavePlayer. Rather than having a common base class, this relies on duck-typing. I figured it didn't make sense to have a base class given that WasapiWavePlayer will likely replace WinmmWavePlayer altogether at some point.
4. WavePlayer is set to one of these two classes during initialisation based on a new advanced configuration setting. WASAPI defaults to disabled.
5. WasapiWavePlayer.feed can take a ctypes pointer and size instead of a Python bytes object. This avoids the overhead of additional memory copying and Python objects in cases where we are given a direct pointer to memory anyway, which is true for most (if not all) speech synthesisers.
6. For compatibility, WinmmWavePlayer.feed supports a ctypes pointer as well, but it just converts it to a Python bytes object.
7. eSpeak and oneCore have been updated to pass a ctypes pointer to WavePlayer.feed.
8. When playWaveFile is used asynchronously, it now feeds audio on the background thread, rather than calling feed on the current thread. This is necessary because the WASAPI code blocks once the buffer (400 ms) is full, rather than having variable sized buffers. Even with the WinMM code, playWaveFile code could block for a short time (nvwave.playWaveFile not fully async #10413). This should improve that also.
9. WasapiWavePlayer supports associating a stream with a specific audio session, which allows that session to be separately configurable in the system Volume Mixer. NVDA tones and wave files have been split into a separate "NVDA sounds" session. WinmmWavePlayer has a new setSessionVolume method that can be used to set the volume of a session. This at least partially addresses Ability to adjust volume of sounds #1409.
  • Loading branch information
jcsteh authored Apr 30, 2023
1 parent b93fc54 commit e77bde9
Show file tree
Hide file tree
Showing 12 changed files with 1,023 additions and 31 deletions.
11 changes: 11 additions & 0 deletions nvdaHelper/local/nvdaHelperLocal.def
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,14 @@ EXPORTS
getOleClipboardText
_nvdaControllerInternal_reportLiveRegion
_nvdaControllerInternal_openConfigDirectory
wasPlay_create
wasPlay_destroy
wasPlay_open
wasPlay_feed
wasPlay_stop
wasPlay_sync
wasPlay_pause
wasPlay_resume
wasPlay_setSessionVolume
wasPlay_startup
wasPlay_getDevices
1 change: 1 addition & 0 deletions nvdaHelper/local/sconscript
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ localLib=env.SharedLibrary(
"UIAUtils.cpp",
"mixer.cpp",
"oleUtils.cpp",
"wasapi.cpp",
],
LIBS=[
"advapi32.lib",
Expand Down
Loading

0 comments on commit e77bde9

Please sign in to comment.