Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3 versions of NVDA produce a scratch in the speech when finishing the end of a line #10185

Closed
hozosch opened this issue Sep 10, 2019 · 23 comments · Fixed by #14697 or #15195
Closed
Labels
bug component/audio NVDA's audio output (nvWave, issues with usb audio etc). good first issue github features these at https://github.com/nvaccess/nvda/contribute p3 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority z goodForNewDev (archived)
Milestone

Comments

@hozosch
Copy link

hozosch commented Sep 10, 2019

Steps to reproduce:

Open a dokument
Press NVDA+A for "say all"

Actual behavior:

You will find that NVDA scratches sometimes. Mostly at the end of a line, either a line you did with enter, or the automatic line change if one long line is too big for the screen.

Expected behavior:

NVDA shouldn't scratch a single time

System configuration

NVDA installed/portable/running from source:

portable

NVDA version:

alpha-18574,56f5fcd2

Windows version:

7 professional sp1 64 bit

Name and version of other software in use when reproducing the issue:

doesn't matter here

Other information about your system:

Other questions

Does the issue still occur after restarting your PC?

yes

Have you tried any other versions of NVDA? If so, please report their behaviors.

No. The latest stable obviously doesn't do that, since it's still running on python 2. However, I didn't try it with the threshold or any other version there may be on a different branch running python 3.

@zstanecic
Copy link
Contributor

zstanecic commented Sep 10, 2019 via email

@hozosch
Copy link
Author

hozosch commented Sep 10, 2019

O yes, so it seems, as I'm indeed using espeak, that it's an issue in the portation of espeak to python 3. Maybe we should open an issue on espeak NG?

@Brian1Gaff
Copy link

Brian1Gaff commented Sep 10, 2019 via email

@hozosch
Copy link
Author

hozosch commented Sep 10, 2019 via email

@Brian1Gaff
Copy link

Brian1Gaff commented Sep 10, 2019 via email

@zstanecic
Copy link
Contributor

zstanecic commented Sep 10, 2019 via email

@Brian1Gaff
Copy link

Brian1Gaff commented Sep 11, 2019 via email

@Adriani90

This comment has been minimized.

@hozosch
Copy link
Author

hozosch commented Sep 21, 2019

I should point out that it only happens with the compiled snapshots. I tested the prototype from source as well, and there, it didn't scratch, at least from what I can remember.

@LeonarddeR
Copy link
Collaborator

The thought that this may be caused by speech refactor isn't strange at all, there have been major changes on how speech synthesizers process speech as well as how say all works.
I've been able to reproduce this in applications where movement through text is laggy, such as in word documents with lots of data in it.

Cc @michaelDCurran

@hozosch
Copy link
Author

hozosch commented Dec 13, 2019

I'm now running 2019.3 beta 1 installed on windows 10 18363, and I still have that problem. It's whenever NVDA scrolls, and sometimes when it finishes a sentence. I think it's a speech refactor problem, because I was using an inofficial eloquence driver ported to python3 as well once, which also did it.

@zstanecic
Copy link
Contributor

zstanecic commented Dec 13, 2019 via email

@hozosch
Copy link
Author

hozosch commented Dec 13, 2019

I primarily use Espeak, and it still happens with this synth. In sapi4 and 5 it doesn't happen, I don't have the money to buy vocalizer.

@Adriani90
Copy link
Collaborator

Can you try on a different Laptop or PC? Can you reproduce it? Do you use multiple Sound Cards?
I am using NVDA 2019.3 Beta 1 on different PCs both with eSPeak and Eloquence as Wella s one core voices, both on Windows 10 and Windows 8.1. I cannot reproduce this Problem.

It might be a Sound issue from your Sound Card. Try also with different Headsets or Speakers.

@hozosch
Copy link
Author

hozosch commented Dec 13, 2019

I don't have any other OS or PC to test with. When I opened the issue, I had windows 7 on this machine I'm using now, and there it was also reproduceable. I also don't have any other sound card except the internal one with the realtec driver installed. Espeak is the only synth with which this happens, if we exclude that inofficial synth.

@zstanecic
Copy link
Contributor

zstanecic commented Dec 13, 2019 via email

@feerrenrut
Copy link
Contributor

I notice this issue on my machine also (with espeak). I'm suspicious of the way that we send audio to the sound device, there are much newer API's for this. It will require some investigation.

@feerrenrut feerrenrut added component/audio NVDA's audio output (nvWave, issues with usb audio etc). bug z goodForNewDev (archived) p3 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority labels Feb 3, 2020
@feerrenrut
Copy link
Contributor

I'm labeling this as goodForNewDev, since very little NVDA specific knowledge is required to investigate. I expect the problem is limited to how we send audio data.

@Adriani90
Copy link
Collaborator

@hozosch could you please test with last alpha? Is this still reproducible?

@jcsteh
Copy link
Contributor

jcsteh commented Aug 12, 2020

This has the same cause (and thus the same investigation is required) as I describe in #11061 (comment).

@hozosch
Copy link
Author

hozosch commented Aug 13, 2020

So it has! Well, as I expected, I can still reproduce it with the latest alpha version.

@seanbudd seanbudd added the good first issue github features these at https://github.com/nvaccess/nvda/contribute label Apr 30, 2021
michaelDCurran pushed a commit that referenced this issue Apr 30, 2023
NVDA's existing audio output code (nvwave) is largely very old and uses WinMM, a very old legacy Windows audio API. It is also written in pure Python, contains quite a few threading locks necessitated by WinMM, and parts of it have become rather difficult to reason about. There are several known stability and audio glitching issues that are difficult to solve with the existing code.

Description of user facing changes
At the very least, this fixes audio glitches at the end of some utterances as described in #10185 and #11061.
I haven't noticed a significant improvement in responsiveness on my system, but my system is also very powerful. It's hard to know whether the stability issues (e.g. #11169) are fixed or not. Time will tell as I run with this more.

Description of development approach
1. The bulk of the WASAPI implementation is written in C++. The WASAPI interfaces are easy to access in C++ and difficult to access in Python. In addition, this allows for the best possible performance, given that we regularly and continually stream audio data.
2. The WinMM code fired callbacks by waiting for the previous chunk to finish playing before sending the next chunk, which could result in buffer underruns (glitches) if callbacks were close together (Python 3 versions of NVDA produce a scratch in the speech when finishing the end of a line #10185 and Texts with multiple line spacings are voiced with NVDA + down arrow and voices crack #11061). In contrast, the WASAPI code uses the audio playback clock to fire callbacks independent of data buffering, eliminating glitches caused by callbacks.
3. The WinMM WavePlayer class is renamed to WinmmWavePlayer. The WASAPI version is called WasapiWavePlayer. Rather than having a common base class, this relies on duck-typing. I figured it didn't make sense to have a base class given that WasapiWavePlayer will likely replace WinmmWavePlayer altogether at some point.
4. WavePlayer is set to one of these two classes during initialisation based on a new advanced configuration setting. WASAPI defaults to disabled.
5. WasapiWavePlayer.feed can take a ctypes pointer and size instead of a Python bytes object. This avoids the overhead of additional memory copying and Python objects in cases where we are given a direct pointer to memory anyway, which is true for most (if not all) speech synthesisers.
6. For compatibility, WinmmWavePlayer.feed supports a ctypes pointer as well, but it just converts it to a Python bytes object.
7. eSpeak and oneCore have been updated to pass a ctypes pointer to WavePlayer.feed.
8. When playWaveFile is used asynchronously, it now feeds audio on the background thread, rather than calling feed on the current thread. This is necessary because the WASAPI code blocks once the buffer (400 ms) is full, rather than having variable sized buffers. Even with the WinMM code, playWaveFile code could block for a short time (nvwave.playWaveFile not fully async #10413). This should improve that also.
9. WasapiWavePlayer supports associating a stream with a specific audio session, which allows that session to be separately configurable in the system Volume Mixer. NVDA tones and wave files have been split into a separate "NVDA sounds" session. WinmmWavePlayer has a new setSessionVolume method that can be used to set the volume of a session. This at least partially addresses Ability to adjust volume of sounds #1409.
@nvaccessAuto nvaccessAuto added this to the 2023.2 milestone Apr 30, 2023
@CyrilleB79
Copy link
Collaborator

Reopening since WASAPI is not enabled by default anymore (#15172).

@CyrilleB79 CyrilleB79 reopened this Jul 25, 2023
@hozosch
Copy link
Author

hozosch commented Jul 25, 2023

I also noticed that this happens only at speeds faster than the default 30. I will need to make a more extensive test with vasapi enabled.

@seanbudd seanbudd modified the milestones: 2023.2, 2023.3 Jul 26, 2023
seanbudd added a commit that referenced this issue Jul 26, 2023
Reintroduces #14697
Closes #10185
Closes #11061
Closes #11615

Summary of the issue:
WASAPI usage should be reenabled by default on alpha so wider testing can occur

Description of user facing changes
WASAPI is re-enabled - refer to #14697 for benefits

Description of development approach
change feature flag default value to enabled
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug component/audio NVDA's audio output (nvWave, issues with usb audio etc). good first issue github features these at https://github.com/nvaccess/nvda/contribute p3 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority z goodForNewDev (archived)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants