Can anything be done about Crackling/Static ocassionally when playing?

[MDoder97]

Expected Results

No crackling.

Observed Results

Crackling every 10-30 seconds on most tabs

Steps to Reproduce

Playing most tabs will produce this.
It seems to be a rendering / lag issue. If you point me in the right direction I'll have a look at the source too.

Also I'd be thankful if you could feel me in on this question -> #553

Thanks.

[Danielku15]

I cannot confirm that there is a general bug in alphaTab which would cause this. To me it rather sounds your specific setup is suffering of performance issues (your device or your page). I recommend opening a Discussion where you provide more insight on your precise setup and tests you have done.

[MDoder97]

I should have been more detailed.
This actually happens on the default implementation you provided as well.
Try this tab for example in the default setup - https://firebasestorage.googleapis.com/v0/b/guitarflow-95c71.appspot.com/o/tabs%2FRed%20Hot%20Chili%20Peppers%20-%20Californication.gp3?alt=media&token=21ce9eb8-eee8-4173-a595-a9a5f861f3da

It should happen somewhere in the first 2-3 lines
Try this and let me know if you hear it.

[MDoder97]

@Danielku15 I started debugging this but as I said, even the copied barebones implementation you provided cause this so I'm not sure how to do profiling here. Similar static can be heard sometimes when looping certain parts the moment cursor snaps back. I can send you a video from different devices I'm using so you can see what I'm talking about if you want.
I'm in if you need a hand to figure it out.

Also, I didn't want to open another discussion to ask this but, do you you have any resource for good electric guitar soundfonts viable for alphaTab? As in, not 20mb in size but more like 1.5-2max? I've been scouring the net but the only good ones are also really large, the one you provided turns out to be the most versatile.

[Danielku15]

Videos will not allow me to analyze the problem. I recommend you do some research on how to do javascript profiling and then do some tests on your environment. Here some links to get started:

https://developers.google.com/web/tools/chrome-devtools/rendering-tools
https://developer.mozilla.org/en-US/docs/Tools/Performance
https://blog.appsignal.com/2020/02/20/effective-profiling-in-google-chrome.html

Also, I didn't want to open another discussion to ask this but

Please exactly do that. Keep one discussion about one topic.

[Danielku15]

@MDoder97 Were you able to find out anything more in your tests? I would be eager to find out what is causing this performance issues for you. I did some research but without narrowing down the root cause any improvements will be hard to do.

There might be tricks like described here. If the cursor animations are burning too much CPU, a CSS style might move them to the GPU.

.at-cursors {
    transform: translateZ(0);
}

Or we can make a feature to disable animations and only have a "ticking" cursor. We could also try to detect high CPU usage and try to make things less CPU intense (e.g. disable or reduce animation frames, switch non-smooth scrolling).

The will-change property might boost as well some things like:

.at-cursor-bar, .at-cursor-beat {
    will-change: left, top;
}

Or moving from a left/top to a translate approach might also be faster.

But these are all proposals related to the scrolling and animations. If the actual audio synthesis is to blame. Maybe a change of buffer sizes might do the trick.

Experience-wise strange audio noises are coming from the fact that the main UI thread is overloaded (from other things) and then the Web Audio API behaves weird. Audio Worklets might solve this for modern browsers on HTTPs.

[Danielku15]

Ping @MDoder97 : You got any more insights already on this topic?

[MDoder97]

@Danielku15 Hey Daniel, sorry for not responding. I've been completely swamped with work lately so I haven't been able to test it yet. I'll try out everything you mentioned here and let you know, hopefully in the next week or so.

[Reiji777]

@Danielku15 Not sure if this is what exactly @MDoder97 is experiencing: on my end when accessing the tutorial on the official website, the tab can't be played smoothly on one of my Android devices, meaning I could hear the glitches here and there, and also the tab is not played at a consistent BPM. On iOS and PC it works great.

The spec of my Android device is not very bad I assume:

• Model: Samsung SM-A750C
• CPU: Exynos 7885 (2.2GHz Dual-core (Cortex®-A73) +1.6GHz Hexa-core (Cortex®-A53))
• Android version: 10
• Browser: Chrome 91.0.4472.88

Weirdly, I have an older Android device on which the playback works fine.

• CPU: 4 x 2.1 GHz + 4 x 1.7 GHz
• Android version: 8
• Browser: Chorme 90.0.4430.210

I also try to turn off the cursor completely by setting enableCursor to be false,, but still the issue exists.
The code is here (identical to the official tutorial except for the disabled cursor): https://jsbin.com/solatavuke/

I did a basic profiling but can't see anything that's causing the poor performance. Please see below the profiling screenshot. I did it through chrome://inspect on my PC connecting the Android device.

Though maybe not helpful, here's a short video to demonstrate the issue.
https://drive.google.com/file/d/1AS9Y2242YzEe-L6PaaHAm1wj4VDksB0k/view?usp=sharing

Let me know if you need any further info to hunt down the issue. Happy to provide more info.

Also do you still plan to work on #480? I'm not very familiar with the Web Audio API, but if there's anything I can contribute or sponsor please let me know. Thanks again for your efforts on this project!

[Danielku15]

Update 2: I just gave it a very brief try to use Audio Worklets if available in the browser. It is just a very early attempt which follows the same approach of the old architecture to synthesize the audio in a WebWorker, and the put it to a ringbuffer within the worklet.

This way the Audio Output can pull the samples from the worklet without putting pressure to the main UI thread. There is still a lot of communication going via UI thread to pass on various messages. I see some chances to maybe better off-load stuff from the UI but it's a start worth investigating on performance side.

It is based on the branch with all other improvements and allows the same parameters as described above: https://demo.alphatab.net/crm/audioworklets/

[Reiji777]

@Danielku15 I profiled all the four variations plus the audioworklet one. Despite the bugs (weird sounds at the beginning — seems several notes are played at the same time; cursor runs later than the actual note being played), the audioworklet version runs smoothly without cracklings.

https://drive.google.com/file/d/1OqiJdzlKeYhLktPRc9ruWw6aKeKjojP6/view?usp=sharing

I also tested the audioworklet version on an old Android device with a pretty weak spec (fire hd 8 6th generation), and it runs smoothly even on that.

Also I noticed a bug for all the versions above: if I click on a note to move the cursor when playback is stopped, and I start playing again, the cursor won't move to show the playback status any more.

[Danielku15]

The current improvements are quite patchy and not well designed. I will definitly need some hours to get them clean and regression tested. After we were not really improving the situation I rather went the straight forward approaches to get some improvements in to profile them again.

Unfortunately the improvements on the highlighting and animation did not really bring the desired effect on this device. It's really crazy how these slow "update layer trees" won't disappear. My own code really performs well and fast. But there seem to be side effects in the engine causing real trouble.

I will try to prepare some proper version soon with the audio worklets and clean improvements in place.

[Danielku15]

@Reiji777 I uploaded a new version of the audio worklets here: https://demo.alphatab.net/crm/audioworklets/ this version has not any improvement beside the buffer fixes and the audio worklets. Most other improvements didn't anyhow have an effect on the load of the UI layer so I rather went back to improving the audio playback only.

I still plan to bring in some performance improvements separately, but only if they really make sense from a performance and maintainability perspective.

I see it as most important to fix the crackling audio. Reducing the load on devices in general is a second priority.

[Reiji777]

@Danielku15 Thanks for the updates Daniel. Below is the profiling result.
Overall I think it works great. Some issues I encountered below might be edge cases so just to share for your information.
https://drive.google.com/file/d/1wPAR9b948E-IylF0xTrkscDPVoO2X1v3/view?usp=sharing

There are three folders inside:

• Cracklings: when I opened the link for the first time, I heard the cracklings. This couldn't be reproduced after I refreshed the page. I doubt if this was because for the first time, the page was actually using some previously cached script, so just to share as a reference. I noticed in the profiling snapshot there were two workers running, while the other cases only had one.

• No Crackings: After refreshing the page as mentioned above, I don't hear the cracklings any more.

• Cracklings with buzz: The two profiles above are conducted on Chrome, while this one was on the system's default browser. (com.sec.android.app.sbrowser (87.0.4280.141)) I could always hear the cracklings with the buzz noises.

[lannka]

FYI that I experienced similar issue on my OnePlus 7 Pro phone. Cracks are heard every 5s. Disabling cursor does reduce the amount of cracks.

[Danielku15]

I integrated today the improvements and various improvements and fixes are now available via 1.3.0-alpha.127 It combines various fixes which should solve most of the performance issues and improves the playback via audio worklets. Changes are roughly:

1. Added Audio Worklets for playback.
2. Improved cursor positioning and animation by using transforms instead of positions
3. Improved lookup of currently played beat during playback
4. Added setting which allows disabling of beat cursor animations and element highlighting
5. Use default browser smooth scrolling over custom scrolling animation
6. Some smaller perf improvements.

https://demo.alphatab.net/crm/crackling/ is running using the updated build if you like to do some further testing.

[Reiji777]

@Danielku15 Thanks for the updates! As I was testing the latest version, it seems to be working fine on every device except for iPhone. Just submitted an issue: #646