Debounce tree-sitter parsing

[AlfGalf]

I am currently editing an ≈40000 line C++ file and it is very painful because after every keypress it takes around 0.5 seconds to (I assume) update the syntax highlighting.
This is especially true when pasting a chunk of code. It slows to an absolute crawl.

I would propose debouncing the syntax highlighting ans possibly other processes that run after a keypress to run 0.1 seconds after the last keypress or something similar?

[kirawi]

I brought this up a couple years ago on Matrix. At the time the feeling was, IIRC, that it wasn't desirable because the tree-sitter motions are a core method of movement in Helix and that it still wouldn't behave great with long incremental parses like in your case.

[AlfGalf]

I am willing to put some work into this as I'd like to improve things for editing these large files.
I'll look at throwing something together for the solution I suggested but if theres a better way forward Id be keen to here it?

[kirawi]

You should speak with one of the maintainers about this beforehand to see if such a change would be rejected before putting work into it.

[AlfGalf]

@pascalkuthe penny for your thoughts?

My current plan would be something along the line of moving parsing on a different thread to allow text entry to still be received.

Then I would want to allow the parsing to be interrupted if new input is received and restarted with that new input.

Movement operators would then have to spin until a parse was completed.

I am not sure what I would do about syntax highlighting just yet, perhaps just let it be wrong as hopefully it would only be for a short period of time? Perhaps try to move the existing highlighting with a best guess until updated?

[pascalkuthe]

it's a really hard problem to solve and it depends mostly on how well tree sitter supports this.

We definitely should not always be parsinlg in a background thread. Parsing in a background thread adds latency as therads take time to startup. We also definitely don't want to render multiple frames for each edit (both inefficient and causes flickering).

Y4ou can set the tree sitter parser timeout to something lower (right now we set it to 500ms and disable TS if we hit that timeout) and once the timeout is reached send the parse to a background thread and continue parsing there.

All of this is made more difficult by the fact that you can not edit the old tree while parsing so you would need to clone the tree when sending it to the background thread which adds more overhead. I am also not sure how well tree sitter would handle multiple edits to the same tree before reparsing.

It's also unclear to me how tree sitter handles running queries on an edited but not yet reparsed tree. If that doens't work then there is no point in doing this since flickering highlights is just as bad as no highlights. At that point it would be better to just disable highlights after a shorter timeout (like 200ms instead of 500ms).

Indeed any tree sitter movement would need to spin until the latest parse is complex which would add a lot of extra complexity and I am not sure how feasible that is.

You can give it a try to implement what I described but it will be quite involved and tree-sitter may not support this well (and even if it does I am quite wary of the complexity this introduces). This would likely also be a fairly complex change (for something fairly niche) that would conflict with larger changes in the works that I would prioritize over this (I am currently entirely rewriting our TS integration for example).

[AlfGalf]

Ahh okay, definitely more complicated than it would seem.
I will have a play around to try understand the problem space more.