Fully asynchronous request handling

[pepeiborra]

#727 made the Shake queue able to concurrently handle more than one rule, but ghcide still handled LSP requests sequentially. This change makes the LSP handler multi-threaded, so now ghcide no longer blocks while e.g. computing an expensive code lens.

This change also adds a custom request protocol for testing, which can be generally useful. For instance, tests can now query the location of the interface files cache dir which will be useful for testing #760

Reviewers - things to watch for:

• New space leaks
• New race conditions

To do:

• run benchmarks

[wz1000]

Very nice, looks good to me. Now let's see how many tests break because of the ordering of messages...

[wz1000]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.

[pepeiborra]

The test suite is failing because the new CustomHandler responds to a message sent by testSessionWait in a way that lsp-test cannot parse.

The message that causes the issue:

ghcide/test/src/Development/IDE/Test.hs

Line 61 in 597a0c5

void $ sendRequest (CustomClientMethod "non-existent-method") ()

I've raised this up in the lsp-test issue tracker lukel97/lsp-test#73

[wz1000]

Maybe we can merge after marking the test as known broken.

[pepeiborra]

It's all the tests that use testSessionWait

[pepeiborra]

Early benchmarks not looking good - edit performance is much slower. There is increased memory usage but that seems to be a regression upstream

[wz1000]

What is master, upstream, upstream1 and upstream2?

[pepeiborra]

What is master, upstream, upstream1 and upstream2?

I'll publish the whole benchmark artifacts once they are ready. In that screenshot:

• Master points to v0.3.0 in my repo
• Upstream is Haskell/ghcide:master
• Upstream1 is upstream~1
• etc

[pepeiborra]

After re-running the benchmarks in an isolated non-throttling CPU it looks like this branch does not introduce any time or space regression. Will upload full results shortly.

I have opened #771 to address the pre-existing space leak.

[pepeiborra]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.

[pepeiborra]

The tests are now passing after upgrading to a fixed lsp-test version.

The artefacts from the ghcide_bench_linux_stack pipeline show no change in memory usage. There's also no significant change in time usage, but this not a reliable measurement since the benchmark runs in a shared environment.

Local benchmarks do show a 10% time decrease in the edit experiment.