Project architecture to support plugins

[alanz]

We have taken the same approach to plugins we had in haskell-ide-engine, with the same intention.

1. We have a plugin descriptor which defines the operations provided by the plugin
2. This descriptor is given a "label" when it is configured in exe/Main.hs, which serves to uniquely identify it within the particular set of plugins configured in that Main.hs, and thus the built hls executable.

This allows us to contemplate the next step wrt plugins

I propose the we split hls up

• a slimmed-down core, that basically corresponds to the plugins built into ghcide.
• an API package that can be used by plugin authors to write plugins for use in hls. The corresponds to the hie-plugin-api as originally envisaged.
• a fat package that depends on the core, the api, and all known plugins at this time, and provides an exe depending on them.

The fat module is the equivalent of what hls is today.

The intention is to facilitate two things, much the way LSP does for servers and clients

• tool authors can provide a module exposing an hls plugin API for their tool, built against the API module. e.g. a package fourmolu-hls could expose a plugin descriptor for fourmolou.
• users can choose which plugins they actually want to use, and not pay the compilation tax for ones they are not interested in.

To make this work cleanly, we could provide a utility that would take a descriptor file listing the plugin packages and the names to be used for them, and generate an appropriate exe/Main.hs.

And of course the existing Main.hs would have to be slimmed down to hosting a plugin config section and then invoking something in a library.

[pepeiborra]

I wanted the same thing when I was working on my retrie plugin.

One thought: since plugins need to access the ghcide internals as well, hls-core should probably reexport those.

Slimming down Main.hs is currently blocked by ghcide, but there's already an issue in the ghcide tracker to discuss that.

[alanz]

Slimming down Main.hs is currently blocked by ghcide, but there's already an issue in the ghcide tracker to discuss that.

https://github.com/digital-asset/ghcide/issues/478, I think.

[jneira]

Afaiu, after #379 the part that would remain would be

we could provide a utility that would take a descriptor file listing the plugin packages and the names to be used for them, and generate an appropriate exe/Main.hs.

[alanz]

I think a validating step would be to make a PR on an upstream such as one of the formatters, to include the plugin descriptor in it. This will help us to fine tune the API for the hls-plugin-api, by proving it in real life.

[jneira]

Issues related to complete the architecture:

• code organization: Create hls-plugin-api and move plugins to exe #379, Split HLS plugins into internal libraries #421
• plugins specific config: Regularise the config for plugins #513
• test infrastructure: Test infrastructure for plugins #576

[jneira]

I've started a POC to enable the inclusion of plugins at compile time (we dont have other alternative, right?) using cabal flags and CPP conditions: https://github.com/haskell/haskell-language-server/compare/master...jneira:cpplugins?expand=1

To assemble the desired hls version you could do:

• cabal install: all plugins and formatters by default like the actual behaviour
• cabal install -f-all-plugins -f-all-formatters: only ghcide plugin (it is fixed)
• cabal install -f-all-formatters -f+floskell: all non formatting plugins and only floskell as formatter
• `cabal intall -f-all-plugins -f-all-formatters -f+hlint -f+fourmolu: only ghcide and hlint plugins and fourmolu as formatter

The plan would be change the install script to take in account plugins and:

• let user list the concrete list of plugins
• prepare some predefined plugins sets and set the appropiate cabal flags underneath:
  • all
  • none
  • floskell (all non formatter plugins and floskell). This would be for all formatters
  • etc

[alanz]

@jneira That can work. We have to balance having an "it just works" experience (downloading the right exe) with being able to customize things.

If we download a fat exe, then compile time and deps are not the problem, but tweaking which features are visible is. So at least we need some way to have a list of enabled/disabled plugins in a config file. Perhaps we could consider doing something like https://erlang-ls.github.io/configuration/ to tweak this.

Configuring the compiled-in plugins via flags controls the "official" list, and is a step in the right direction.

I still think we should be able to end up with a config file that generates the part of the Main.hs that gets compiled, probably wrapped in a local cabal project to build it.

(

haskell-language-server/exe/Main.hs

Lines 49 to 66 in 488fd46

	[ GhcIde.descriptor "ghcide"
	, Pragmas.descriptor "pragmas"
	, Floskell.descriptor "floskell"
	, Fourmolu.descriptor "fourmolu"
	, Tactic.descriptor "tactic"
	-- , genericDescriptor "generic"
	-- , ghcmodDescriptor "ghcmod"
	, Ormolu.descriptor "ormolu"
	, StylishHaskell.descriptor "stylish-haskell"
	, Retrie.descriptor "retrie"
	#if AGPL
	, Brittany.descriptor "brittany"
	#endif
	, Eval.descriptor "eval"
	, ExplicitImports.descriptor "importLens"
	, ModuleName.descriptor "moduleName"
	, Hlint.descriptor "hlint"
	]

)

The config could be as simple as a list of haskell packages and the string to install them under.

[jneira]

mmm, I see, I had thought in to add a common config options for all plugins to be able to enable or disable at runtime, via the config provided by the client.
It would be a generalization of the hlintOn actual config option.
The config could be a list of JSON maps

[{ id: "hlint", 
    enabled: true, 
    config: { prop1: val1,}},
,
]

would fit that with your idea?

[alanz]

would fit that with your idea?

Yes, provided it is idiomatic json. I would expect eventual JSON "paths" something like
plugins.hlint.enabled : true
plugins.hlint.config.prop1: val1
etc

[michaelpj]

I still think we should be able to end up with a config file that generates the part of the Main.hs that gets compiled, probably wrapped in a local cabal project to build it.

As a user, this doesn't sound very appealing to me. I'd much rather just build it with cabal, setting some flags if necessary.

A potentially very evil suggestion... have we thought about doing actual dynamic object loading? i.e. have separately distributable artifacts for plugins (object files) which the main exectuable can load dynamically at runtime provided they're in some discoverable location? I've never done this with Haskell, but if it was possible it might be a nice UX.

[alanz]

As a user, this doesn't sound very appealing to me.

I foresee a future where there are a lot of plugins, and it becomes the norm for user organisations to have custom ones, providing specific features local to their environment.

Packing all of this into a master cabal file will not work. And I think the dynamic module approach brings even bigger problems than it solves. And the installed plugins at a given site will not change often.

What could make the process smooth would be if the config builder had a ui like in ghcup tui, which is aware of common plugins, and can manage things.

[jneira]

Issues related to complete the architecture:

• code organization: Create hls-plugin-api and move plugins to exe #379, Split HLS plugins into internal libraries #421
• plugins specific config: Regularise the config for plugins #513
• test infrastructure: Test infrastructure for plugins #576

All the checks are done so i think the plugin architectura,in its actual desig, is complete