ENSIME (scala) support

[fommil]

Please support ENSIME as a backend:

https://github.com/ensime/ensime-server

[vheon]

I'm just learning scala so I would love to get my hands dirty on this maybe adding first a JSON-based protocol since it would be much simpler to implement the integration then considering that the vim YouCompleteMe client talks to the YouCompleteMe server via a JSON based protocol. I will look into this in my spare time and as soon as @Valloric pass by we will hear his thoughts on this.

[fommil]

that is something we are looking into, @rorygraves might have something to say on the matter.

[rorygraves]

I'm definitely keen on being able to support JSON/other protocol implementations. The problem right now is that the protocol layer is not well separated meaning that a lot of places in the code directly see s-expressions. I'm working on refactoring the protocol out into a more modular setup. However, the code is intricate and I stand a high chance of breaking something subtle. To mitigate this I am actually writing a bunch of front to back integration tests to protect against myself, these should be submitted shortly then I can get on the refactor in the next 2-3 weeks.

[Valloric]

This should be filed against ycmd since that's where all the completers are.

@rorygraves It would be ideal if ENSIME could provide a simple HTTP+JSON interface. If you're worried about overhead, read this first. If ENSIME had such an interface, talking to it would be trivial for any client.

If you're looking for inspiration on how to craft the REST API, take a look at the ycmd example client. The API has been iterated on for almost a year and is the product of much experience in this area.

[fommil]

@Valloric ENSIME isn't just a completer, it also has a bunch of AST support for the scala language. Would ycmd be able to support the other features?

BTW, when we do make a JSON protocol available it will be a direct translation of the current SExp protocol... the project is far too mature to be coming up with a new protocol and it would be incredibly fiddly to maintain different domain objects for different wire formats. If it was JSON REST, but we don't create an API just for ycmd, is that enough?

[rorygraves]

@Valloric @fommil

Val,
Thanks for your feedback. Supporting JSON (sockets or HTTP) is definately on my radar. I would probably need websockets or similar as its not all request/response (there is aync events as well).

I agree with your article on performance and have no real concerns there - we are already across the wire (the ensime server is seperate from emacs).

I concur with the I think that a JSON (with or without HTTP) interface will certainly be occuring in the near future. But we have a number of steps to achieve before then:

1. Ensuring we have enough code coverage to make refactoring safe
2. Tease out the protocol layer (and abstract) - there is no good interface right now.
3. Create a nice clean abstraction layer that allows switching of encodings.

The layer gives some interesting possibilities in terms of editors and other deep inspection that is currently really hard to do.

Cheers

Rory

[Valloric]

ENSIME isn't just a completer, it also has a bunch of AST support for the scala language. Would ycmd be able to support the other features?

@fommil @rorygraves ycmd (and YCM) already provides features beyond code-completion; things like file diagnostics (error/warnings), GoToDeclaration/Definition etc. More is planned.

ycmd uses libclang which builds a full AST for C/C++/ObjC and can be queried arbitrarly. Same with OmniSharp for C# support and Jedi for Python. So ENSIME would fit in.

I encourage you to provide a simple & clean API through JSON, but even if you just expose what you currently have ycmd can use it. For code completion, your API can really be simple: full contents of the unsaved files + absolute filenames and line/column number of the user's cursor. The server would then return a structured response listing the possible completion strings available at that location (this is how ycmd works); I say structured because you want to describe is the string a function name (and if it is, param names, overloads etc), a class name or whatever. But that can be v2; v1 can be just a list of strings. That's enough for ycmd to build fuzzy-matching and other code-completion niceties on top of.

It shouldn't be hard to adapt whatever internal API you have to provide this, but you'd probably know better.

But we have a number of steps to achieve before then

All of those sound reasonable and higher priority. By all means, get a test suite in place and clean up the interface. :)

[Valloric]

Closing this issue; those who want to add a semantic completer using ENSIME should send a pull request to ycmd.

[rorygraves]

@Valloric It may well happen, we just have a large chunk of refactoring to do beforehand :)

[Valloric]

@rorygraves Oh I believe you, I just use the issue tracker as "things I have to do;" if the issue is not blocked on me, I tend to close it (lowers cognitive load for me and makes it easier to maintain things).