Implement Kakoune-style client-server model for sharing buffers across windows in different terminals

[malte-v]

One thing I really like about Kakoune is that it lets me use my X11 window manager for window management (duh) while still sharing buffers across windows. Kakoune achieves this using a sort of client-server model where one Kakoune process can attach to another one running in a different terminal, so effectively multiple terminals are controlled by a single editor instance. You could alternatively use tmux if you want the "vim experience", but I really enjoy not having to think about two layers of window management (editor windows/desktop windows). One other nice consequence is that you can arbitrarily mix windows containing editor buffers and windows belonging to other (graphical) applications.

What are your thoughts?

[malte-v]

I guess you could also keep the built-in window splitting features if people want to have that. It would just make things more complicated since one would have to distinguish between windows from different terminals and window splits.

[pickfire]

I think kakoune approach is nice for people who can do many terminals (like the time when I was using a tiling vm). Otherwise it doesn't help much. I still think it would be nice to have it, not sure how to implement it though. Wonder if @archseer have any thoughts on this.

[cessen]

I'm against removing the built-in splitting features, as I expect that to be my primary workflow. I don't really view it as two layers of window management. (And come to think of it, I'm now remembering that this is one of the reasons I didn't dive more into kakoune, despite it otherwise seeming really cool.)

As for client/server capabilities, I'm pretty indifferent. I certainly wouldn't try to stand in the way of such a feature, but don't consider it important myself.

[CBenoit]

I don't really use the splitting feature because i3 does the job for me, but I think it can be useful to some people. However, I really like the client/server architecture of kakoune allowing me to open multiple windows when working on the same instance (very useful when working using multiple screens). That's not the only approach though: in (doom) emacs I used multiple windows but I'm not sure it was client/server architecture.

[vv9k]

I'm against removing the built-in splitting features, as I expect that to be my primary workflow. I don't really view it as two layers of window management.

Same for me, I use both tmux and neovim splits + tabs together and it works perfectly. I use tmux to launch a terminal next to the editor or tail logs while I use neovim splits and tabs to open separate documents.

EDIT:
The flip side of the coin is that I have to memorize two sets of keybindings to navigate through the splits but In the end the distinction between terminal and editor splits is worth it at least for my use case.

[CBenoit]

So consensus so far appears to be in favor of keeping the built-in splits.
However, I'm strongly in favor of the ability to share buffers across windows.

[simias]

For me an additional advantage of being able to spawn several windows for the same editing session is to be able to use multiple monitors easily. Back when I used Emacs it was my main use for frames, for instance in order to have a compilation buffer on my secondary monitor and not clutter my main editing window.

Vim is really, really bad at this window management stuff IMO, so it's really not an example worth following (it still baffles me how anybody thought that Vim tabs were a good idea in their current implementation for instance).

[archseer]

I would like to see some sort of a client/server session one day in the future. The editing model is built on operational transforms, so I've been wanting to experiment with a collab mode for pair programming: one server instance and two user clients connected via ssh.

All this is of course a whole lot of work.

[pickfire]

two user clients

I wish the two clients can have different configurations but that is even harder.

[CBenoit]

two user clients

I wish the two clients can have different configurations but that is even harder.

Actually it would probably be the default behavior. We would work with transactions, not configurations.

I would like to see some sort of a client/server session one day in the future. The editing model is built on operational transforms, so I've been wanting to experiment with a collab mode for pair programming: one server instance and two user clients connected via ssh.

All this is of course a whole lot of work.

Not for tomorrow but very happy to hear that.
I was very excited when I saw CoreMirror 6 collab API.

[kirawi]

Are transactions like CRDTs, or OTs?

[Cons-Cat]

I like that Kakoune works very well with the Kitty terminal because of this. Kakoune itself can create a new Kitty split, then launch some process (like Ranger or Broot or a Git porcelain) inside that new window automatically, using the same mechanism as making another window for code. It makes the workflow feel very cohesive imo.

I don't mind having Emacs manage splits because Emacs has some Lisp package equivalent to every terminal app I'd ever use, but it seems really impossible imho for anything other than Emacs to achieve that.

[kirawi]

Perhaps this is something that would require an RFC?

[cessen]

For me an additional advantage of being able to spawn several windows for the same editing session is to be able to use multiple monitors easily.

That makes a lot of sense. I also use multiple monitors, but I usually end up using my editor in only one of them. Maybe I would end up using the client/server model more than I think.

I assume you could have multiple different server sessions going at once, though, right...? I often find myself with multiple logically distinct editor sessions open (e.g. one working on a project, another on an unrelated file for an unrelated task that came up, and even occasionally yet another with a second unrelated project opened up), and I wouldn't want them accidentally cross-interacting due to being forced into the same server session.

[CBenoit]

I assume you could have multiple different server sessions going at once, though, right...? I often find myself with multiple logically distinct editor sessions open (e.g. one working on a project, another on an unrelated file for an unrelated task that came up, and even occasionally yet another with a second unrelated project opened up), and I wouldn't want them accidentally cross-interacting due to being forced into the same server session.

With kakoune I open a new window on the same editor session by typing the new command in the editor (working on a single big project and I need several views), and a distinct editor session by opening my file with kak in another terminal.
My common workflow is one big project with several windows and various terminals for unrelated files and stuff like you mention, it all works quite well 🙂
(Also, since I'm using i3 manipulating these windows is quite easy and convenient using only the keyboard)

[cessen]

With kakoune I open a new window on the same editor session by typing the new command in the editor (working on a single big project and I need several views), and a distinct editor session by opening my file with kak in another terminal.

Okay, awesome. So it defaults to a new session, but you can opt-in to an existing session. That sounds good.

Ultimately, I think I'll still prefer having the application handle splits. Mainly because it's something that gets carried with the application, so I can always depend on its behavior regardless of environment. But you never know for sure without trying. :-)

(Also, since I'm using i3 manipulating these windows is quite easy and convenient using only the keyboard)

(I'm using AwesomeWM, so it's even easier for me. ;-) )

[cessen]

With the renewed discussion on the Matrix channel about a client/server architecture, I've been thinking more about this. And I'm a bit concerned that the discussion over there is maybe heading in an "ultimate architecture that handles every use case" direction, which is something I've rarely seen work out in practice, and usually ends up with over-engineered solutions that collapse under their own weight.

So I'd like to suggest that for now we focus on just solving the "single user running sessions locally" use-case, and tackle other use-cases in the future once we have that implemented and really working well.

That may require re-working the architecture to some extent down the line to accommodate other use-cases. But I think it will also let us actually get something working in a reasonable time-frame, and furthermore will give us valuable experience that may inform how we want to handle those other use-cases.

Somewhat relatedly:

[...] so I've been wanting to experiment with a collab mode for pair programming: one server instance and two user clients connected via ssh.

This is a very cool idea, and I like it. But I don't know if an editor-specific solution is really what we want here. For example, if I'm working at a company and want to pair-program with a coworker, I'm unlikely to convince them to use Helix for that purpose.

To give something like this any hope of being useful in practice, I suspect it would need to involve some kind of protocol that any editor can implement. And I'm not sure that's something worth tackling from within the Helix project. (Although experiments are always great, of course!)

[VarLad]

I'd like to point out that we can use https://github.com/zellij-org/zellij for this

[gbaranski]

I thought how to solve this, and I came up with making an universal editor backend that would work with many frontends, and editors such as helix maybe could use that under the hood. I explained my vision at gbaranski/eb. What do you think?

[kirawi]

It seems to me that it may be helpful to draw upon the LSP and DAP protocols for inspiration here. They're both very mature and capable protocols that address significantly complex and variable aspects of the target abstraction.

[haarts]

(I had to search for the DAP and LSP protocol pages, so I've linked both for convenience.)

[pantosaur]

+1. I am excited for this project and I would adopt this 100% if it was built around a client-server architecture like a few other editors have been doing, the most notable being kakoune. My personal argument for this is that there are already a bunch of different window layering programs that do what they do, and window management from inside the text editor isn't necessary. I think this approach is exactly the approach of not attempting to "handle every use case", and the most workflow agnostic.

[hovsater]

Another benefit we would get from this client/server architecture would be the ability to share language-servers. Right now, each Helix instance will spawn its own language server. For instance, I sometimes use different Helix instances to simulate "tabs". Coming from Vim, I used tabs mostly as a context switcher. One tab would have buffers and splits related to a specific portion of the project, while another tab would be a different portion of the project. In Helix I spawn different instances to accomplish the same thing. Unfortunately, this also means that another language server spawns all in vein. A client/server architecture would solve this nicely. 🙂

[johalun]

While we're at it, I would love it if the server/client design could also be used like EmacsClient I mentioned in #2177 with one mode where "hx-client" would just tell the running Helix instance to open the file and then return immediately, and a "wait" mode where hx-client would wait until the file is closed in Helix (useful for editing git commit messages in an already running editor instance).

[robem]

I'm very much in favor of a client-server architecture because it fits my workflow.
My window manager is good at managing windows, my terminal multiplexer is good at managing the terminal space, and my editor should be good at editing files. Usually, I have a single terminal split into multiple panes (and tabs). One for version control, multiple for editing, and at least one shell. I switch/scale/move them via my terminal multiplexer. With helix, I keep a single pane for editing and have to use different shortcuts inside my terminal depending on whether helix runs inside the pane. This is mildly annoying.

[yuuyins]

i need client-server. one of the main reasons i will be using emacs for the following years. but also make sure we can use this remotely smoothly; emacs client-server is not suitable for remote connections which is a pain

[kirawi]

How could this be handled in a cross-platform way? Windows doesn't support daemon processes like unix.

[Philipp-M]

We could do it in a peer 2 peer fashion, e.g. using libp2p, this could even allow connecting into a browser session.
Which in turn could also enable collaborative editing over the network as well if done properly (I'm thinking of CRDTs)

[archseer]

We could do it in a peer 2 peer fashion, e.g. using libp2p, this could even allow connecting into a browser session. Which in turn could also enable collaborative editing over the network as well if done properly (I'm thinking of CRDTs)

Yeah I have branches on both of these: Used diamond-types for CRDTs, and libp2p for a direct collaborative session. I think this is a bit orthogonal to this issue though.

[Philipp-M]

Oh really, cool :)

Do you have the branches public somewhere? I haven't found them yet.

CRDTs are certainly something separate. (lib)p2p though I think can indeed help abstract the networking/communication/discovering layer away.

For example (rough idea):

1. At the startup of helix discover other clients locally
2. Check which documents are open in each of these instances map each instance as master of these documents
3. If a client that is not master of a document, wants to open that document, it connects to/registers at the master of that document
4. Every time an edit occurs, it is transferred to the master, which does the actual edit
5. When the master gets closed, it could transfer the document state to the first connected slave. I guess this will get tricky, because we would need to make the state Serializable (related Persistent State (session) #401)

Edit: Thinking about it, this approach would make handling with language servers quite difficult though, it may need some namespacing for the language server (roots?)

[Randalix]

I've created a bounty for it. Would be great to see this feature added...

[wongjiahau]

Another benefit we would get from this client/server architecture would be the ability to share language-servers. Right now, each Helix instance will spawn its own language server. For instance, I sometimes use different Helix instances to simulate "tabs". Coming from Vim, I used tabs mostly as a context switcher. One tab would have buffers and splits related to a specific portion of the project, while another tab would be a different portion of the project. In Helix I spawn different instances to accomplish the same thing. Unfortunately, this also means that another language server spawns all in vein. A client/server architecture would solve this nicely. 🙂

I think what you wanted is probably a tab...

[jakoschiko]

Maybe another good reason for a client-server model: I'm working heavily with Ctrl+Z instead using multiplexers. When I'm editing Rust code and want to use cargo, sometimes it happens that I press Ctrl+Z in the wrong moment. The LSP seems to hold a lock and my cargo command will be blocked indefinitely.

✦ > cargo run
    Blocking waiting for file lock on package cache

If the LSP runs on the server side, it can do its work and release the lock while the client side is suspended via Ctrl+Z.

[evelant]

A client-server model would likely be great for adding "helix mode" to other popular editors. I imagine helix driving the vscode editor (as the vscode-neovim plugin does) or jetbrains ides. I love the editing experience of helix but I'm also accustomed to using all the tools offered by vscode/jetbrains that can't be easily replicated in helix so I end up using vscode-neovim and ideavim because it's the closest I can get to helix in those editors.

[zamlz]

Hello! Any updates on the client/server feature for helix?