Explore options for Windows support and impact on plugin refactor

[izgeri]

Overview

We have been asked to verify that based on our current understanding, the plugins that are written based on this interface will not have to be thrown out based on future work to expand support to plugins that can run on Windows. If we discover that additional research is needed to answer this question, we'll need to regroup and determine when to schedule that research.

There are two concerns prompting this spike:

• Will our plugin interface as we’ve defined it have to change when we add Windows support
• Will connector plugins written to the current spec for buildmode=plugin have to be changed to be runnable in Windows, and by how much

We had the following ideas for addressing this in the feature proposal:

• Contribute upstream to fix buildmode=plugin to make it windows compatible
• Add support for buildmode=c-shared
• Expand plugin workflow to include (optional?) gRPC server

The outcome of this short spike is to document our current understanding of the impact of each potential solution and the risk to the current effort from these potential future changes.

Definition of done

• We've documented (in issue comments here, at a minimum) our understanding the impact buildmode=c-shared support will have on Secretless core and on external connector plugins written to the standard in Secretless has a public plugin API for service connectors #693
• We've documented (in issue comments here, at a minimum) our understanding of the impact adding gRPC support will have on Secretless core and on external connector plugins written to the standard in Secretless has a public plugin API for service connectors #693
• We've documented any other ideas we have for adding support for Windows binaries of Secretless that support external plugins, including potentially contributing support for Windows to buildmode=plugin

[izgeri]

Note: this spike is timeboxed to 1 day.

[izgeri]

This thread has some notes about issues when external parties build external plugins for a go project using go plugins - it especially seems to be referring to vendoring problems referenced in this go issue

This thread suggests considering using

• an embedded scripting language
• building a go plugin as an executable, having the host run it, and communicate with it using any arbitrary RPC system over stdin/out (for ex, Hashi's go-plugin system)

[izgeri]

The GitHub issue for go plugins lacking Windows support - the last comment is Really, this is waiting on somebody who knows the subject area and wants to send some code.

[izgeri]

Currently external plugins are included by placing the shared library (.so) files (compiled with buildmode=plugin in a specific customizable directory in the Secretless container, and Secretless loads them at runtime

Notes from check-in:

• We could make a mega-Secretless that is compiled using all supported/approved plugins (including external) - eg we build it, and plugins are added at compile time
• We could provide instructions for compiling your own Secretless with a set of plugins that you select - eg we tell others how to build it, and plugins are added at compile time
• @doodlesbykumbi did a POC of using buildmode=c-shared and had difficulty doing anything beyond something trivial in Linux
• gRPC seems to be a favorite for plugins, but @sgnn7 is looking into it and raises that passing connections (eg net.Conn, http.Request) over gRPC may not be possible
  • Related ideas to make this work:
    • Plugins are written as their own programs and you build a go binary. On startup Secretless will find the plugins in a known and customizable directory and run a system exec to start each one. When a new connection comes in, Secretless delegates it to the plugin by sending it a socket for data transfer between the plugin and Secretless.
      • TBD if go-plugin works in Windows
      • TBD how exactly this will change the interface, but there would likely be some changes - looking into revising it slightly now to replace net.Conn with something more lightweight / flexible that we can continue to use once we want to add this support (eg change net.Conn in the interface to an object we define that only had read/write/close - we'll review to find the exact methods needed)
      • To really understand this and be sure we're on the right path, we'd want to build a POC of the workflow (~1 developer week)

[izgeri]

@sgnn7's initial POC that works on OSX

[sgnn7]

Seem to work on Windows too

[sgnn7]

Seems like we can use gRPC go-plugin for cross-platform plugin handling. We'd need to pass locations of sockets to plugins rather than net.Conn but that should be majority of the changes needed. We probably could also wrap current plugins in some library code that could make current style of plugins compatible with the gRPC system. There may be a need for codegen to automate majority of the work here.

[sgnn7]

Extra option to consider: Golang interpreters

Users provide us with .go files and we run that in context of secretless. May be slow and require similar handling of sockets like gRPC to pass connections around but would make things much easier for the writers.

Edit: https://godoc.org/github.com/containous/yaegi

[sgnn7]

Simple PoC of stdlib-based interpreter-based provider plugin: https://github.com/cyberark/secretless-broker/pull/868/files

[izgeri]

Nice work @sgnn7 and @doodlesbykumbi!

[izgeri]

Note: in further discussions yaegi will require close collaboration with plugin authors to ensure Secretless core has the libraries the plugin will need, and there could be real difficulty with supporting plugins that require different versions of the same library