diff --git a/README.md b/README.md index d2d833d..16a7b6e 100644 --- a/README.md +++ b/README.md @@ -19,8 +19,8 @@ _wand_ is a toolkit for common and often recurring project processes for the tas The provided [API packages][go-pkg-pkg] allow users to compose their own, reusable set of tasks and helpers or built up on the [reference implementation][go-pkg-elder]. - **Adapts to any “normal“ or [“mono“][trunkbasedev-monorepos] repository layout** — handle as many module _commands_ as you want. _wand_ uses an abstraction by managing every `main` package as _application_ so that tasks can be processed for all or just individual _commands_. -- **Runs any `main` package of a [Go module][go-docs-ref-mod] without the requirement for the user to install it beforehand** — thanks to the awesome [gobin][] project, there is no need for the user to `go get` the `main` package of a Go module in order to run its compiled executable. -- **Comes with support for basic [Go toolchain][go-pkg-cmd/go] commands and popular modules from the Go ecosystem** — run common commands like `go build` and `go test` or great tools like [goimports][go-pkg-golang.org/x/tools/cmd/goimports], [golangci-lint][go-pkg-github.com/golangci/golangci-lint/cmd/golangci-lint] and [gox][go-pkg-github.com/mitchellh/gox] in no time. +- **Runs any `main` package of a [Go module][go-docs-ref-mod] without the requirement for the user to install it beforehand** — [Go 1.16 introduced `go install` command support for the `pkg@version` module syntax][go-docs-rln-1.16#modules] which is internally used by the [`gotool` task runner][go-pkg-stc-task/gotool#runner] to install executables of `main` packages into custom locations without “polluting“ a projects `go.mod` file. +- **Comes with support for basic [Go toolchain][go-pkg-cmd/go] commands and popular modules from the Go ecosystem** — run common commands like `go build`, `go install` and `go test` or great tools like [gofumpt][go-pkg-github.com/mvdan/gofumpt], [golangci-lint][go-pkg-github.com/golangci/golangci-lint/cmd/golangci-lint] and [gox][go-pkg-github.com/mitchellh/gox] in no time. See the [API](#api) and [“Elder Wand“](#elder-wand) sections for more details. The [user guides](#user-guides) for more information about how to build your own tasks and runners and the [examples](#examples) for different repositories layouts (single or [“monorepo“][trunkbasedev-monorepos]) and use cases. @@ -81,32 +81,28 @@ The [`app`][go-pkg-app] package provides the functionality for application confi The [`task`][go-pkg-task] package defines the API for runner of commands. [`Runner`][go-pkg-if-task#runner] is the base interface while [`RunnerExec` interface][go-pkg-if-task#runnerexec] is a specialized for (binary) executables of a command. -The package already provides runners for the [Go toolchain][go-pkg-cmd/go] and the [gobin][] Go module: +The package already provides runners for the [Go toolchain][go-pkg-cmd/go] and [gotool][go-pkg-task/gotool] to handle Go module-based executables: - **Go Toolchain** — to interact with the [Go toolchain][go-pkg-cmd/go], also known as the `go` executable, the [`golang.Runner`][go-pkg-stc-task/golang#runner] can be used. -- **`gobin` Go Module** — to install and run [Go module][go-docs-ref-mod] `main` the [`Runner`][go-pkg-stc-task/gobin#runner] makes use of the [`github.com/myitcv/gobin`][go-pkg-github.com/myitcv/gobin] command. - 1. **Go Executable Installation** — Using the [`go install`][go-pkg-cmd/go#install] or [`go get`][go-pkg-cmd/go#print_env] command for a [Go module][go-ref-mod] `main` package, the resulting executables are placed in the Go executable search path that is defined by the [`GOBIN` environment variable][go-pkg-cmd/go#env_vars] (see the [`go env` command][go-pkg-cmd/go#print_env] to show or modify the Go toolchain environment). - Even though executables are installed “globally“ for the current user, any [`go.mod` file][go-ref-mod#go.mod] in the current working directory will be updated to include the Go module. This is the default behavior of the [`go get` command][go-pkg-cmd/go#print_env] when running in [“module mode“][go-pkg-cmd/go#mod_cmds] (see [`GO111MODULE` environment variable). - Next to this problem, installed executables will also overwrite any previously installed executable of the same module/package regardless of the version. Therefore only one version of a executable can be installed at a time which makes it impossible to work on different projects that make use of the same executable but require different versions. - 2. **History and Future** — The installation concept for `main` package executables has always been a somewhat controversial point which unfortunately, partly for historical reasons, does not offer an optimal and user-friendly solution up to now. +- **`gotool` Go module-based executables** — to install and run [Go module-based][go-docs-ref-mod] `main` packages, the [`gotool.Runner`][go-pkg-stc-task/gotool#runner] makes use of the Go 1.16 `go install` command features. + 1. **Go Executable Installation** — [Go 1.16 introduced `go install` command support for the `pkg@version` module syntax][go-docs-rln-1.16#modules] which allows to install commands without “polluting“ a projects `go.mod` file. The resulting executables are placed in the Go executable search path that is defined by the [`GOBIN` environment variable][go-pkg-cmd/go#env_vars] (see the [`go env` command][go-pkg-cmd/go#print_env] to show or modify the Go toolchain environment). + The problem is that installed executables will overwrite any previously installed executable of the same module/package regardless of the version. Therefore only one version of an executable can be installed at a time which makes it impossible to work on different projects that make use of the same executable but require different versions. + 2. **UX Before Go 1.16** — The installation concept for `main` package executables was always a somewhat controversial point which unfortunately, partly for historical reasons, did not offer an optimal and user-friendly solution until Go 1.16. The [`go` command][go-pkg-cmd/go] is a fantastic toolchain that provides many great features one would expect to be provided out-of-the-box from a modern and well designed programming language without the requirement to use a third-party solution: from compiling code, running unit/integration/benchmark tests, quality and error analysis, debugging utilities and many more. - Unfortunately this does not apply for the [`go install` command][go-pkg-cmd/go#install] of Go versions less or equal to 1.15. - The general problem of tool dependencies is a long-time known issue/weak point of the current Go toolchain and is a highly rated change request from the Go community with discussions like [golang/go#30515][gh-golang/go#30515], [golang/go#25922][gh-golang/go#25922] and [golang/go#27653][gh-golang/go#27653] to improve this essential feature, but they've been around for quite a long time without a solution that works without introducing breaking changes and most users and the Go team agree on. - Luckily, this topic was finally picked up for the next [upcoming Go release version 1.16][gh-golang/go-ms-145] and [golang/go#40276][gh-golang/go#40276] introduces a way to install executables in module mode outside a module. - The [release note preview also already includes details about this change][go-docs-tip-rln-1.16#mods] and how installation of executables from Go modules will be handled in the future. - 3. **The Workaround** — Beside the great news and anticipation about an official solution for the problem the usage of a workaround is almost inevitable until Go 1.16 is finally released. - The [official Go wiki][gh-golang/go-wiki] provides a section on [“How can I track tool dependencies for a module?“][gh-golang/go-wiki-mods#tool_deps] that describes a workaround that tracks tool dependencies. It allows to use the Go module logic by using a file like `tools.go` with a dedicated `tools` build tag that prevents the included module dependencies to be picked up for “normal“ executable builds. This approach works fine for non-main packages, but CLI tools that are only implemented in the `main` package can not be imported in such a file. - In order to tackle this problem, a well-known user from the community created [gobin][], an experimental, module-aware command to install and run `main` packages. - It allows to install or run `main` package commands without “polluting“ the `go.mod` file. Modules are downloaded in version-aware mode into a cache path within the [users local cache directory][go-pkg-func-os#usercachedir]. This way it prevents problems due to already installed executables by placing each version of an executable in its own directory. - The decision to use a cache directory instead of sub-directories within the `GOBIN` path doesn't require to mess with the users setup and keep the Go toolchain specific paths clean and unchanged. - `gobin` is still in an early development state, but has already received a lot of positive feedback and is used in many projects. There are also members of the core Go team that have contributed to the project and the chance is high that the changes for Go 1.16 were influenced or partially ported from it. See [gobin‘s FAQ page][gobin-wiki-faq] in the repository wiki for more details about the project. - It is currently the best workaround to… - 1. prevent the Go toolchain to pick up the [`GOMOD` (`go env GOMOD`) environment variable][go-pkg-cmd/go#print_env] that is initialized automatically with the path to the [`go.mod` file][go-ref-mod#go.mod] in the current working directory. - 2. install `main` package executables locally for the current user without “polluting“ the `go.mod` file. - 3. install `main` package executables locally for the current user without overriding already installed executables of different versions. - 4. **The Go Module `Runner`** — To allow to manage the tool dependency problem, this package provides a command runner that uses `gobin` in order to prevent the problems described in the sections above like the “pollution“ of the "go.mod" file and allows to… - 1. install `gobin` itself into `GOBIN` ([`go env GOBIN`][go-pkg-cmd/go#print_env]). - 2. run any Go module command by installing `main` package executables locally for the current user into the dedicated `gobin` cache. + This did not apply for the [`go install` command][go-pkg-cmd/go#install] of Go versions less than 1.16. + The general problem of tool dependencies was a long-time known issue/weak point of the Go toolchain and was a highly rated change request from the Go community with discussions like [golang/go#30515][gh-golang/go#30515], [golang/go#25922][gh-golang/go#25922] and [golang/go#27653][gh-golang/go#27653] to improve this essential feature. They have been around for quite a long time without a solution that worked without introducing breaking changes and most users and the Go team agree on. + Luckily, this topic was [finally resolved in the Go release version 1.16][go-docs-rln-1.16#modules] and and [golang/go#40276][gh-golang/go#40276] introduced a way to install executables in module mode outside a module. + 3. **The Leftover Drawback** — Even though the `go install` command works totally fine to globally install executables, the problem that only a single version can be installed at a time is still left. The executable is placed in the path defined by `go env GOBIN` so the previously installed executable will be overridden. It is not possible to install multiple versions of the same package and `go install` still messes up the local user environment. + 4. **The Workaround** — To work around the leftover drawback, the [`gotool` package][go-pkg-task/gotool] provides a runner that uses `go install` under the hood, but allows to place the compiled executable in a custom cache directory instead of `go env GOBIN`. It checks if the executable already exists, installs it if not so, and executes it afterwards. + The concept of storing dependencies locally on a per-project basis is well-known from the [`node_modules` directory][npm-docs-cli-v7-config-folders#node_modules] of the [Node][] package manager [npm][]. Storing executables in a cache directory within the repository (not tracked by Git) allows to use `go install` mechanisms while not affect the global user environment and executables stored in `go env GOBIN`. + The runner achieves this by temporarily changing the `GOBIN` environment variable to the custom cache directory during the execution of `go install`. + The only known disadvantage is the increased usage of storage disk space, but since most Go executables are small in size anyway, this is perfectly acceptable compared to the clearly outweighing advantages. Note that the runner dynamically runs executables based on the given task so the `Validate` method is a _NOOP_. + This is currently the best workaround to… + 1. install `main` package executables locally for the current user without “polluting“ the `go.mod` file. + 2. install `main` package executables locally for the current user without overriding already installed executables of different versions. + 5. **Future Changes** — The provided runner is still not a clean solution that uses the Go toolchain without any special logic so as soon as the following changes are made to the Go toolchain (Go 1.17 or later), the runner will be removed again: + - [golang/go#42088][] — tracks the process of adding support for the Go module syntax to the `go run` command. This will allow to let the Go toolchain handle the way how compiled executable are stored, located and executed. + - [golang/go#44469][golang/go#44469#c-784534876] — tracks the process of making `go install` aware of the `-o` flag like the `go build` command which is the only reason why the provided runner exists. ### Project Metadata @@ -118,7 +114,7 @@ The package also already provides a [VCS `Repository` interface reference implem ### Tasks -The [`task`][go-pkg-task] package defines the API for tasks. [`Task`][go-pkg-if-task#task] is the base interface while [`Exec`][go-pkg-if-task#exec] and [`GoModule`][go-pkg-if-task#gomodule] are a specialized to represent the (binary) executable of either an “external“ or Go module based command. +The [`task`][go-pkg-task] package defines the API for tasks. [`Task`][go-pkg-if-task#task] is the base interface while [`Exec`][go-pkg-if-task#exec] and [`GoModule`][go-pkg-if-task#gomodule] are a specialized to represent the (binary) executable of either an “external“ or Go module-based command. The package also already provides tasks for basic [Go toolchain][go-pkg-cmd/go] commands and popular modules from the Go ecosystem: @@ -231,7 +227,7 @@ See the [examples](#examples) to learn about more uses cases and way how to stru ### Build It Yourself -_wand_ comes with tasks and runners for common [Go toolchain][go-pkg-cmd/go] commands, the [gobin][] and other popular modules from the Go ecosystem, but the chance is high that you want to build your own for your specific use cases. +_wand_ comes with tasks and runners for common [Go toolchain][go-pkg-cmd/go] commands, [gotool][go-pkg-task/gotool] to handle Go module-based executables and other popular modules from the Go ecosystem, but the chance is high that you want to build your own for your specific use cases. #### Custom Tasks @@ -246,7 +242,7 @@ To create your own task that is compatible with the _wand_ API, implement the [` - the returned type of the `ID() *project.GoModuleID` method must provide the import path and module version of the target `main` package. For sample code of a custom task please see the [examples](#examples) section. -Based on your task kind, you can also either use one of the [already provided command runners](#command-runners), like for the [Go toolchain][go-pkg-task/golang] and [gobin][], or [implement your own custom runner](#custom-runners). +Based on your task kind, you can also either use one of the [already provided command runners](#command-runners), like for the [Go toolchain][go-pkg-task/golang] and [gotool][go-pkg-task/gotool], or [implement your own custom runner](#custom-runners). #### Custom Runners @@ -295,9 +291,6 @@ The guide also includes information about [minimal, complete, and verifiable exa [contrib-guide-versioning]: https://github.com/svengreb/wand/blob/main/CONTRIBUTING.md#versioning [contrib-guide]: https://github.com/svengreb/wand/blob/main/CONTRIBUTING.md [gh-blob-magefile/mage/go.mod]: https://github.com/magefile/mage/blob/d30a2cfe/go.mod -[gh-golang/go-ms-145]: https://github.com/golang/go/milestone/145 -[gh-golang/go-wiki-mods#tool_deps]: https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module -[gh-golang/go-wiki]: https://github.com/golang/go/wiki [gh-golang/go#25922]: https://github.com/golang/go/issues/25922 [gh-golang/go#27653]: https://github.com/golang/go/issues/27653 [gh-golang/go#30515]: https://github.com/golang/go/issues/30515 @@ -315,7 +308,7 @@ The guide also includes information about [minimal, complete, and verifiable exa [gnu-make-docs-shell]: https://www.gnu.org/software/make/manual/html_node/Choosing-the-Shell.html [gnu-make-repo]: https://savannah.gnu.org/git/?group=make [go-docs-ref-mod]: https://golang.org/ref/mod -[go-docs-tip-rln-1.16#mods]: https://tip.golang.org/doc/go1.16#modules +[go-docs-rln-1.16#modules]: https://golang.org/doc/go1.16#modules [go-pkg-al-github.com/magefile/mage/mg#namespace]: https://pkg.go.dev/github.com/magefile/mage/mg#Namespace [go-pkg-al-task#kind]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#Kind [go-pkg-al-task#kindbase]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#KindBase @@ -328,7 +321,6 @@ The guide also includes information about [minimal, complete, and verifiable exa [go-pkg-cmd/go#env_vars]: https://pkg.go.dev/cmd/go/#hdr-Environment_variables [go-pkg-cmd/go#env]: https://pkg.go.dev/cmd/go#hdr-Print_Go_environment_information [go-pkg-cmd/go#install]: https://pkg.go.dev/cmd/go#hdr-Compile_and_install_packages_and_dependencies -[go-pkg-cmd/go#mod_cmds]: https://golang.org/ref/mod#mod-commands [go-pkg-cmd/go#print_env]: https://pkg.go.dev/cmd/go/#hdr-Print_Go_environment_information [go-pkg-cmd/go#test]: https://pkg.go.dev/cmd/go#hdr-Test_packages [go-pkg-cmd/gofmt]: https://pkg.go.dev/cmd/gofmt @@ -336,10 +328,9 @@ The guide also includes information about [minimal, complete, and verifiable exa [go-pkg-fn-elder#new]: https://pkg.go.dev/github.com/svengreb/wand/pkg/elder#New [go-pkg-fn-project#new]: https://pkg.go.dev/github.com/svengreb/wand/pkg/project#New [go-pkg-func-app#newstore]: https://pkg.go.dev/github.com/svengreb/wand/pkg/app#NewStore -[go-pkg-func-os#usercachedir]: https://pkg.go.dev/os/#UserCacheDir [go-pkg-github.com/golangci/golangci-lint/cmd/golangci-lint]: https://pkg.go.dev/github.com/golangci/golangci-lint/cmd/golangci-lint [go-pkg-github.com/mitchellh/gox]: https://pkg.go.dev/github.com/mitchellh/gox -[go-pkg-github.com/myitcv/gobin]: https://pkg.go.dev/github.com/myitcv/gobin +[go-pkg-github.com/mvdan/gofumpt]: https://github.com/mvdan/gofumpt [go-pkg-golang.org/x/tools/cmd/goimports]: https://pkg.go.dev/golang.org/x/tools/cmd/goimports [go-pkg-if-app#store]: https://pkg.go.dev/github.com/svengreb/wand/pkg/app#Store [go-pkg-if-project/vcs#repository]: https://pkg.go.dev/github.com/svengreb/wand/pkg/project/vcs#Repository @@ -356,8 +347,8 @@ The guide also includes information about [minimal, complete, and verifiable exa [go-pkg-project/vcs/git]: https://pkg.go.dev/github.com/svengreb/wand/pkg/project/vcs/git [go-pkg-stc-app#config]: https://pkg.go.dev/github.com/svengreb/wand/pkg/app#Config [go-pkg-stc-project#metadata]: https://pkg.go.dev/github.com/svengreb/wand/pkg/project#Metadata -[go-pkg-stc-task/gobin#runner]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/gobin#Runner [go-pkg-stc-task/golang#runner]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/golang#Runner +[go-pkg-stc-task/gotool#runner]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/gotool#Runner [go-pkg-task]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task [go-pkg-task/fs/clean]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/fs/clean [go-pkg-task/gofumpt]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/gofumpt @@ -368,12 +359,11 @@ The guide also includes information about [minimal, complete, and verifiable exa [go-pkg-task/golang/install]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/golang/install [go-pkg-task/golang/test]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/golang/test [go-pkg-task/golangcilint]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/golangcilint +[go-pkg-task/gotool]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/gotool [go-pkg-task/gox]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task/gox [go-pkg-wand]: https://pkg.go.dev/github.com/svengreb/wand -[go-ref-mod]: https://golang.org/ref/mod -[go-ref-mod#go.mod]: https://golang.org/ref/mod#go-mod-file -[gobin-wiki-faq]: https://github.com/myitcv/gobin/wiki/FAQ -[gobin]: https://github.com/myitcv/gobin +[golang/go#42088]: https:github.com/golang/go/issues/42088 +[golang/go#44469#c-784534876]: https:github.com/golang/go/issues/44469#issuecomment-784534876 [gradle]: https://gradle.org [linux]: https://www.kernel.org [mage-deps]: https://magefile.org/dependencies @@ -385,7 +375,10 @@ The guide also includes information about [minimal, complete, and verifiable exa [mage]: https://magefile.org [make]: https://www.gnu.org/software/make [maven]: https://maven.apache.org +[node]: https://nodejs.org [npm-com]: https://npm.community +[npm-docs-cli-v7-config-folders#node_modules]: https://docs.npmjs.com/cli/v7/configuring-npm/folders#node-modules +[npm]: https://www.npmjs.com [rust-docs-cargo]: https://doc.rust-lang.org/stable/cargo [trunkbasedev-monorepos]: https://trunkbaseddevelopment.com/monorepos [wikip-dsl]: https://en.wikipedia.org/wiki/Domain-specific_language diff --git a/examples/custom_runner/runners.go b/examples/custom_runner/runners.go index 4014d54..bf679fe 100644 --- a/examples/custom_runner/runners.go +++ b/examples/custom_runner/runners.go @@ -53,7 +53,7 @@ func (r *FruitMixerRunner) Handles() task.Kind { // Run runs the command. // It returns an error of type *task.ErrRunner when any error occurs during the command execution. func (r *FruitMixerRunner) Run(t task.Task) error { - tExec, tErr := r.runPrepare(t) + tExec, tErr := r.prepareTask(t) if tErr != nil { return tErr } @@ -67,7 +67,7 @@ func (r *FruitMixerRunner) Run(t task.Task) error { // RunOut runs the command and returns its output. // It returns an error of type *task.ErrRunner when any error occurs during the command execution. func (r *FruitMixerRunner) RunOut(t task.Task) (string, error) { - tExec, tErr := r.runPrepare(t) + tExec, tErr := r.prepareTask(t) if tErr != nil { return "", tErr } @@ -102,8 +102,8 @@ func (r *FruitMixerRunner) Validate() error { return nil } -// runPrepare checks if the given task is of type task.Exec and prepares the task specific environment. -func (r *FruitMixerRunner) runPrepare(t task.Task) (task.Exec, error) { +// prepareTask checks if the given task is of type task.Exec and prepares the task specific environment. +func (r *FruitMixerRunner) prepareTask(t task.Task) (task.Exec, error) { tExec, ok := t.(task.Exec) if t.Kind() != task.KindExec || !ok { return nil, &task.ErrRunner{ diff --git a/go.mod b/go.mod index 0e679ff..259d0bc 100644 --- a/go.mod +++ b/go.mod @@ -1,11 +1,12 @@ module github.com/svengreb/wand -go 1.15 +go 1.16 require ( github.com/Masterminds/semver/v3 v3.1.1 github.com/imdario/mergo v0.3.12 github.com/magefile/mage v1.11.0 + github.com/pkg/errors v0.9.1 // indirect github.com/stretchr/testify v1.7.0 github.com/svengreb/golib v0.1.0 github.com/svengreb/nib v0.2.0 diff --git a/go.sum b/go.sum index d066686..db2e85c 100644 --- a/go.sum +++ b/go.sum @@ -1,4 +1,3 @@ -github.com/Masterminds/semver/v3 v3.1.0 h1:Y2lUDsFKVRSYGojLJ1yLxSXdMmMYTYls0rCvoqmMUQk= github.com/Masterminds/semver/v3 v3.1.0/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs= github.com/Masterminds/semver/v3 v3.1.1 h1:hLg3sBzpNErnxhQtUy/mmLR2I9foDujNK030IGemrRc= github.com/Masterminds/semver/v3 v3.1.1/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs= @@ -16,7 +15,6 @@ github.com/emirpasic/gods v1.12.0 h1:QAUIPSaCu4G+POclxeqb3F+WPpdKqFGlw36+yOzGlrg github.com/emirpasic/gods v1.12.0/go.mod h1:YfzfFFoVP/catgzJb4IKIqXjX78Ha8FMSDh3ymbK86o= github.com/fatih/color v1.10.0 h1:s36xzo75JdqLaaWoiEHk767eHiwo0598uUxyfiPkDsg= github.com/fatih/color v1.10.0/go.mod h1:ELkj/draVOlAH/xkhN6mQ50Qd0MPOk5AAr3maGEBuJM= -github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568 h1:BHsljHzVlRcyQhjrss6TZTdY2VfCqZPbv5k3iBFa2ZQ= github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc= github.com/gliderlabs/ssh v0.2.2 h1:6zsha5zo/TWhRhwqCD3+EarCAgZ2yN28ipRnGPnwkI0= github.com/gliderlabs/ssh v0.2.2/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0= @@ -30,7 +28,6 @@ github.com/go-git/go-git/v5 v5.2.0 h1:YPBLG/3UK1we1ohRkncLjaXWLW+HKp5QNM/jTli2Jg github.com/go-git/go-git/v5 v5.2.0/go.mod h1:kh02eMX+wdqqxgNMEyq8YgwlIOsDOa9homkUq1PoTMs= github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= -github.com/imdario/mergo v0.3.9 h1:UauaLniWCFHWd+Jp9oCEkTBj8VO/9DKg3PV3VCNMDIg= github.com/imdario/mergo v0.3.9/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= @@ -54,15 +51,15 @@ github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= -github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0= github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= -github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0= github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= @@ -97,7 +94,6 @@ gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8 gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME= gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I= gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU= gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= diff --git a/internal/support/exec/exec.go b/internal/support/exec/exec.go new file mode 100644 index 0000000..d29f50d --- /dev/null +++ b/internal/support/exec/exec.go @@ -0,0 +1,48 @@ +// Copyright (c) 2019-present Sven Greb +// This source code is licensed under the MIT license found in the LICENSE file. + +package exec + +import ( + "fmt" + "os" + "os/exec" + "path/filepath" + + glFS "github.com/svengreb/golib/pkg/io/fs" + + taskGo "github.com/svengreb/wand/pkg/task/golang" +) + +// GetGoExecPath looks up the executable search path(s) of the current environment for the Go executable with the given +// name. It looks up the paths defined in the system "PATH" environment variable and continues with the Go specific +// "GOBIN" path. +// See https://pkg.go.dev/cmd/go#hdr-Environment_variables for more details about Go specific environment variables. +func GetGoExecPath(name string) (string, error) { + // Look up the system executable search path(s)... + execPath, pathErr := exec.LookPath(name) + os.Environ() + + // ...and continue with the Go specific executable search path. + if pathErr != nil { + var execDir string + + if execDir = os.Getenv(taskGo.DefaultEnvVarGOBIN); execDir == "" { + if execDir = os.Getenv(taskGo.DefaultEnvVarGOPATH); execDir != "" { + execDir = filepath.Join(execDir, taskGo.DefaultGOBINSubDirName) + } + } + + execPath = filepath.Join(execDir, name) + execExits, fsErr := glFS.RegularFileExists(execPath) + if fsErr != nil { + return "", fmt.Errorf("check if %q exists: %w", execPath, fsErr) + } + + if !execExits { + return "", fmt.Errorf("%q not found in executable search path(s): %v", name, append(os.Environ(), execDir)) + } + } + + return execPath, nil +} diff --git a/pkg/elder/elder.go b/pkg/elder/elder.go index 76d8619..39c387a 100644 --- a/pkg/elder/elder.go +++ b/pkg/elder/elder.go @@ -20,13 +20,13 @@ import ( "github.com/svengreb/wand/pkg/project" "github.com/svengreb/wand/pkg/task" taskFSClean "github.com/svengreb/wand/pkg/task/fs/clean" - taskGobin "github.com/svengreb/wand/pkg/task/gobin" taskGofumpt "github.com/svengreb/wand/pkg/task/gofumpt" taskGoimports "github.com/svengreb/wand/pkg/task/goimports" taskGo "github.com/svengreb/wand/pkg/task/golang" taskGoBuild "github.com/svengreb/wand/pkg/task/golang/build" taskGoTest "github.com/svengreb/wand/pkg/task/golang/test" taskGolangCILint "github.com/svengreb/wand/pkg/task/golangcilint" + taskGoTool "github.com/svengreb/wand/pkg/task/gotool" taskGox "github.com/svengreb/wand/pkg/task/gox" ) @@ -34,26 +34,38 @@ import ( // for applications of a project. type Elder struct { nib.Nib - - as app.Store - gobinRunner *taskGobin.Runner - goRunner *taskGo.Runner - opts *Options - project *project.Metadata + as app.Store + goRunner *taskGo.Runner + goToolRunner *taskGoTool.Runner + opts *Options + project *project.Metadata } -// Bootstrap runs initialization tasks to ensure the wand is operational. -// If an error occurs it will be of type *task.ErrRunner. -func (e *Elder) Bootstrap() error { - if valErr := e.gobinRunner.Validate(); valErr != nil { - e.Infof("Installing %q", e.gobinRunner.GoMod()) - if installErr := e.gobinRunner.Install(e.goRunner); installErr != nil { - e.Errorf("Failed to install %q: %v", e.gobinRunner.GoMod(), installErr) - return fmt.Errorf("install %q: %w", e.gobinRunner.GoMod(), installErr) +// Bootstrap runs initialization tasks to ensure the wand is operational and sets up the local development environment +// by allowing to install executables from Go module-based "main" packages. +// The paths must be valid Go module import paths, that can optionally include the version suffix, in the "pkg@version" +// format. See https://pkg.go.dev/github.com/svengreb/wand/pkg/task/gotool for more details about the installation +// runner. +// It returns a slice of errors with type *task.ErrRunner containing any error that occurs during the execution. +func (e *Elder) Bootstrap(goModuleImportPaths ...string) []error { + var errs []error + for _, r := range []task.Runner{e.goRunner, e.goToolRunner} { + if err := r.Validate(); err != nil { + errs = append(errs, err) } } - return nil + for _, path := range goModuleImportPaths { + gm, gmErr := project.GoModuleFromImportPath(path) + if gmErr != nil { + errs = append(errs, gmErr) + } + if installErr := e.goToolRunner.Install(gm); installErr != nil { + errs = append(errs, installErr) + } + } + + return errs } // Clean is a task to remove filesystem paths, e.g. output data like artifacts and reports from previous development, @@ -66,11 +78,8 @@ func (e *Elder) Clean(appName string, opts ...taskFSClean.Option) ([]string, err if acErr != nil { return []string{}, fmt.Errorf("get %q application configuration: %w", appName, acErr) } - t, tErr := taskFSClean.New(e.GetProjectMetadata(), ac, opts...) - if tErr != nil { - return []string{}, tErr - } + t := taskFSClean.New(e.GetProjectMetadata(), ac, opts...) return t.Clean() } @@ -147,10 +156,10 @@ func (e *Elder) GoBuild(appName string, opts ...taskGoBuild.Option) error { func (e *Elder) Gofumpt(opts ...taskGofumpt.Option) error { t, tErr := taskGofumpt.New(opts...) if tErr != nil { - return tErr + return fmt.Errorf(`create "gofumpt" task: %w`, tErr) } - return e.gobinRunner.Run(t) + return e.goToolRunner.Run(t) } // Goimports is a task for the "golang.org/x/tools/cmd/goimports" Go module command. @@ -165,10 +174,10 @@ func (e *Elder) Gofumpt(opts ...taskGofumpt.Option) error { func (e *Elder) Goimports(opts ...taskGoimports.Option) error { t, tErr := taskGoimports.New(opts...) if tErr != nil { - return tErr + return fmt.Errorf(`create "goimports" task: %w`, tErr) } - return e.gobinRunner.Run(t) + return e.goToolRunner.Run(t) } // GolangCILint is a task to run the "github.com/golangci/golangci-lint/cmd/golangci-lint" Go module @@ -185,10 +194,10 @@ func (e *Elder) Goimports(opts ...taskGoimports.Option) error { func (e *Elder) GolangCILint(opts ...taskGolangCILint.Option) error { t, tErr := taskGolangCILint.New(opts...) if tErr != nil { - return tErr + return fmt.Errorf(`create "golangci-lint" task: %w`, tErr) } - return e.gobinRunner.Run(t) + return e.goToolRunner.Run(t) } // GoTest is a task to run the Go toolchain "test" command. @@ -233,10 +242,10 @@ func (e *Elder) Gox(appName string, opts ...taskGox.Option) error { t, tErr := taskGox.New(ac, opts...) if tErr != nil { - return tErr + return fmt.Errorf(`create "gox" task: %w`, tErr) } - return e.gobinRunner.Run(t) + return e.goToolRunner.Run(t) } // RegisterApp creates and stores a new application configuration. @@ -289,10 +298,10 @@ func (e *Elder) RegisterApp(name, displayName, pathRel string) error { return nil } -// Validate ensures that all tasks are properly initialized and operational. +// Validate ensures that the wand is properly initialized and operational. // It returns an error of type *task.ErrRunner when the validation of any of the supported task fails. func (e *Elder) Validate() error { - for _, t := range []task.Runner{e.goRunner, e.gobinRunner} { + for _, t := range []task.Runner{e.goRunner} { if err := t.Validate(); err != nil { return fmt.Errorf("failed to validate runner: %w", err) } @@ -305,7 +314,8 @@ func (e *Elder) Validate() error { // // The module name is determined automatically using the "runtime/debug" package. // The absolute path to the root directory is automatically set based on the current working directory. -// When the WithGenWandDataDir option is set to `true` the directory for wand specific data will be auto-generated. +// When the WithDisableAutoGenWandDataDir option is set to `false` the auto-generation of the directory for wand +// specific data will be disabled. // Note that the working directory must be set manually when the "magefile" is not placed in the root directory by // pointing Mage to it: // - "-d " option to set the directory from which "magefiles" are read (defaults to "."). @@ -336,11 +346,23 @@ func New(opts ...Option) (*Elder, error) { e.goRunner = taskGo.NewRunner(e.opts.goRunnerOpts...) - gobinRunner, gobinRunnerErr := taskGobin.NewRunner(e.opts.gobinRunnerOpts...) - if gobinRunnerErr != nil { - return nil, fmt.Errorf("failed to create %q runner: %w", "gobin", gobinRunnerErr) + goToolRunnerOpts := append( + []taskGoTool.RunnerOption{ + taskGoTool.WithToolsBinDir(filepath.Join(e.project.Options().WandDataDir, DefaultGoToolsBinDir)), + }, + e.opts.goToolRunnerOpts..., + ) + goToolRunner, goToolRunnerErr := taskGoTool.NewRunner(e.goRunner, goToolRunnerOpts...) + if goToolRunnerErr != nil { + return nil, fmt.Errorf("create %q runner: %w", taskGoTool.RunnerName, goToolRunnerErr) + } + e.goToolRunner = goToolRunner + + if !e.opts.disableAutoGenWandDataDir { + if err := generateWandDataDir(e.project.Options().WandDataDir); err != nil { + return nil, fmt.Errorf("generate wand specific data directory %q: %w", e.project.Options().WandDataDir, err) + } } - e.gobinRunner = gobinRunner if err := e.RegisterApp(e.project.Options().Name, e.project.Options().DisplayName, project.AppRelPath); err != nil { e.ExitPrintf(1, nib.ErrorVerbosity, "registering application %q: %v", e.project.Options().Name, err) diff --git a/pkg/elder/gitignore.tmpl b/pkg/elder/gitignore.tmpl new file mode 100644 index 0000000..a0d3510 --- /dev/null +++ b/pkg/elder/gitignore.tmpl @@ -0,0 +1,2 @@ +# Do not track cached data like compiled executables of Go module-based "main" packages. +cache/ diff --git a/pkg/elder/options.go b/pkg/elder/options.go index 2d6a64c..f8cb18a 100644 --- a/pkg/elder/options.go +++ b/pkg/elder/options.go @@ -4,12 +4,30 @@ package elder import ( + //nolint:golint // Idiomatic for the Go 1.16 "embed" package. + _ "embed" + "fmt" + "os" + "path/filepath" + + glFS "github.com/svengreb/golib/pkg/io/fs" "github.com/svengreb/nib" "github.com/svengreb/nib/inkpen" "github.com/svengreb/wand/pkg/project" - taskGobin "github.com/svengreb/wand/pkg/task/gobin" taskGo "github.com/svengreb/wand/pkg/task/golang" + taskGoTool "github.com/svengreb/wand/pkg/task/gotool" +) + +var ( + // DefaultGoToolsBinDir is the default directory for compiled executables of Go module-based "main" packages. + DefaultGoToolsBinDir = filepath.Join(project.DefaultWandCacheDataDir, "tools", "bin") + + // wandDataGitIgnoreFileName is the name for the written wandDataGitIgnoreTmpl file. + wandDataGitIgnoreFileName = ".gitignore" + + //go:embed gitignore.tmpl + wandDataGitIgnoreTmpl []byte ) // Option is a wand option. @@ -17,12 +35,16 @@ type Option func(*Options) // Options are wand options. type Options struct { - // gobinRunnerOpts are "gobin" runner options. - gobinRunnerOpts []taskGobin.RunnerOption + // disableAutoGenWandDataDir indicates whether the auto-generation of the directory for wand specific data should be + // disabled. + disableAutoGenWandDataDir bool // goRunnerOpts are Go toolchain runner options. goRunnerOpts []taskGo.RunnerOption + // goToolRunnerOpts are Go module-based tool runner options. + goToolRunnerOpts []taskGoTool.RunnerOption + // nib is the log-level based line printer for human-facing messages. nib nib.Nib @@ -40,10 +62,11 @@ func NewOptions(opts ...Option) *Options { return opt } -// WithGobinRunnerOptions sets "gobin" runner options. -func WithGobinRunnerOptions(opts ...taskGobin.RunnerOption) Option { +// WithDisableAutoGenWandDataDir indicates whether the auto-generation of the directory for wand specific data should be +// disabled. +func WithDisableAutoGenWandDataDir(disableAutoGenWandDataDir bool) Option { return func(o *Options) { - o.gobinRunnerOpts = append(o.gobinRunnerOpts, opts...) + o.disableAutoGenWandDataDir = disableAutoGenWandDataDir } } @@ -54,6 +77,13 @@ func WithGoRunnerOptions(opts ...taskGo.RunnerOption) Option { } } +// WithGoToolRunnerOptions sets Go module-based tool runner options. +func WithGoToolRunnerOptions(opts ...taskGoTool.RunnerOption) Option { + return func(o *Options) { + o.goToolRunnerOpts = append(o.goToolRunnerOpts, opts...) + } +} + // WithNib sets the log-level based line printer for human-facing messages. func WithNib(n nib.Nib) Option { return func(o *Options) { @@ -69,3 +99,23 @@ func WithProjectOptions(opts ...project.Option) Option { o.projectOpts = append(o.projectOpts, opts...) } } + +// generateWandDataDir generates the wand specific data directory structure and files. +func generateWandDataDir(path string) error { + if err := os.MkdirAll(path, os.ModePerm); err != nil { + return fmt.Errorf("make %q directory structure: %w", path, err) + } + + gitIgnoreFilePath := filepath.Join(path, wandDataGitIgnoreFileName) + gitIgnoreExists, fsErr := glFS.RegularFileExists(gitIgnoreFilePath) + if fsErr != nil { + return fmt.Errorf("check regular file %q: %w", gitIgnoreFilePath, fsErr) + } + if !gitIgnoreExists { + if err := os.WriteFile(gitIgnoreFilePath, wandDataGitIgnoreTmpl, os.ModePerm); err != nil { + return fmt.Errorf("write %q: %w", gitIgnoreFilePath, err) + } + } + + return nil +} diff --git a/pkg/project/gomodule.go b/pkg/project/gomodule.go index 9a635c3..1058e9e 100644 --- a/pkg/project/gomodule.go +++ b/pkg/project/gomodule.go @@ -5,15 +5,19 @@ package project import ( "fmt" + "path/filepath" + "strings" "github.com/Masterminds/semver/v3" ) const ( // GoModuleVersionLatest is the "version query suffix" for the latest version of a Go module. - // // See https://golang.org/ref/mod#version-queries for more details. GoModuleVersionLatest = "latest" + + // GoModuleVersionSuffixSeparator is the character that separates the Go module version from a import path. + GoModuleVersionSuffixSeparator = "@" ) // GoModuleID stores partial information to identify a Go module. @@ -46,9 +50,41 @@ type GoModuleID struct { Version *semver.Version } +// ExecName returns the name of the compiled executable when the Go module Path is a "main" package. +func (gm GoModuleID) ExecName() string { + return filepath.Base(gm.Path) +} + func (gm GoModuleID) String() string { if gm.Version != nil && !gm.IsLatest { return fmt.Sprintf("%s@%s", gm.Path, gm.Version.Original()) } return fmt.Sprintf("%s@%s", gm.Path, GoModuleVersionLatest) } + +// GoModuleFromImportPath creates a GoModuleID from the given import path. +// The path must be a valid Go module import path, that can optionally include the version suffix, in the "pkg@version" +// format. +func GoModuleFromImportPath(importPath string) (*GoModuleID, error) { + pathElements := strings.Split(importPath, GoModuleVersionSuffixSeparator) + if len(pathElements) == 0 { + return nil, fmt.Errorf("invalid import path: %q", importPath) + } + + gm := &GoModuleID{Path: pathElements[0]} + // Handle as latest Go module version when the import path has no separator or the suffix equals "latest". + if len(pathElements) == 1 || pathElements[len(pathElements)-1] == GoModuleVersionLatest { + gm.IsLatest = true + return gm, nil + } + + version, semVerErr := semver.NewVersion(pathElements[1]) + if semVerErr != nil { + return nil, &ErrProject{ + Err: fmt.Errorf("parse version from import path %q: %w", importPath, semVerErr), + Kind: ErrDetermineGoModuleInformation, + } + } + gm.Version = version + return gm, nil +} diff --git a/pkg/project/options.go b/pkg/project/options.go index ce093dc..2f5ce72 100644 --- a/pkg/project/options.go +++ b/pkg/project/options.go @@ -22,6 +22,12 @@ const ( // and production artifacts as well as distribution bundles, static web files or metric/statistic reports. DefaultBaseOutputDir = "out" + // DefaultWandCacheDataDir is the default directory for wand specific cache data. + DefaultWandCacheDataDir = "cache" + + // DefaultWandDir is the default directory for wand specific data. + DefaultWandDir = ".wand" + // DefaultVersion is the default version for a project vcs.Repository. DefaultVersion = "v0.0.0" ) @@ -52,6 +58,9 @@ type Options struct { // VCSKind is the VCS kind of the project Repository. VCSKind vcs.Kind + + // WandDataDir is the path to the directory for wand specific data. + WandDataDir string } // Option is a project option. @@ -110,6 +119,13 @@ func WithVCSKind(kind vcs.Kind) Option { } } +// WithWandDataDir sets the path to the directory for wand specific data. +func WithWandDataDir(wandDataDir string) Option { + return func(o *Options) { + o.WandDataDir = wandDataDir + } +} + // newOptions creates new project options. // The absolute path to the root directory is automatically set based on the current working directory and the Go module // name is automatically determined using the runtime/debug package. @@ -144,6 +160,7 @@ func newOptions(opts ...Option) (*Options, error) { Repository: vcsNone.New(), RootDirPathAbs: rootDirPath, VCSKind: vcs.KindNone, + WandDataDir: filepath.Join(rootDirPath, DefaultWandDir), } for _, o := range opts { o(opt) diff --git a/pkg/task/fs/clean/clean.go b/pkg/task/fs/clean/clean.go index 07b29af..21fd7f9 100644 --- a/pkg/task/fs/clean/clean.go +++ b/pkg/task/fs/clean/clean.go @@ -86,11 +86,6 @@ func (t *Task) Options() task.Options { // New creates a new task. //nolint:gocritic // The app.Config struct is passed as value by design to ensure immutability. -func New(proj project.Metadata, ac app.Config, opts ...Option) (*Task, error) { - opt, optErr := NewOptions(opts...) - if optErr != nil { - return nil, fmt.Errorf("create %q task options: %w", taskName, optErr) - } - - return &Task{ac: ac, proj: proj, opts: opt}, nil +func New(proj project.Metadata, ac app.Config, opts ...Option) *Task { + return &Task{ac: ac, proj: proj, opts: NewOptions(opts...)} } diff --git a/pkg/task/fs/clean/options.go b/pkg/task/fs/clean/options.go index fda6df9..03c58a3 100644 --- a/pkg/task/fs/clean/options.go +++ b/pkg/task/fs/clean/options.go @@ -27,7 +27,7 @@ type Options struct { } // NewOptions creates new task options. -func NewOptions(opts ...Option) (*Options, error) { +func NewOptions(opts ...Option) *Options { opt := &Options{ name: taskName, } @@ -35,7 +35,7 @@ func NewOptions(opts ...Option) (*Options, error) { o(opt) } - return opt, nil + return opt } // WithLimitToAppOutputDir indicates whether only paths within the configured application output directory should be diff --git a/pkg/task/golang/golang.go b/pkg/task/golang/golang.go index 22956ee..dfb1eaa 100644 --- a/pkg/task/golang/golang.go +++ b/pkg/task/golang/golang.go @@ -34,9 +34,9 @@ func (r *Runner) Handles() task.Kind { // Run runs the command. // It returns an error of type *task.ErrRunner when any error occurs during the command execution. func (r *Runner) Run(t task.Task) error { - tExec, tErr := r.runPrepare(t) + tExec, tErr := r.prepareTask(t) if tErr != nil { - return tErr + return fmt.Errorf("runner %q: %w", RunnerName, tErr) } if r.opts.Quiet { @@ -48,9 +48,9 @@ func (r *Runner) Run(t task.Task) error { // RunOut runs the command and returns its output. // It returns an error of type *task.ErrRunner when any error occurs during the command execution. func (r *Runner) RunOut(t task.Task) (string, error) { - tExec, tErr := r.runPrepare(t) + tExec, tErr := r.prepareTask(t) if tErr != nil { - return "", tErr + return "", fmt.Errorf("runner %q: %w", RunnerName, tErr) } return sh.OutputWith(r.opts.Env, r.opts.Exec, tExec.BuildParams()...) @@ -64,7 +64,7 @@ func (r *Runner) Validate() error { execExits, fsErr := glFS.RegularFileExists(r.opts.Exec) if fsErr != nil { return &task.ErrRunner{ - Err: fmt.Errorf("command runner %q: %w", RunnerName, fsErr), + Err: fmt.Errorf("runner %q: %w", RunnerName, fsErr), Kind: task.ErrRunnerValidation, } } @@ -73,7 +73,7 @@ func (r *Runner) Validate() error { path, pathErr := exec.LookPath(r.opts.Exec) if pathErr != nil { return &task.ErrRunner{ - Err: fmt.Errorf("command runner %q: %q not found in PATH: %w", RunnerName, r.opts.Exec, pathErr), + Err: fmt.Errorf("runner %q: %q not found in PATH: %w", RunnerName, r.opts.Exec, pathErr), Kind: task.ErrRunnerValidation, } } @@ -83,8 +83,9 @@ func (r *Runner) Validate() error { return nil } -// runPrepare checks if the given task is of type task.Exec and prepares the task specific environment. -func (r *Runner) runPrepare(t task.Task) (task.Exec, error) { +// prepareTask checks if the given task is of type task.Exec and prepares the task specific environment. +// It returns an error of type *task.ErrRunner when any error occurs during the execution. +func (r *Runner) prepareTask(t task.Task) (task.Exec, error) { tExec, ok := t.(task.Exec) if t.Kind() != task.KindExec || !ok { return nil, &task.ErrRunner{ diff --git a/pkg/task/golang/install/install.go b/pkg/task/golang/install/install.go index fec33c5..aa48777 100644 --- a/pkg/task/golang/install/install.go +++ b/pkg/task/golang/install/install.go @@ -1,8 +1,8 @@ -// +build go1.16 - // Copyright (c) 2019-present Sven Greb // This source code is licensed under the MIT license found in the LICENSE file. +// +build go1.16 + // Package install provides a task for the Go toolchain "install" command. // It requires at least Go version 1.16 which comes with support to install commands via `go install` (1) without // affecting the `main` module and will not "pollute" the `go.mod` file (2) anymore. diff --git a/pkg/task/gotool/gotool.go b/pkg/task/gotool/gotool.go new file mode 100644 index 0000000..193109d --- /dev/null +++ b/pkg/task/gotool/gotool.go @@ -0,0 +1,335 @@ +// Copyright (c) 2019-present Sven Greb +// This source code is licensed under the MIT license found in the LICENSE file. + +// +build go1.16 + +// Package gotool provides a runner to install and run compiled executables of Go module-based "main" packages. +// +// Go Executable Installation +// +// As of Go 1.16 `go install` supports the `pkg@version` syntax [1] which allows to install commands without "polluting" +// a projects `go.mod` file. The resulting executables are placed in the Go executable search path that is defined by +// the "GOBIN" environment variable [2] (see the "go env" command [3] to show or modify the Go toolchain environment). +// The problem is that installed executables will overwrite any previously installed executable of the same +// module/package regardless of the version. Therefore only one version of an executable can be installed at a time +// which makes it impossible to work on different projects that make use of the same executable but require different +// versions. +// +// UX Before Go 1.16 +// +// The installation concept for "main" package executables was always a somewhat controversial point which +// unfortunately, partly for historical reasons, did not offer an optimal and user-friendly solution until Go 1.16. +// The "go" command [4] is a fantastic toolchain that provides many great features one would expect to be provided +// out-of-the-box from a modern and well designed programming language without the requirement to use a third-party +// solution: from compiling code, running unit/integration/benchmark tests, quality and error analysis, debugging +// utilities and many more. +// This did not apply for the "go install" command [1] of Go versions less than 1.16. +// +// The general problem of tool dependencies was a long-time known issue/weak point of the Go toolchain and was a highly +// rated change request from the Go community with discussions like https://github.com/golang/go/issues/30515, +// https://github.com/golang/go/issues/25922 and https://github.com/golang/go/issues/27653 to improve this essential +// feature. They have been around for quite a long time without a solution that worked without introducing breaking +// changes and most users and the Go team agree on. +// Luckily, this topic was finally resolved in the Go release version 1.16 [5] and +// https://github.com/golang/go/issues/40276 introduced a way to install executables in module mode outside a module. +// +// The Leftover Drawback +// +// Even though the "go install" command works totally fine to globally install executables, the problem that only a +// single version can be installed at a time is still left. The executable is placed in the path defined by +// "go env GOBIN" so the previously installed executable will be overridden. It is not possible to install multiple +// versions of the same package and "go install" still messes up the local user environment. +// +// The Workaround +// +// To work around the leftover drawback, this package provides a runner that uses "go install" under the +// hood, but allows to place the compiled executable in a custom cache directory instead of `go env GOBIN`. It checks if the +// executable already exists, installs it if not so, and executes it afterwards. +// +// The concept of storing dependencies locally on a per-project basis is well-known from the "node_modules" +// directory [6] of the Node [7] package manager npm [8]. Storing executables in a cache directory within the +// repository (not tracked by Git) allows to use "go install" mechanisms while not affect the global user environment +// and executables stored in "go env GOBIN". +// The runner achieves this by temporarily changing the "GOBIN" environment variable to the custom cache directory +// during the execution of "go install". +// +// The only known disadvantage is the increased usage of storage disk space, but since most Go executables are small in +// size anyway, this is perfectly acceptable compared to the clearly outweighing advantages. +// +// Note that the runner dynamically runs executables based on the given task so the "Validate" method is a NOOP. +// +// Future Changes +// +// The provided runner is still not a clean solution that uses the Go toolchain without any special logic so as soon as +// the following changes are made to the Go toolchain (Go 1.17 or later), the runner will be removed again: +// +// - https://github.com/golang/go/issues/42088 tracks the process of adding support for the Go module syntax to the +// "go run" command. This will allow to let the Go toolchain handle the way how compiled executable are stored, +// located and executed. +// - https://github.com/golang/go/issues/44469#issuecomment-784534876 tracks the process of making "go install" aware of +// the "-o" flag like the "go build" command which is the only reason why the provided runner exists. +// +// References +// +// [1]: https://pkg.go.dev/cmd/go#hdr-Compile_and_install_packages_and_dependencies +// [2]: https://pkg.go.dev/cmd/go/#hdr-Environment_variables +// [3]: https://pkg.go.dev/cmd/go/#hdr-Print_Go_environment_information +// [4]: https://pkg.go.dev/cmd/go +// [5]: https://golang.org/doc/go1.16#modules +// [6]: https://docs.npmjs.com/cli/v7/configuring-npm/folders#node-modules +// [7]: https://nodejs.org +// [8]: https://www.npmjs.com +package gotool + +import ( + "fmt" + "os" + "path/filepath" + + "github.com/magefile/mage/sh" + glFS "github.com/svengreb/golib/pkg/io/fs" + + osSupport "github.com/svengreb/wand/internal/support/os" + "github.com/svengreb/wand/pkg/project" + "github.com/svengreb/wand/pkg/task" + taskGo "github.com/svengreb/wand/pkg/task/golang" + taskGoInstall "github.com/svengreb/wand/pkg/task/golang/install" +) + +// Runner is runner to install and run compiled executables of Go module-based "main" packages. +// +// Go Executable Installation +// +// As of Go 1.16 `go install` supports the `pkg@version` syntax [1] which allows to install commands without "polluting" +// a projects `go.mod` file. The resulting executables are placed in the Go executable search path that is defined by +// the "GOBIN" environment variable [2] (see the "go env" command [3] to show or modify the Go toolchain environment). +// The problem is that installed executables will overwrite any previously installed executable of the same +// module/package regardless of the version. Therefore only one version of an executable can be installed at a time +// which makes it impossible to work on different projects that make use of the same executable but require different +// versions. +// +// UX Before Go 1.16 +// +// The installation concept for "main" package executables was always a somewhat controversial point which +// unfortunately, partly for historical reasons, did not offer an optimal and user-friendly solution until Go 1.16. +// The "go" command [4] is a fantastic toolchain that provides many great features one would expect to be provided +// out-of-the-box from a modern and well designed programming language without the requirement to use a third-party +// solution: from compiling code, running unit/integration/benchmark tests, quality and error analysis, debugging +// utilities and many more. +// This did not apply for the "go install" command [1] of Go versions less than 1.16. +// +// The general problem of tool dependencies was a long-time known issue/weak point of the Go toolchain and was a highly +// rated change request from the Go community with discussions like https://github.com/golang/go/issues/30515, +// https://github.com/golang/go/issues/25922 and https://github.com/golang/go/issues/27653 to improve this essential +// feature. They have been around for quite a long time without a solution that worked without introducing breaking +// changes and most users and the Go team agree on. +// Luckily, this topic was finally resolved in the Go release version 1.16 [5] and +// https://github.com/golang/go/issues/40276 introduced a way to install executables in module mode outside a module. +// +// The Leftover Drawback +// +// Even though the "go install" command works totally fine to globally install executables, the problem that only a +// single version can be installed at a time is still left. The executable is placed in the path defined by +// "go env GOBIN" so the previously installed executable will be overridden. It is not possible to install multiple +// versions of the same package and "go install" still messes up the local user environment. +// +// The Workaround +// +// To work around the leftover drawback, this runner uses "go install" under the hood, but allows to place the compiled +// executable in a custom cache directory instead of `go env GOBIN`. It checks if the executable already exists, +// installs it if not so, and executes it afterwards. +// +// The concept of storing dependencies locally on a per-project basis is well-known from the "node_modules" +// directory [6] of the Node [7] package manager npm [8]. Storing executables in a cache directory within the +// repository (not tracked by Git) allows to use "go install" mechanisms while not affect the global user environment +// and executables stored in "go env GOBIN". +// The runner achieves this by temporarily changing the "GOBIN" environment variable to the custom cache directory +// during the execution of "go install". +// +// The only known disadvantage is the increased usage of storage disk space, but since most Go executables are small in +// size anyway, this is perfectly acceptable compared to the clearly outweighing advantages. +// +// Note that the runner dynamically runs executables based on the given task so the "Validate" method is a NOOP. +// +// Future Changes +// +// This runner is still not a clean solution that uses the Go toolchain without any special logic so as soon as the +// following changes are made to the Go toolchain (Go 1.17 or later), the runner will be removed again: +// +// - https://github.com/golang/go/issues/42088 tracks the process of adding support for the Go module syntax to the +// "go run" command. This will allow to let the Go toolchain handle the way how compiled executable are stored, +// located and executed. +// - https://github.com/golang/go/issues/44469#issuecomment-784534876 tracks the process of making "go install" aware of +// the "-o" flag like the "go build" command which is the only reason why this runner exists. +// +// References +// +// [1]: https://pkg.go.dev/cmd/go#hdr-Compile_and_install_packages_and_dependencies +// [2]: https://pkg.go.dev/cmd/go/#hdr-Environment_variables +// [3]: https://pkg.go.dev/cmd/go/#hdr-Print_Go_environment_information +// [4]: https://pkg.go.dev/cmd/go +// [5]: https://golang.org/doc/go1.16#modules +// [6]: https://docs.npmjs.com/cli/v7/configuring-npm/folders#node-modules +// [7]: https://nodejs.org +// [8]: https://www.npmjs.com +type Runner struct { + goRunner *taskGo.Runner + opts *RunnerOptions +} + +// Handles returns the supported task kind. +func (r *Runner) Handles() task.Kind { + return task.KindGoModule +} + +// Install installs the executable of the given Go module. +// It returns an error of type *task.ErrRunner when any error occurs during the command execution. +// +// // See https://pkg.go.dev/cmd/go#hdr-Compile_and_install_packages_and_dependencies for more details. +func (r *Runner) Install(goModule *project.GoModuleID) error { + _, err := r.prepareExec(goModule) + if err != nil { + return &task.ErrRunner{ + Err: fmt.Errorf("runner %q: %w", RunnerName, err), + Kind: task.ErrRun, + } + } + + return nil +} + +// Run runs the command. +// It returns an error of type *task.ErrRunner when any error occurs during the command execution. +func (r *Runner) Run(t task.Task) error { + tGM, tErr := r.prepareTask(t) + if tErr != nil { + return fmt.Errorf("runner %q: %w", RunnerName, tErr) + } + + execPath, preExecErr := r.prepareExec(tGM.ID()) + if preExecErr != nil { + return &task.ErrRunner{ + Err: fmt.Errorf("runner %q: %w", RunnerName, preExecErr), + Kind: task.ErrRunnerValidation, + } + } + + if r.opts.Quiet { + return sh.RunWith(r.opts.Env, execPath, tGM.BuildParams()...) + } + return sh.RunWithV(r.opts.Env, execPath, tGM.BuildParams()...) +} + +// RunOut runs the command and returns its output. +// It returns an error of type *task.ErrRunner when any error occurs during the command execution. +func (r *Runner) RunOut(t task.Task) (string, error) { + tGM, tErr := r.prepareTask(t) + if tErr != nil { + return "", fmt.Errorf("runner %q: %w", RunnerName, tErr) + } + + execPath, preExecErr := r.prepareExec(tGM.ID()) + if preExecErr != nil { + return "", &task.ErrRunner{ + Err: fmt.Errorf("runner %q: %w", RunnerName, preExecErr), + Kind: task.ErrRunnerValidation, + } + } + + return sh.OutputWith(r.opts.Env, execPath, tGM.BuildParams()...) +} + +// Validate validates the runner. +// This runner uses dynamic executables based on the given task so this method is a NOOP. +func (r *Runner) Validate() error { + return nil +} + +// buildExecDir builds and returns the path to the directory for the executable. +func (r *Runner) buildExecDir(goModule *project.GoModuleID) string { + path := filepath.Join(r.opts.toolsBinDir, goModule.ExecName()) + + if goModule.Version != nil && !goModule.IsLatest { + return filepath.Join(path, goModule.Version.String()) + } + + return filepath.Join(path, project.GoModuleVersionLatest) +} + +// install installs the compiled executable of a Go module-based "main" package. +// It returns an error of type *task.ErrRunner when any error occurs during the installation. +func (r *Runner) install(execDir string, goModule *project.GoModuleID) error { + env := osSupport.EnvSliceToMap(os.Environ()) + for k, v := range r.opts.Env { + env[k] = v + } + // Override the "GOBIN" environment variable to use the given path for the compiled executable. + env[taskGo.DefaultEnvVarGOBIN] = execDir + + t := taskGoInstall.New( + taskGoInstall.WithModulePath(goModule.Path), + taskGoInstall.WithModuleVersion(goModule.Version), + taskGoInstall.WithEnv(env), + ) + + if err := r.goRunner.Run(t); err != nil { + return fmt.Errorf("run %q: %w", t.Name(), err) + } + + return nil +} + +// prepareExec prepares the ensure that the executable exists and returns the path. +func (r *Runner) prepareExec(goModule *project.GoModuleID) (string, error) { + execDir := r.buildExecDir(goModule) + execPath := filepath.Join(execDir, goModule.ExecName()) + + exists, fsErr := glFS.RegularFileExists(execPath) + if fsErr != nil { + return "", fmt.Errorf("check executable %q: %w", execPath, fsErr) + } + + if !exists { + if err := os.MkdirAll(execDir, os.ModePerm); err != nil { + return "", fmt.Errorf("create directory structure %q for execuable: %w", execDir, err) + } + + if err := r.install(execDir, goModule); err != nil { + return "", fmt.Errorf("install executable %q: %w", execPath, err) + } + } + + return execPath, nil +} + +// prepareTask checks if the given task is of type task.GoModule and prepares the task specific environment. +// It returns an error of type *task.ErrRunner when any error occurs during the execution. +func (r *Runner) prepareTask(t task.Task) (task.GoModule, error) { + tGM, ok := t.(task.GoModule) + if t.Kind() != task.KindGoModule || !ok { + return nil, &task.ErrRunner{ + Err: fmt.Errorf("expected %q but got %q", r.Handles(), t.Kind()), + Kind: task.ErrUnsupportedTaskKind, + } + } + + for k, v := range tGM.Env() { + r.opts.Env[k] = v + } + + return tGM, nil +} + +// NewRunner creates a new command runner for Go module-based tools. +// It returns an error of type *task.ErrRunner when any error occurs during the creation. +func NewRunner(goRunner *taskGo.Runner, opts ...RunnerOption) (*Runner, error) { + opt, optErr := NewRunnerOptions(opts...) + if optErr != nil { + return nil, &task.ErrRunner{ + Err: fmt.Errorf("create %q runner options: %w", RunnerName, optErr), + Kind: task.ErrInvalidRunnerOpts, + } + } + + return &Runner{goRunner: goRunner, opts: opt}, nil +} diff --git a/pkg/task/gotool/options.go b/pkg/task/gotool/options.go new file mode 100644 index 0000000..90fc74d --- /dev/null +++ b/pkg/task/gotool/options.go @@ -0,0 +1,67 @@ +// Copyright (c) 2019-present Sven Greb +// This source code is licensed under the MIT license found in the LICENSE file. + +package gotool + +import ( + "fmt" + "path/filepath" +) + +const ( + // RunnerName is the name of the runner. + RunnerName = "gotool" +) + +// RunnerOption is a runner option. +type RunnerOption func(*RunnerOptions) + +// RunnerOptions are runner options. +type RunnerOptions struct { + // Env is the runner specific environment. + Env map[string]string + + // toolsBinDir is the path to the directory where compiled executables of Go module-based "main" packages are placed. + toolsBinDir string + + // Quiet indicates whether the runner output should be minimal. + Quiet bool +} + +// NewRunnerOptions creates new runner options. +func NewRunnerOptions(opts ...RunnerOption) (*RunnerOptions, error) { + opt := &RunnerOptions{ + Env: make(map[string]string), + } + for _, o := range opts { + o(opt) + } + + if !filepath.IsAbs(opt.toolsBinDir) { + return nil, fmt.Errorf("expect an absolute path for tool binaries directory, but got %q", opt.toolsBinDir) + } + + return opt, nil +} + +// WithEnv sets the runner specific environment. +func WithEnv(env map[string]string) RunnerOption { + return func(o *RunnerOptions) { + o.Env = env + } +} + +// WithToolsBinDir sets the path to the directory where compiled binaries of Go module-based tools are placed. +// Defaults to DefaultToolsBinDir. +func WithToolsBinDir(toolsBinDir string) RunnerOption { + return func(o *RunnerOptions) { + o.toolsBinDir = toolsBinDir + } +} + +// WithQuiet indicates whether the runner output should be minimal. +func WithQuiet(quiet bool) RunnerOption { + return func(o *RunnerOptions) { + o.Quiet = quiet + } +}