From d9cd6ee28d3c424f73410d4112dc9e081bfcb384 Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Tue, 12 Mar 2024 17:34:40 -0700 Subject: [PATCH 01/21] Added RFD 0144 - Client Tools Updates --- rfd/0144-client-tools-updates.md | 309 +++++++++++++++++++++++++++++++ 1 file changed, 309 insertions(+) create mode 100644 rfd/0144-client-tools-updates.md diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md new file mode 100644 index 0000000000000..e12a2545b2c76 --- /dev/null +++ b/rfd/0144-client-tools-updates.md @@ -0,0 +1,309 @@ +--- +authors: Russell Jones (rjones@goteleport.com) and Bernard Kim (bernard@goteleport.com) +state: draft +--- + +# RFD 0144 - Client Tools Updates + +## Required Approvers + +* Engineering: @sclevine && @bernardjkim && @r0mant +* Product: @klizhentas || @xinding33 +* Security: @reedloden + +## What/Why + +This RFD describes how client tools (like `tsh` and `tctl`) can be kept up to +date, either using automatic updates or self-managed updates. + +Keeping client tools updated helps with security (fixes for known security +vulnerabilities are pushed to endpoints), bugs (fixes for resolved issues are +pushed to endpoints), and compatibility (users no longer have to learn and +understand [Teleport component +compatibility](https://goteleport.com/docs/upgrading/overview/#component-compatibility) +rules). + +## Details + +### Summary + +Client tools (like `tsh` and `tctl`) will automatically download and install +the version of client tools recommended by the Teleport cluster. + +The `cluster_maintenance_config` resource will be updated to include a +`tools_version` field to control the version of client tools advertised by the +cluster. This resource will not be configurable on Teleport Cloud. It will be +managed by the Teleport Cloud team. + +If cluster administrators want to opt-out of automatic updates and manage +updates themselves, they will be able to watch the `cluster_maintenance_config` +resource for changes and push out updates to endpoints manually. + +Inspiration drawn from https://go.dev/doc/toolchain. + +### Implementation + +#### Client tools + +##### Automatic updates + +The first time a user executes `tsh login` with a version of `tsh` that +supports automatic updates, the user will be prompted to enroll in automatic +updates. This choice will be saved in client tools configuration. + +``` +$ tsh --proxy=proxy.example.com login +Keep client tools like tsh and tctl automatically updated? [YES/no] + +[...] +``` + +``` +$ cat ~/.tsh/config/config.yaml +enable_autoupdate: true | false +``` + +Once enrolled, upon login client tools will check the `tools_version` +advertised by the cluster at `/v1/webapi/ping`. If this version differs from +that of the running binary, the recommended version will be downloaded and +installed. + +The original binaries will not be overwritten by automatic updates, instead +per-cluster binaries with permissions `0555` will be stored at +`~/.tsh/bin/proxyName/{tctl,tsh}`. This will act as both a cache and allow +users to connect to different Teleport clusters (running different versions of +Teleport) without juggling multiple versions of client tools. + +``` +$ tree ~/.tsh +~/.tsh +├── bin +│ └── proxy.example.com +│ ├── tctl +│ └── tsh +├── current-profile +├── keys +│ └── proxy.example.com +│ ├── cas +│ │ └── example.com.pem +│ ├── certs.pem +│ ├── foo +│ ├── foo-ssh +│ │ └── example.com-cert.pub +│ ├── foo-x509.pem +│ └── foo.pub +├── known_hosts +└── proxy.example.com.yaml +``` + +Users can cancel client tools updates using `Ctrl-C`. This may be needed if the +user is on a low bandwidth connection (LTE or public Wi-Fi), if the Teleport +download server is inaccessible, or the user urgently needs to access the +cluster and can not wait for the update to occur. + +``` +$ tsh login --proxy=proxy.example.com +Client tools are out of date, updating to vX.Y.Z. +Update progress: [▒▒▒▒▒▒ ] (Ctrl-C to Cancel) + +[...] +``` + +After downloading, client tools will re-execute the downloaded version. The +child process will inherit all environment variables and flags. Re-execution +will occur only if the parent process is not `tsh` or `tctl`, to prevent +infinite loops. + +An environment variable `TELEPORT_TOOLS_VERSION` will be introduced to enable +overriding and using a specific version of client tools. This field can be used +either as an emergency workaround for a known issue or for debugging purposes + +##### Errors and warnings + +If the user does not enroll in automatic updates and the cluster administrator +does not self-manage client tools updates as outlined in [Self-managed client +tools updates](#self-managed-client-tools-updates), a series of warnings and +errors with increasing urgency will be shown to the user. + +If the version of client tools is within the same major version as advertised +by the cluster, a warning will be shown to urge the user to enroll in automatic +updates. Warnings will not prevent the user from using client tools that are +slightly out of date. + +``` +$ tsh login --proxy=proxy.example.com +Warning: Client tools are out of date, update to vX.Y.Z. + +Update Teleport to vX.Y.Z from https://goteleport.com/download or your system +package manager. + +Run "tsh autoupdate enroll" to enroll in automatic updates and keep client +tools like tsh and tctl automatically updated. + +[...] +``` + +If the version of client tools is 1 major version below the version advertised +by the cluster, a warning will be shown that indicates some functionality may +not work. + +``` +$ tsh login --proxy=proxy.example.com +WARNING: Client tools are 1 major version out of date, update to vX.Y.Z. + +Some functionality may not work. Update Teleport to vX.Y.Z from +https://goteleport.com/download or your system package manager. + +Run "tsh autoupdate enroll" to enroll in automatic updates and keep client +tools like tsh and tctl automatically updated. +``` + +If the version of client tools is 2 (or more) versions lower than the version +advertised by the cluster or 1 (or more) version greater than the version +advertised by the cluster, an error will be shown and will require the user to +use the `--skip-version-check` flag. + +``` +$ tsh login --proxy=proxy.example.com +ERROR: Client tools are N major versions out of date, update to vX.Y.Z. + +Some functionality will not work. Update Teleport to vX.Y.Z from +https://goteleport.com/download or your system package manager. + +Run "tsh autoupdate enroll" to enroll in automatic updates and keep client +tools like tsh and tctl automatically updated. + +Use the "--skip-version-check" flag to bypass this check and attempt to connect +to this cluster. +``` + +#### Self-managed client tools updates + +Cluster administrators that want to self-manage client tools updates will be +able to watch for changes to client tools versions which can then be used to +trigger other integrations (using MDM software like Jamf) to update the +installed version of client tools on endpoints. + +``` +$ tctl autoupdate watch +{"tools_version": "1.0.0"} +{"tools_version": "1.0.1"} +{"tools_version": "2.0.0"} + +[...] +``` + +Cluster administrators who prefer to use the Teleport API can create a watcher +on the `types.ClusterMaintenanceConfig` resource and trigger other integrations +on changes to the `tools_version` field. + +``` +// Create a watcher to monitor for changes to client tools version. +watch, err := p.TeleportClient.NewWatcher(ctx, types.Watch{ + Kinds: []types.WatchKind{ + types.WatchKind{Kind: types.ClusterMaintenanceConfig}, + }, +}) +if err != nil { + return trace.Wrap(err) +} +defer watch.Close() + +// Loop forever watching for client tools version changes. +for { + select { + case e := <-watch.Events(): + if e.Resource == nil { + continue + } + resource, ok := e.Resource.(types.ClusterMaintenanceConfig) + if !ok { + continue + } + + // Process the version change by notifying downstream tooling (like + // Jamf). + if err := handle(resource); err != nil { + log.Printf("Failed to handle version change: %v", err) + } + case <-watch.Done(): + fmt.Println("The watcher job is finished.") + return nil + } +} +``` + +##### Cluster configuration + +> [!NOTE] +> This resource will not be configurable on Teleport Cloud. It will be managed +> by the Teleport Cloud team. + +The `types.ClusterMaintenanceConfig` resource will be updated to include a +`tools_version` field. This field can be updated directly from the resource or +using `tctl`. + +``` +kind: cluster_maintenance_config +spec: + # tools_version is the version of client tools the proxy will advertise. Can + # be auto (match the version of the proxy), off (don't advertise any version), + # or an exact semver formatted version. + tools_version: auto | off | X.Y.Z + + [...] +``` + +``` +$ tctl autoupdate update --set-tools-version=1.0.1 +Automatic updates configuration has been updated. +``` + +Three modes will be supported: `auto`, `X.Y.Z` (exact version), and `off`. + +* `auto`: Cluster will advertise the version of the proxy as the version of + client tools. +* `X.Y.Z`: Cluster will advertise a specific version of client tools. +* `off`: Cluster will not advertise any version, clients will not try to + update. + +All new self-hosted clusters will be opted into `auto`. This way, by default +even self-hosted customers will have automatic updates available. + +Existing clusters will be opted into `off` to not introduce unexpected +behavior. + +The `tools_version` field will then be available from the unauthenticated +endpoint `/v1/webapi/ping`. + +``` +curl https://proxy.example.com/v1/webapi/ping | jq . +{ + "tools_version": "X.Y.Z", + + [...] +} +``` + +TODO(russjones): Teleport Connect support. + +### Costs + +Some additional costs will be incurred as Teleport downloads will increase in +frequency. + +### Out of scope + +How Cloud will push changes to `cluster_maintenance_config` is out of scope +for this RFD and will be handled by a separate Cloud specific RFD. + +### Security + +The initial version of automatic updates will rely on TLS to establish +connection authenticity to the Teleport download server. The authenticity of +assets served from the download server is out of scope for this RFD. Cluster +administrators concerned with the authenticity of assets served from the +download server can use self-managed updates with system package managers which +are signed. + +Phase 2 will use The Upgrade Framework (TUF) to implement secure updates. From f7bda091b5a1767e6699dfe23ecd1944d0418190 Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Mon, 25 Mar 2024 17:53:48 -0700 Subject: [PATCH 02/21] Fix. --- rfd/0144-client-tools-updates.md | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index e12a2545b2c76..2cb1afbc5d223 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -70,9 +70,13 @@ installed. The original binaries will not be overwritten by automatic updates, instead per-cluster binaries with permissions `0555` will be stored at -`~/.tsh/bin/proxyName/{tctl,tsh}`. This will act as both a cache and allow -users to connect to different Teleport clusters (running different versions of -Teleport) without juggling multiple versions of client tools. +`~/.tsh/bin/proxyName/{tctl,tsh}`. A locking mechanism built around +[syscall.Flock](https://pkg.go.dev/syscall#Flock) on Linux and macOS and +[LockFileEx](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex) +on Windows to only allow a single writer to update the version of client tools +at a time. This will act as both a cache and allow users to connect to +different Teleport clusters (running different versions of Teleport) without +juggling multiple versions of client tools. ``` $ tree ~/.tsh @@ -104,7 +108,7 @@ cluster and can not wait for the update to occur. ``` $ tsh login --proxy=proxy.example.com Client tools are out of date, updating to vX.Y.Z. -Update progress: [▒▒▒▒▒▒ ] (Ctrl-C to Cancel) +Update progress: [▒▒▒▒▒▒ ] (Ctrl-C to cancel update) [...] ``` @@ -114,9 +118,13 @@ child process will inherit all environment variables and flags. Re-execution will occur only if the parent process is not `tsh` or `tctl`, to prevent infinite loops. -An environment variable `TELEPORT_TOOLS_VERSION` will be introduced to enable -overriding and using a specific version of client tools. This field can be used -either as an emergency workaround for a known issue or for debugging purposes +An environment variable `TELEPORT_TOOLS_VERSION` will be introduced to use a +specific version of client tools `X.Y.Z` or turn `off` client tools updates +completely. This environment variable can be used as a emergency workaround for +a known issue, pinning to a specific version in CI/CD, or for debugging. + +Automatic updates will not be used if `tctl` is connecting to the Auth Service +over localhost. ##### Errors and warnings @@ -285,8 +293,6 @@ curl https://proxy.example.com/v1/webapi/ping | jq . } ``` -TODO(russjones): Teleport Connect support. - ### Costs Some additional costs will be incurred as Teleport downloads will increase in @@ -297,6 +303,11 @@ frequency. How Cloud will push changes to `cluster_maintenance_config` is out of scope for this RFD and will be handled by a separate Cloud specific RFD. +Automatic updates for Teleport Connect are out of scope for this RFD as it uses +a different install/update mechanism. For now it will call `tsh` with +`TELEPORT_TOOLS_VERSION=off` until automatic updates support can be added to +Connect. + ### Security The initial version of automatic updates will rely on TLS to establish From 3b8a132b9b855c16312b822149049a55f09a5bff Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Tue, 26 Mar 2024 09:23:03 -0700 Subject: [PATCH 03/21] Fix. --- rfd/0144-client-tools-updates.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 2cb1afbc5d223..9aca6ce239ef9 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -114,14 +114,14 @@ Update progress: [▒▒▒▒▒▒ ] (Ctrl-C to cancel update) ``` After downloading, client tools will re-execute the downloaded version. The -child process will inherit all environment variables and flags. Re-execution -will occur only if the parent process is not `tsh` or `tctl`, to prevent +child process will inherit all environment variables and flags. +`TELEPORT_TOOLS_VERSION=off` will be added during re-execution to prevent infinite loops. -An environment variable `TELEPORT_TOOLS_VERSION` will be introduced to use a -specific version of client tools `X.Y.Z` or turn `off` client tools updates -completely. This environment variable can be used as a emergency workaround for -a known issue, pinning to a specific version in CI/CD, or for debugging. +An environment variable `TELEPORT_TOOLS_VERSION` will be introduced that can be +`X.Y.Z` (use specific version) or `off` (do not try to update). This +environment variable can be used as a emergency workaround for a known issue, +pinning to a specific version in CI/CD, or for debugging. Automatic updates will not be used if `tctl` is connecting to the Auth Service over localhost. From f8ead2b407aef91b72494207716008e6badc042b Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Fri, 29 Mar 2024 17:51:07 -0700 Subject: [PATCH 04/21] Fix. --- rfd/0144-client-tools-updates.md | 189 +++++++++++++------------------ 1 file changed, 81 insertions(+), 108 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 9aca6ce239ef9..925c16a20d8ac 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -28,16 +28,16 @@ rules). ### Summary Client tools (like `tsh` and `tctl`) will automatically download and install -the version of client tools recommended by the Teleport cluster. +the version of client tools required by the Teleport cluster. -The `cluster_maintenance_config` resource will be updated to include a -`tools_version` field to control the version of client tools advertised by the -cluster. This resource will not be configurable on Teleport Cloud. It will be -managed by the Teleport Cloud team. +The `cluster_maintenance_config` resource will be updated to allow cluster +administrators to turn client tools automatic updates `on` (default) or `off`. +A `autoupdate_version` resource will be added to allow cluster administrators +to control the version of tools pushed to clients. -If cluster administrators want to opt-out of automatic updates and manage -updates themselves, they will be able to watch the `cluster_maintenance_config` -resource for changes and push out updates to endpoints manually. +Cloud customers will be able turn client tools automatic updates `off` and +self-manage updates. However, the version required by the cluster will continue +to be controlled by the Teleport Cloud team. Inspiration drawn from https://go.dev/doc/toolchain. @@ -47,42 +47,35 @@ Inspiration drawn from https://go.dev/doc/toolchain. ##### Automatic updates -The first time a user executes `tsh login` with a version of `tsh` that -supports automatic updates, the user will be prompted to enroll in automatic -updates. This choice will be saved in client tools configuration. - -``` -$ tsh --proxy=proxy.example.com login -Keep client tools like tsh and tctl automatically updated? [YES/no] - -[...] -``` - -``` -$ cat ~/.tsh/config/config.yaml -enable_autoupdate: true | false -``` - -Once enrolled, upon login client tools will check the `tools_version` -advertised by the cluster at `/v1/webapi/ping`. If this version differs from -that of the running binary, the recommended version will be downloaded and -installed. +Upon `tsh login`, client tools will check `/v1/webapi/ping` if automatic +updates are enabled and if the version required by the cluster differs from +that of the running binary. If the version differs, the required version will +be downloaded and used. The original binaries will not be overwritten by automatic updates, instead -per-cluster binaries with permissions `0555` will be stored at -`~/.tsh/bin/proxyName/{tctl,tsh}`. A locking mechanism built around -[syscall.Flock](https://pkg.go.dev/syscall#Flock) on Linux and macOS and +additional binaries will be downloaded and stored in `~/.tsh/bin` with +permissions `0555`. A `~/.tsh/version.yaml` file will be used to maintain a +mapping of logged in proxy to tools version. This file will be used to +auto-purge unused binaries. + +Locking mechanisms built around +[syscall.Flock](https://pkg.go.dev/syscall#Flock) (Linux and macOS) and [LockFileEx](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex) -on Windows to only allow a single writer to update the version of client tools -at a time. This will act as both a cache and allow users to connect to -different Teleport clusters (running different versions of Teleport) without -juggling multiple versions of client tools. +(Windows) will be used to allow multiple client tools to run/update at the same +time. + +A side effect of automatic updates will be users will be able to connect to +different Teleport clusters (running different versions of Teleport, including +pre-releases) without juggling multiple versions of client tools. ``` $ tree ~/.tsh ~/.tsh ├── bin -│ └── proxy.example.com +│ ├── 15.0.0 +│ │ ├── tctl +│ │ └── tsh +│ └── 15.1.1 │ ├── tctl │ └── tsh ├── current-profile @@ -97,7 +90,8 @@ $ tree ~/.tsh │ ├── foo-x509.pem │ └── foo.pub ├── known_hosts -└── proxy.example.com.yaml +├── proxy.example.com.yaml +└── version.yaml ``` Users can cancel client tools updates using `Ctrl-C`. This may be needed if the @@ -113,7 +107,7 @@ Update progress: [▒▒▒▒▒▒ ] (Ctrl-C to cancel update) [...] ``` -After downloading, client tools will re-execute the downloaded version. The +After downloading, client tools will re-execute the downloaded version. The child process will inherit all environment variables and flags. `TELEPORT_TOOLS_VERSION=off` will be added during re-execution to prevent infinite loops. @@ -128,10 +122,11 @@ over localhost. ##### Errors and warnings -If the user does not enroll in automatic updates and the cluster administrator -does not self-manage client tools updates as outlined in [Self-managed client -tools updates](#self-managed-client-tools-updates), a series of warnings and -errors with increasing urgency will be shown to the user. +If cluster administrator has chosen not to enroll client tools in automatic +updates and does not self-manage client tools updates as outlined in +[Self-managed client tools updates](#self-managed-client-tools-updates), a +series of warnings and errors with increasing urgency will be shown to the +user. If the version of client tools is within the same major version as advertised by the cluster, a warning will be shown to urge the user to enroll in automatic @@ -175,7 +170,7 @@ use the `--skip-version-check` flag. $ tsh login --proxy=proxy.example.com ERROR: Client tools are N major versions out of date, update to vX.Y.Z. -Some functionality will not work. Update Teleport to vX.Y.Z from +Your cluster requires {tsh,tctl} vX.Y.Z. Update Teleport from https://goteleport.com/download or your system package manager. Run "tsh autoupdate enroll" to enroll in automatic updates and keep client @@ -201,92 +196,70 @@ $ tctl autoupdate watch [...] ``` -Cluster administrators who prefer to use the Teleport API can create a watcher -on the `types.ClusterMaintenanceConfig` resource and trigger other integrations -on changes to the `tools_version` field. - -``` -// Create a watcher to monitor for changes to client tools version. -watch, err := p.TeleportClient.NewWatcher(ctx, types.Watch{ - Kinds: []types.WatchKind{ - types.WatchKind{Kind: types.ClusterMaintenanceConfig}, - }, -}) -if err != nil { - return trace.Wrap(err) -} -defer watch.Close() - -// Loop forever watching for client tools version changes. -for { - select { - case e := <-watch.Events(): - if e.Resource == nil { - continue - } - resource, ok := e.Resource.(types.ClusterMaintenanceConfig) - if !ok { - continue - } - - // Process the version change by notifying downstream tooling (like - // Jamf). - if err := handle(resource); err != nil { - log.Printf("Failed to handle version change: %v", err) - } - case <-watch.Done(): - fmt.Println("The watcher job is finished.") - return nil - } -} -``` - ##### Cluster configuration +The `cluster_maintenance_config` resource will be updated to allow cluster +administrators to turn client tools automatic updates `on` (default) or `off`. +A `autoupdate_version` resource will be added to allow cluster administrators +to control the version of tools pushed to clients. + > [!NOTE] -> This resource will not be configurable on Teleport Cloud. It will be managed -> by the Teleport Cloud team. +> While Cloud customers will be able to use `cluster_maintenance_config` to +> turn client tools automatic updates `off` to self-manage updates, they will +> not be able to control the version of client tools in `autoupdate_verson`. +> That will continue to be managed by the Teleport Cloud team. -The `types.ClusterMaintenanceConfig` resource will be updated to include a -`tools_version` field. This field can be updated directly from the resource or -using `tctl`. +Both resources can either be updated directly or by using `tctl` helper +functions. ``` kind: cluster_maintenance_config spec: - # tools_version is the version of client tools the proxy will advertise. Can - # be auto (match the version of the proxy), off (don't advertise any version), - # or an exact semver formatted version. - tools_version: auto | off | X.Y.Z + # tools_auto_update allows turning client tools updates on or off at the + # cluster level. Only turn client tools automatic updates off if self-managed + # updates are in place. + tools_auto_update: on|off [...] ``` - ``` -$ tctl autoupdate update --set-tools-version=1.0.1 +$ tctl autoupdate update --set-tools-auto-update=off Automatic updates configuration has been updated. ``` -Three modes will be supported: `auto`, `X.Y.Z` (exact version), and `off`. +All Cloud clusters will be opted into `tools_auto_update: on`. + +All new self-hosted clusters will be opted into `tools_auto_update: on`. +Existing self-hosted clusters will be opted into `off` to not introduce +unexpected behavior. -* `auto`: Cluster will advertise the version of the proxy as the version of - client tools. -* `X.Y.Z`: Cluster will advertise a specific version of client tools. -* `off`: Cluster will not advertise any version, clients will not try to - update. +``` +kind: autoupdate_version +spec: + # tools_version is the version of client tools the cluster will advertise. + # Can be auto (match the version of the proxy) or an exact semver formatted + # version. + tools_version: auto|X.Y.Z +``` +``` +$ tctl autoupdate update --set-tools-version=1.0.1 +Automatic updates configuration has been updated. +``` -All new self-hosted clusters will be opted into `auto`. This way, by default -even self-hosted customers will have automatic updates available. +For Cloud clusters, `tools_version` will always be `X.Y.Z`, with the version +being controlled by the Cloud team. -Existing clusters will be opted into `off` to not introduce unexpected -behavior. +For self-hosted clusters, the default will be `tools_version: auto` which will +match the tools version to the version of the proxy. This way, by default even +self-hosted customers will have automatic updates available. -The `tools_version` field will then be available from the unauthenticated -endpoint `/v1/webapi/ping`. +The above configuration will then be available from the unauthenticated +endpoint `/v1/webapi/ping` which clients will consult. ``` curl https://proxy.example.com/v1/webapi/ping | jq . { + "tools_auto_update": true, "tools_version": "X.Y.Z", [...] @@ -300,8 +273,8 @@ frequency. ### Out of scope -How Cloud will push changes to `cluster_maintenance_config` is out of scope -for this RFD and will be handled by a separate Cloud specific RFD. +How Cloud will push changes to `autoupdate_version` is out of scope for this +RFD and will be handled by a separate Cloud specific RFD. Automatic updates for Teleport Connect are out of scope for this RFD as it uses a different install/update mechanism. For now it will call `tsh` with From 17a01b9f2acd7d9f043935eabd3651add7a0c4ac Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Sat, 30 Mar 2024 07:25:43 -0700 Subject: [PATCH 05/21] Fix. --- rfd/0144-client-tools-updates.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 925c16a20d8ac..a31faa00d327e 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -54,7 +54,7 @@ be downloaded and used. The original binaries will not be overwritten by automatic updates, instead additional binaries will be downloaded and stored in `~/.tsh/bin` with -permissions `0555`. A `~/.tsh/version.yaml` file will be used to maintain a +permissions `0555`. A `~/.tsh/bin/version.yaml` file will be used to maintain a mapping of logged in proxy to tools version. This file will be used to auto-purge unused binaries. @@ -75,23 +75,23 @@ $ tree ~/.tsh │ ├── 15.0.0 │ │ ├── tctl │ │ └── tsh -│ └── 15.1.1 -│ ├── tctl -│ └── tsh +│ ├── 15.1.1 +│ │ ├── tctl +│ │ └── tsh +│ └── version.yaml ├── current-profile ├── keys │ └── proxy.example.com │ ├── cas │ │ └── example.com.pem │ ├── certs.pem -│ ├── foo -│ ├── foo-ssh +│ ├── rjones +│ ├── rjones-ssh │ │ └── example.com-cert.pub -│ ├── foo-x509.pem -│ └── foo.pub +│ ├── rjones-x509.pem +│ └── rjones.pub ├── known_hosts -├── proxy.example.com.yaml -└── version.yaml +└── proxy.example.com.yaml ``` Users can cancel client tools updates using `Ctrl-C`. This may be needed if the From 4f55019c99b6226c937295ac07369cb3fa823d88 Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Sun, 31 Mar 2024 08:38:40 -0700 Subject: [PATCH 06/21] Fix. --- rfd/0144-client-tools-updates.md | 62 ++++++++++++++++---------------- 1 file changed, 30 insertions(+), 32 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index a31faa00d327e..0e40c0997c56d 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -27,17 +27,12 @@ rules). ### Summary -Client tools (like `tsh` and `tctl`) will automatically download and install -the version of client tools required by the Teleport cluster. +Client tools like `tsh` and `tctl` will automatically download and install the +required version for the Teleport cluster. -The `cluster_maintenance_config` resource will be updated to allow cluster -administrators to turn client tools automatic updates `on` (default) or `off`. -A `autoupdate_version` resource will be added to allow cluster administrators -to control the version of tools pushed to clients. - -Cloud customers will be able turn client tools automatic updates `off` and -self-manage updates. However, the version required by the cluster will continue -to be controlled by the Teleport Cloud team. +Enrollment in automatic updates for client tools will be required at the +cluster level. Cluster administrators using MDM software like Jamf will be able +to manage updates independently. Inspiration drawn from https://go.dev/doc/toolchain. @@ -47,22 +42,21 @@ Inspiration drawn from https://go.dev/doc/toolchain. ##### Automatic updates -Upon `tsh login`, client tools will check `/v1/webapi/ping` if automatic -updates are enabled and if the version required by the cluster differs from -that of the running binary. If the version differs, the required version will -be downloaded and used. +When `tsh login` is executed, client tools will check `/v1/webapi/ping` to +determine if automatic updates are enabled. If the cluster's required version +differs from the current binary, client tools will download and re-execute +using the version required by the cluster. -The original binaries will not be overwritten by automatic updates, instead -additional binaries will be downloaded and stored in `~/.tsh/bin` with -permissions `0555`. A `~/.tsh/bin/version.yaml` file will be used to maintain a -mapping of logged in proxy to tools version. This file will be used to -auto-purge unused binaries. +The original client tools binaries won't be overwritten. Instead, additional +binaries will be downloaded and stored in `~/.tsh/bin` with `0555` permissions. +A file named `version.yaml` in the same directory will maintain the mapping +between proxies and tools versions, to support auto-purge of unused binaries. -Locking mechanisms built around -[syscall.Flock](https://pkg.go.dev/syscall#Flock) (Linux and macOS) and +To enable concurrent operation and updates of multiple client tools, locking +mechanisms utilizing [syscall.Flock](https://pkg.go.dev/syscall#Flock) (for +Linux and macOS) and [LockFileEx](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex) -(Windows) will be used to allow multiple client tools to run/update at the same -time. +(for Windows) will be employed. A side effect of automatic updates will be users will be able to connect to different Teleport clusters (running different versions of Teleport, including @@ -85,11 +79,11 @@ $ tree ~/.tsh │ ├── cas │ │ └── example.com.pem │ ├── certs.pem -│ ├── rjones -│ ├── rjones-ssh +│ ├── foo +│ ├── foo-ssh │ │ └── example.com-cert.pub -│ ├── rjones-x509.pem -│ └── rjones.pub +│ ├── foo-x509.pem +│ └── foo.pub ├── known_hosts └── proxy.example.com.yaml ``` @@ -198,14 +192,17 @@ $ tctl autoupdate watch ##### Cluster configuration +Enrollment of clients in automatic updates will be enforced at the cluster +level. + The `cluster_maintenance_config` resource will be updated to allow cluster administrators to turn client tools automatic updates `on` (default) or `off`. A `autoupdate_version` resource will be added to allow cluster administrators -to control the version of tools pushed to clients. +to manage the version of tools pushed to clients. > [!NOTE] > While Cloud customers will be able to use `cluster_maintenance_config` to -> turn client tools automatic updates `off` to self-manage updates, they will +> turn client tools automatic updates `off` and self-manage updates, they will > not be able to control the version of client tools in `autoupdate_verson`. > That will continue to be managed by the Teleport Cloud team. @@ -249,9 +246,10 @@ Automatic updates configuration has been updated. For Cloud clusters, `tools_version` will always be `X.Y.Z`, with the version being controlled by the Cloud team. -For self-hosted clusters, the default will be `tools_version: auto` which will -match the tools version to the version of the proxy. This way, by default even -self-hosted customers will have automatic updates available. +For self-hosted clusters, the default will be `tools_version: auto` matching +the tools version with the proxy version. This way, automatic updates will be +available to self-hosted customers by default, without any extra effort from +the cluster administrator. The above configuration will then be available from the unauthenticated endpoint `/v1/webapi/ping` which clients will consult. From c8c384539e24e0de366a9e46b2687b2b0108e0c4 Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Sun, 31 Mar 2024 08:42:05 -0700 Subject: [PATCH 07/21] Fix. --- rfd/0144-client-tools-updates.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 0e40c0997c56d..05174c82f725f 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -50,7 +50,8 @@ using the version required by the cluster. The original client tools binaries won't be overwritten. Instead, additional binaries will be downloaded and stored in `~/.tsh/bin` with `0555` permissions. A file named `version.yaml` in the same directory will maintain the mapping -between proxies and tools versions, to support auto-purge of unused binaries. +between proxies and tools versions, to support auto-purge (and free up disk +space) of unused binaries. To enable concurrent operation and updates of multiple client tools, locking mechanisms utilizing [syscall.Flock](https://pkg.go.dev/syscall#Flock) (for @@ -209,7 +210,7 @@ to manage the version of tools pushed to clients. Both resources can either be updated directly or by using `tctl` helper functions. -``` +```yaml kind: cluster_maintenance_config spec: # tools_auto_update allows turning client tools updates on or off at the @@ -230,7 +231,7 @@ All new self-hosted clusters will be opted into `tools_auto_update: on`. Existing self-hosted clusters will be opted into `off` to not introduce unexpected behavior. -``` +```yaml kind: autoupdate_version spec: # tools_version is the version of client tools the cluster will advertise. From 406f734c3360f67d2c6bc43de199eb832b323b3d Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Sun, 31 Mar 2024 08:43:27 -0700 Subject: [PATCH 08/21] Fix. --- rfd/0144-client-tools-updates.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 05174c82f725f..325eef1632df2 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -228,8 +228,8 @@ Automatic updates configuration has been updated. All Cloud clusters will be opted into `tools_auto_update: on`. All new self-hosted clusters will be opted into `tools_auto_update: on`. -Existing self-hosted clusters will be opted into `off` to not introduce -unexpected behavior. +Existing self-hosted clusters will be opted into `tools_auto_update: off` to +not introduce unexpected behavior. ```yaml kind: autoupdate_version @@ -245,7 +245,7 @@ Automatic updates configuration has been updated. ``` For Cloud clusters, `tools_version` will always be `X.Y.Z`, with the version -being controlled by the Cloud team. +controlled by the Cloud team. For self-hosted clusters, the default will be `tools_version: auto` matching the tools version with the proxy version. This way, automatic updates will be From 10b4b24bacc55beb7788790488c100b52ea90e25 Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Mon, 1 Apr 2024 09:22:09 -0700 Subject: [PATCH 09/21] Fix. --- rfd/0144-client-tools-updates.md | 41 +++++++++++++++++--------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 325eef1632df2..b8e77a888426f 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -13,7 +13,7 @@ state: draft ## What/Why -This RFD describes how client tools (like `tsh` and `tctl`) can be kept up to +This RFD describes how client tools like `tsh` and `tctl` can be kept up to date, either using automatic updates or self-managed updates. Keeping client tools updated helps with security (fixes for known security @@ -30,9 +30,10 @@ rules). Client tools like `tsh` and `tctl` will automatically download and install the required version for the Teleport cluster. -Enrollment in automatic updates for client tools will be required at the -cluster level. Cluster administrators using MDM software like Jamf will be able -to manage updates independently. +Enrollment in automatic updates for client tools will be controlled at the +cluster level. By default all clusters will be opted into automatic updates for +client tools. Cluster administrators using MDM software like JamF will be able +opt-out manually manage updates. Inspiration drawn from https://go.dev/doc/toolchain. @@ -50,14 +51,13 @@ using the version required by the cluster. The original client tools binaries won't be overwritten. Instead, additional binaries will be downloaded and stored in `~/.tsh/bin` with `0555` permissions. A file named `version.yaml` in the same directory will maintain the mapping -between proxies and tools versions, to support auto-purge (and free up disk -space) of unused binaries. +between proxies and tools versions, to support auto-purge and free up disk +space from unused binaries. -To enable concurrent operation and updates of multiple client tools, locking -mechanisms utilizing [syscall.Flock](https://pkg.go.dev/syscall#Flock) (for -Linux and macOS) and +To enable concurrent operation of client tools, a locking mechanisms utilizing +[syscall.Flock](https://pkg.go.dev/syscall#Flock) (for Linux and macOS) and [LockFileEx](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex) -(for Windows) will be employed. +(for Windows) will be used. A side effect of automatic updates will be users will be able to connect to different Teleport clusters (running different versions of Teleport, including @@ -102,15 +102,14 @@ Update progress: [▒▒▒▒▒▒ ] (Ctrl-C to cancel update) [...] ``` -After downloading, client tools will re-execute the downloaded version. The -child process will inherit all environment variables and flags. -`TELEPORT_TOOLS_VERSION=off` will be added during re-execution to prevent -infinite loops. - An environment variable `TELEPORT_TOOLS_VERSION` will be introduced that can be -`X.Y.Z` (use specific version) or `off` (do not try to update). This -environment variable can be used as a emergency workaround for a known issue, -pinning to a specific version in CI/CD, or for debugging. +`X.Y.Z` (use specific version) or `off` (do not update). This environment +variable can be used as a emergency workaround for a known issue, pinning to a +specific version in CI/CD, or for debugging. + +During re-execution, child process will inherit all environment variables and +flags. `TELEPORT_TOOLS_VERSION=off` will be added during re-execution to +prevent infinite loops. Automatic updates will not be used if `tctl` is connecting to the Auth Service over localhost. @@ -179,7 +178,7 @@ to this cluster. Cluster administrators that want to self-manage client tools updates will be able to watch for changes to client tools versions which can then be used to -trigger other integrations (using MDM software like Jamf) to update the +trigger other integrations (using MDM software like JamF) to update the installed version of client tools on endpoints. ``` @@ -202,6 +201,10 @@ A `autoupdate_version` resource will be added to allow cluster administrators to manage the version of tools pushed to clients. > [!NOTE] +> Client tools configuration is broken into two resources to [prevent +> updates](https://github.com/gravitational/teleport/blob/master/lib/modules/modules.go#L332-L355) +> to `autoupdate_verson` on Cloud. +> > While Cloud customers will be able to use `cluster_maintenance_config` to > turn client tools automatic updates `off` and self-manage updates, they will > not be able to control the version of client tools in `autoupdate_verson`. From f0a46e46b6d9c3fc815bce4ae71b0c17e155c33d Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Tue, 2 Apr 2024 17:05:08 -0700 Subject: [PATCH 10/21] Fix. --- rfd/0144-client-tools-updates.md | 37 +++++++++++++++++++------------- 1 file changed, 22 insertions(+), 15 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index b8e77a888426f..2982ba8102431 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -49,10 +49,10 @@ differs from the current binary, client tools will download and re-execute using the version required by the cluster. The original client tools binaries won't be overwritten. Instead, additional -binaries will be downloaded and stored in `~/.tsh/bin` with `0555` permissions. -A file named `version.yaml` in the same directory will maintain the mapping -between proxies and tools versions, to support auto-purge and free up disk -space from unused binaries. +binaries will be downloaded and stored in `~/.tsh/versions` with `0755` +permissions. A file named `versions.yaml` in the same directory will maintain +the mapping between proxies and tools versions, to support auto-purge and free +up disk space from unused binaries. To enable concurrent operation of client tools, a locking mechanisms utilizing [syscall.Flock](https://pkg.go.dev/syscall#Flock) (for Linux and macOS) and @@ -66,14 +66,6 @@ pre-releases) without juggling multiple versions of client tools. ``` $ tree ~/.tsh ~/.tsh -├── bin -│ ├── 15.0.0 -│ │ ├── tctl -│ │ └── tsh -│ ├── 15.1.1 -│ │ ├── tctl -│ │ └── tsh -│ └── version.yaml ├── current-profile ├── keys │ └── proxy.example.com @@ -86,7 +78,17 @@ $ tree ~/.tsh │ ├── foo-x509.pem │ └── foo.pub ├── known_hosts -└── proxy.example.com.yaml +├── proxy.example.com.yaml +└── versions + ├── 15.0.0 + │ └── bin + │ ├── tctl + │ └── tsh + ├── 15.1.1 + │ └── bin + │ ├── tctl + │ └── tsh + └── versions.yaml ``` Users can cancel client tools updates using `Ctrl-C`. This may be needed if the @@ -177,8 +179,8 @@ to this cluster. #### Self-managed client tools updates Cluster administrators that want to self-manage client tools updates will be -able to watch for changes to client tools versions which can then be used to -trigger other integrations (using MDM software like JamF) to update the +able to get and watch for changes to client tools versions which can then be +used to trigger other integrations (using MDM software like JamF) to update the installed version of client tools on endpoints. ``` @@ -190,6 +192,11 @@ $ tctl autoupdate watch [...] ``` +``` +$ tctl autoupdate get +{"tools_version": "2.0.0"} +``` + ##### Cluster configuration Enrollment of clients in automatic updates will be enforced at the cluster From 5c82438fda39ad9e7c5f091eb18bc8cb9c9cbd1f Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Tue, 2 Apr 2024 17:07:46 -0700 Subject: [PATCH 11/21] Fix. --- rfd/0144-client-tools-updates.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 2982ba8102431..0960ad3a0863c 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -43,10 +43,10 @@ Inspiration drawn from https://go.dev/doc/toolchain. ##### Automatic updates -When `tsh login` is executed, client tools will check `/v1/webapi/ping` to -determine if automatic updates are enabled. If the cluster's required version -differs from the current binary, client tools will download and re-execute -using the version required by the cluster. +When `tsh login` is executed, client tools will check +`/v1/webapi/automaticupdates` to determine if automatic updates are enabled. If +the cluster's required version differs from the current binary, client tools +will download and re-execute using the version required by the cluster. The original client tools binaries won't be overwritten. Instead, additional binaries will be downloaded and stored in `~/.tsh/versions` with `0755` @@ -110,7 +110,7 @@ variable can be used as a emergency workaround for a known issue, pinning to a specific version in CI/CD, or for debugging. During re-execution, child process will inherit all environment variables and -flags. `TELEPORT_TOOLS_VERSION=off` will be added during re-execution to +flags. `TELEPORT_TOOLS_VERSION=off` will be added during re-execution to prevent infinite loops. Automatic updates will not be used if `tctl` is connecting to the Auth Service @@ -263,10 +263,10 @@ available to self-hosted customers by default, without any extra effort from the cluster administrator. The above configuration will then be available from the unauthenticated -endpoint `/v1/webapi/ping` which clients will consult. +endpoint `/v1/webapi/automaticupdates` which clients will consult. ``` -curl https://proxy.example.com/v1/webapi/ping | jq . +curl https://proxy.example.com/v1/webapi/automaticupdates | jq . { "tools_auto_update": true, "tools_version": "X.Y.Z", From 4adb7cd7bd518df25ba647bf7e091ce96bcb6a84 Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Wed, 3 Apr 2024 07:58:10 -0700 Subject: [PATCH 12/21] Fix. --- rfd/0144-client-tools-updates.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 0960ad3a0863c..cb51e945df94a 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -43,10 +43,10 @@ Inspiration drawn from https://go.dev/doc/toolchain. ##### Automatic updates -When `tsh login` is executed, client tools will check -`/v1/webapi/automaticupdates` to determine if automatic updates are enabled. If -the cluster's required version differs from the current binary, client tools -will download and re-execute using the version required by the cluster. +When `tsh login` is executed, client tools will check `/v1/webapi/ping` to +determine if automatic updates are enabled. If the cluster's required version +differs from the current binary, client tools will download and re-execute +using the version required by the cluster. The original client tools binaries won't be overwritten. Instead, additional binaries will be downloaded and stored in `~/.tsh/versions` with `0755` @@ -136,8 +136,8 @@ Warning: Client tools are out of date, update to vX.Y.Z. Update Teleport to vX.Y.Z from https://goteleport.com/download or your system package manager. -Run "tsh autoupdate enroll" to enroll in automatic updates and keep client -tools like tsh and tctl automatically updated. +Enroll in automatic updates to keep client tools like tsh and tctl +automatically updated. https://goteleport.com/docs/upgrading/automatic-updates [...] ``` @@ -153,8 +153,8 @@ WARNING: Client tools are 1 major version out of date, update to vX.Y.Z. Some functionality may not work. Update Teleport to vX.Y.Z from https://goteleport.com/download or your system package manager. -Run "tsh autoupdate enroll" to enroll in automatic updates and keep client -tools like tsh and tctl automatically updated. +Enroll in automatic updates to keep client tools like tsh and tctl +automatically updated. https://goteleport.com/docs/upgrading/automatic-updates ``` If the version of client tools is 2 (or more) versions lower than the version @@ -169,8 +169,8 @@ ERROR: Client tools are N major versions out of date, update to vX.Y.Z. Your cluster requires {tsh,tctl} vX.Y.Z. Update Teleport from https://goteleport.com/download or your system package manager. -Run "tsh autoupdate enroll" to enroll in automatic updates and keep client -tools like tsh and tctl automatically updated. +Enroll in automatic updates to keep client tools like tsh and tctl +automatically updated. https://goteleport.com/docs/upgrading/automatic-updates Use the "--skip-version-check" flag to bypass this check and attempt to connect to this cluster. @@ -263,10 +263,10 @@ available to self-hosted customers by default, without any extra effort from the cluster administrator. The above configuration will then be available from the unauthenticated -endpoint `/v1/webapi/automaticupdates` which clients will consult. +endpoint `/v1/webapi/ping` which clients will consult. ``` -curl https://proxy.example.com/v1/webapi/automaticupdates | jq . +curl https://proxy.example.com/v1/webapi/ping | jq . { "tools_auto_update": true, "tools_version": "X.Y.Z", From 8f3106285eb5fdee1a155ab73f27a0afa506667c Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Wed, 3 Apr 2024 08:02:38 -0700 Subject: [PATCH 13/21] Fix. --- rfd/0144-client-tools-updates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index cb51e945df94a..35fe45b68556d 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -266,7 +266,7 @@ The above configuration will then be available from the unauthenticated endpoint `/v1/webapi/ping` which clients will consult. ``` -curl https://proxy.example.com/v1/webapi/ping | jq . +$ curl https://proxy.example.com/v1/webapi/ping | jq . { "tools_auto_update": true, "tools_version": "X.Y.Z", From e56db99f84d687e642c14ca5ebd01b0f6bddb355 Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Tue, 16 Apr 2024 17:43:00 -0700 Subject: [PATCH 14/21] Fix. --- rfd/0144-client-tools-updates.md | 90 +++++++++++++++----------------- 1 file changed, 42 insertions(+), 48 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 35fe45b68556d..7628210be0b31 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -31,9 +31,12 @@ Client tools like `tsh` and `tctl` will automatically download and install the required version for the Teleport cluster. Enrollment in automatic updates for client tools will be controlled at the -cluster level. By default all clusters will be opted into automatic updates for -client tools. Cluster administrators using MDM software like JamF will be able -opt-out manually manage updates. +cluster level. By default all Cloud clusters will be opted into automatic +updates for client tools. Cluster administrators using MDM software like Jamf +will be able opt-out manually manage updates. + +Self-hosted clusters will be be opted out, but have the option to use the same +automatic update mechanism. Inspiration drawn from https://go.dev/doc/toolchain. @@ -46,26 +49,24 @@ Inspiration drawn from https://go.dev/doc/toolchain. When `tsh login` is executed, client tools will check `/v1/webapi/ping` to determine if automatic updates are enabled. If the cluster's required version differs from the current binary, client tools will download and re-execute -using the version required by the cluster. - -The original client tools binaries won't be overwritten. Instead, additional -binaries will be downloaded and stored in `~/.tsh/versions` with `0755` -permissions. A file named `versions.yaml` in the same directory will maintain -the mapping between proxies and tools versions, to support auto-purge and free -up disk space from unused binaries. - -To enable concurrent operation of client tools, a locking mechanisms utilizing +using the version required by the cluster. All other `tsh` subcommands (like +`tsh ssh ...`) will always use the downloaded version. + +The original client tools binaries won't be overwritten. Instead, an additional +binary will be downloaded and stored in `~/.tsh/bin` with `0755` permissions. +A hash of the binary will be performed and checked against a known value to +ensure the file has not been corrupted during download. To enable concurrent +operation of client tools, a locking mechanisms utilizing [syscall.Flock](https://pkg.go.dev/syscall#Flock) (for Linux and macOS) and [LockFileEx](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex) (for Windows) will be used. -A side effect of automatic updates will be users will be able to connect to -different Teleport clusters (running different versions of Teleport, including -pre-releases) without juggling multiple versions of client tools. - ``` $ tree ~/.tsh ~/.tsh +├── bin +│ ├── tctl +│ └── tsh ├── current-profile ├── keys │ └── proxy.example.com @@ -78,17 +79,7 @@ $ tree ~/.tsh │ ├── foo-x509.pem │ └── foo.pub ├── known_hosts -├── proxy.example.com.yaml -└── versions - ├── 15.0.0 - │ └── bin - │ ├── tctl - │ └── tsh - ├── 15.1.1 - │ └── bin - │ ├── tctl - │ └── tsh - └── versions.yaml +└── proxy.example.com.yaml ``` Users can cancel client tools updates using `Ctrl-C`. This may be needed if the @@ -105,9 +96,9 @@ Update progress: [▒▒▒▒▒▒ ] (Ctrl-C to cancel update) ``` An environment variable `TELEPORT_TOOLS_VERSION` will be introduced that can be -`X.Y.Z` (use specific version) or `off` (do not update). This environment -variable can be used as a emergency workaround for a known issue, pinning to a -specific version in CI/CD, or for debugging. +`X.Y.Z` (use specific semver version) or `off` (do not update). This +environment variable can be used as a emergency workaround for a known issue, +pinning to a specific version in CI/CD, or for debugging. During re-execution, child process will inherit all environment variables and flags. `TELEPORT_TOOLS_VERSION=off` will be added during re-execution to @@ -116,6 +107,18 @@ prevent infinite loops. Automatic updates will not be used if `tctl` is connecting to the Auth Service over localhost. +> [!NOTE] +> If a user connects to multiple root clusters, each running a different +> version of Teleport, client tools will attempt to download the differing +> version of Teleport each time the user performs a `tsh login`. +> +> In practice, the number of users impacted by this would be small. Customer +> Cloud tenants would be on the same version and this feature is turned off by +> default for self-hosted cluster. +> +> However, for those people in this situation, the recommendation would be to +> use self-managed updates. + ##### Errors and warnings If cluster administrator has chosen not to enroll client tools in automatic @@ -180,7 +183,7 @@ to this cluster. Cluster administrators that want to self-manage client tools updates will be able to get and watch for changes to client tools versions which can then be -used to trigger other integrations (using MDM software like JamF) to update the +used to trigger other integrations (using MDM software like Jamf) to update the installed version of client tools on endpoints. ``` @@ -203,18 +206,18 @@ Enrollment of clients in automatic updates will be enforced at the cluster level. The `cluster_maintenance_config` resource will be updated to allow cluster -administrators to turn client tools automatic updates `on` (default) or `off`. +administrators to turn client tools automatic updates `on` or `off`. A `autoupdate_version` resource will be added to allow cluster administrators to manage the version of tools pushed to clients. > [!NOTE] > Client tools configuration is broken into two resources to [prevent > updates](https://github.com/gravitational/teleport/blob/master/lib/modules/modules.go#L332-L355) -> to `autoupdate_verson` on Cloud. +> to `autoupdate_version` on Cloud. > > While Cloud customers will be able to use `cluster_maintenance_config` to > turn client tools automatic updates `off` and self-manage updates, they will -> not be able to control the version of client tools in `autoupdate_verson`. +> not be able to control the version of client tools in `autoupdate_version`. > That will continue to be managed by the Teleport Cloud team. Both resources can either be updated directly or by using `tctl` helper @@ -235,19 +238,15 @@ $ tctl autoupdate update --set-tools-auto-update=off Automatic updates configuration has been updated. ``` -All Cloud clusters will be opted into `tools_auto_update: on`. - -All new self-hosted clusters will be opted into `tools_auto_update: on`. -Existing self-hosted clusters will be opted into `tools_auto_update: off` to -not introduce unexpected behavior. +By default, all Cloud clusters will be opted into `tools_auto_update: on`. All +self-hosted clusters will be opted into `tools_auto_update: off`. ```yaml kind: autoupdate_version spec: - # tools_version is the version of client tools the cluster will advertise. - # Can be auto (match the version of the proxy) or an exact semver formatted - # version. - tools_version: auto|X.Y.Z + # tools_version is the semver version of client tools the cluster will + # advertise. + tools_version: X.Y.Z ``` ``` $ tctl autoupdate update --set-tools-version=1.0.1 @@ -257,11 +256,6 @@ Automatic updates configuration has been updated. For Cloud clusters, `tools_version` will always be `X.Y.Z`, with the version controlled by the Cloud team. -For self-hosted clusters, the default will be `tools_version: auto` matching -the tools version with the proxy version. This way, automatic updates will be -available to self-hosted customers by default, without any extra effort from -the cluster administrator. - The above configuration will then be available from the unauthenticated endpoint `/v1/webapi/ping` which clients will consult. From 36d9da8b6f29274d7b815c898c18199b572fa048 Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Tue, 7 May 2024 11:35:20 -0700 Subject: [PATCH 15/21] Fix. --- rfd/0144-client-tools-updates.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 7628210be0b31..f37b5058c7f55 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -54,9 +54,12 @@ using the version required by the cluster. All other `tsh` subcommands (like The original client tools binaries won't be overwritten. Instead, an additional binary will be downloaded and stored in `~/.tsh/bin` with `0755` permissions. -A hash of the binary will be performed and checked against a known value to -ensure the file has not been corrupted during download. To enable concurrent -operation of client tools, a locking mechanisms utilizing + +To validate the binaries have not been corrupted during download, a hash of the +archive will be checked against the expected value. The expected hash value +comes from the archive download path with `.sha256` appended. + +To enable concurrent operation of client tools, a locking mechanisms utilizing [syscall.Flock](https://pkg.go.dev/syscall#Flock) (for Linux and macOS) and [LockFileEx](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex) (for Windows) will be used. From c5a76b23394c19dfe0f1bfe895c8e5ea27a30457 Mon Sep 17 00:00:00 2001 From: Russell Jones Date: Tue, 7 May 2024 11:39:30 -0700 Subject: [PATCH 16/21] Fix. --- rfd/0144-client-tools-updates.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index f37b5058c7f55..949c4c1f25d89 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -107,8 +107,10 @@ During re-execution, child process will inherit all environment variables and flags. `TELEPORT_TOOLS_VERSION=off` will be added during re-execution to prevent infinite loops. -Automatic updates will not be used if `tctl` is connecting to the Auth Service -over localhost. +When `tctl` is used to connect to Auth Service running on the same host over +`localhost`, `tctl` assumes a special administrator role that can perform all +operations on a cluster. In this situation the expectation is for the version +of `tctl` and `teleport` to match so automatic updates will not be used. > [!NOTE] > If a user connects to multiple root clusters, each running a different From dd62e80d00bdbc18e892bc82b119fb33bc821b60 Mon Sep 17 00:00:00 2001 From: Vadym Popov Date: Thu, 15 Aug 2024 23:50:14 -0400 Subject: [PATCH 17/21] Client tools RFD update (#45515) * Add changes proposed for client autoupdate * Add proxy flag and CDN info --- rfd/0144-client-tools-updates.md | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 949c4c1f25d89..8aa5ed5b819cc 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -46,7 +46,7 @@ Inspiration drawn from https://go.dev/doc/toolchain. ##### Automatic updates -When `tsh login` is executed, client tools will check `/v1/webapi/ping` to +When `tsh login` is executed, client tools will check `/v1/webapi/find` to determine if automatic updates are enabled. If the cluster's required version differs from the current binary, client tools will download and re-execute using the version required by the cluster. All other `tsh` subcommands (like @@ -98,6 +98,11 @@ Update progress: [▒▒▒▒▒▒ ] (Ctrl-C to cancel update) [...] ``` +All archive downloads are targeted to the `cdn.teleport.dev` endpoint and depend +on the operating system, platform, and edition. Where edition must be identified +by the original client tools binary, URL pattern: +`https://cdn.teleport.dev/teleport-{, ent-}v15.3.0-{linux, darwin, windows}-{amd64,arm64,arm,386}-{fips-}bin.tar.gz` + An environment variable `TELEPORT_TOOLS_VERSION` will be introduced that can be `X.Y.Z` (use specific semver version) or `off` (do not update). This environment variable can be used as a emergency workaround for a known issue, @@ -191,8 +196,10 @@ able to get and watch for changes to client tools versions which can then be used to trigger other integrations (using MDM software like Jamf) to update the installed version of client tools on endpoints. +By defining the `proxy` flag, we can use the watch command without logging in. + ``` -$ tctl autoupdate watch +$ tctl autoupdate watch --proxy proxy.example.com {"tools_version": "1.0.0"} {"tools_version": "1.0.1"} {"tools_version": "2.0.0"} @@ -201,7 +208,7 @@ $ tctl autoupdate watch ``` ``` -$ tctl autoupdate get +$ tctl autoupdate get --proxy proxy.example.com {"tools_version": "2.0.0"} ``` @@ -210,7 +217,7 @@ $ tctl autoupdate get Enrollment of clients in automatic updates will be enforced at the cluster level. -The `cluster_maintenance_config` resource will be updated to allow cluster +The `cluster_autoupdate_config` resource will be updated to allow cluster administrators to turn client tools automatic updates `on` or `off`. A `autoupdate_version` resource will be added to allow cluster administrators to manage the version of tools pushed to clients. @@ -220,7 +227,7 @@ to manage the version of tools pushed to clients. > updates](https://github.com/gravitational/teleport/blob/master/lib/modules/modules.go#L332-L355) > to `autoupdate_version` on Cloud. > -> While Cloud customers will be able to use `cluster_maintenance_config` to +> While Cloud customers will be able to use `cluster_autoupdate_config` to > turn client tools automatic updates `off` and self-manage updates, they will > not be able to control the version of client tools in `autoupdate_version`. > That will continue to be managed by the Teleport Cloud team. @@ -229,7 +236,7 @@ Both resources can either be updated directly or by using `tctl` helper functions. ```yaml -kind: cluster_maintenance_config +kind: cluster_autoupdate_config spec: # tools_auto_update allows turning client tools updates on or off at the # cluster level. Only turn client tools automatic updates off if self-managed @@ -262,14 +269,16 @@ For Cloud clusters, `tools_version` will always be `X.Y.Z`, with the version controlled by the Cloud team. The above configuration will then be available from the unauthenticated -endpoint `/v1/webapi/ping` which clients will consult. +proxy discovery endpoint `/v1/webapi/find` which clients will consult. +Resources that store information about autoupdate and tools version are cached on +the proxy side to minimize requests to the auth service. In case of an unhealthy +cache state, the last known version of the resources should be used for the response. ``` -$ curl https://proxy.example.com/v1/webapi/ping | jq . +$ curl https://proxy.example.com/v1/webapi/find | jq . { "tools_auto_update": true, "tools_version": "X.Y.Z", - [...] } ``` From e4f4499dadf9b3ec249a37ce0f746a1363af0947 Mon Sep 17 00:00:00 2001 From: Vadym Popov Date: Thu, 22 Aug 2024 15:05:04 -0400 Subject: [PATCH 18/21] Naming adjustments --- rfd/0144-client-tools-updates.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 8aa5ed5b819cc..4b553ca6b3a08 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -217,7 +217,7 @@ $ tctl autoupdate get --proxy proxy.example.com Enrollment of clients in automatic updates will be enforced at the cluster level. -The `cluster_autoupdate_config` resource will be updated to allow cluster +The `autoupdate_config` resource will be updated to allow cluster administrators to turn client tools automatic updates `on` or `off`. A `autoupdate_version` resource will be added to allow cluster administrators to manage the version of tools pushed to clients. @@ -227,7 +227,7 @@ to manage the version of tools pushed to clients. > updates](https://github.com/gravitational/teleport/blob/master/lib/modules/modules.go#L332-L355) > to `autoupdate_version` on Cloud. > -> While Cloud customers will be able to use `cluster_autoupdate_config` to +> While Cloud customers will be able to use `autoupdate_config` to > turn client tools automatic updates `off` and self-manage updates, they will > not be able to control the version of client tools in `autoupdate_version`. > That will continue to be managed by the Teleport Cloud team. @@ -236,12 +236,12 @@ Both resources can either be updated directly or by using `tctl` helper functions. ```yaml -kind: cluster_autoupdate_config +kind: autoupdate_config spec: - # tools_auto_update allows turning client tools updates on or off at the + # tools_autoupdate allows turning client tools updates on or off at the # cluster level. Only turn client tools automatic updates off if self-managed # updates are in place. - tools_auto_update: on|off + tools_autoupdate: on|off [...] ``` @@ -250,8 +250,8 @@ $ tctl autoupdate update --set-tools-auto-update=off Automatic updates configuration has been updated. ``` -By default, all Cloud clusters will be opted into `tools_auto_update: on`. All -self-hosted clusters will be opted into `tools_auto_update: off`. +By default, all Cloud clusters will be opted into `tools_autoupdate: on`. All +self-hosted clusters will be opted into `tools_autoupdate: off`. ```yaml kind: autoupdate_version @@ -277,7 +277,7 @@ cache state, the last known version of the resources should be used for the resp ``` $ curl https://proxy.example.com/v1/webapi/find | jq . { - "tools_auto_update": true, + "tools_autoupdate": true, "tools_version": "X.Y.Z", [...] } From 0cd214f99f09e121e5b96cc0da3d1a175cf4f58b Mon Sep 17 00:00:00 2001 From: Vadym Popov Date: Thu, 22 Aug 2024 15:21:00 -0400 Subject: [PATCH 19/21] Naming adjustments --- rfd/0144-client-tools-updates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 4b553ca6b3a08..942d4e35c00e9 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -246,7 +246,7 @@ spec: [...] ``` ``` -$ tctl autoupdate update --set-tools-auto-update=off +$ tctl autoupdate update --set-tools-autoupdate=off Automatic updates configuration has been updated. ``` From 4d6afa2fd98447d0b16b1313c9208ef501936778 Mon Sep 17 00:00:00 2001 From: Vadym Popov Date: Fri, 11 Oct 2024 12:57:07 -0400 Subject: [PATCH 20/21] Update client tools autoupdate RFD (#47498) --- rfd/0144-client-tools-updates.md | 44 +++++++++++++++++--------------- 1 file changed, 24 insertions(+), 20 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index 942d4e35c00e9..d15e7a83d6ad9 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -199,17 +199,17 @@ installed version of client tools on endpoints. By defining the `proxy` flag, we can use the watch command without logging in. ``` -$ tctl autoupdate watch --proxy proxy.example.com -{"tools_version": "1.0.0"} -{"tools_version": "1.0.1"} -{"tools_version": "2.0.0"} +$ tctl autoupdate client-tools watch --proxy proxy.example.com +{"target_version": "1.0.0"} +{"target_version": "1.0.1"} +{"target_version": "2.0.0"} [...] ``` ``` -$ tctl autoupdate get --proxy proxy.example.com -{"tools_version": "2.0.0"} +$ tctl autoupdate client-tools get --proxy proxy.example.com +{"target_version": "2.0.0"} ``` ##### Cluster configuration @@ -238,34 +238,36 @@ functions. ```yaml kind: autoupdate_config spec: - # tools_autoupdate allows turning client tools updates on or off at the - # cluster level. Only turn client tools automatic updates off if self-managed - # updates are in place. - tools_autoupdate: on|off + tools: + # tools mode allows to enable client tools updates or disable at the + # cluster level. Disable client tools automatic updates only if self-managed + # updates are in place. + mode: enabled|disabled [...] ``` ``` -$ tctl autoupdate update --set-tools-autoupdate=off +$ tctl autoupdate client-tools update --set-mode=disabled Automatic updates configuration has been updated. ``` -By default, all Cloud clusters will be opted into `tools_autoupdate: on`. All -self-hosted clusters will be opted into `tools_autoupdate: off`. +By default, all Cloud clusters will be opted into `tools.mode: enabled`. All +self-hosted clusters will be opted into `tools.mode: disabled`. ```yaml kind: autoupdate_version spec: - # tools_version is the semver version of client tools the cluster will - # advertise. - tools_version: X.Y.Z + tools: + # target_version is the semver version of client tools the cluster will + # advertise. + target_version: X.Y.Z ``` ``` -$ tctl autoupdate update --set-tools-version=1.0.1 +$ tctl autoupdate client-tools update --set-target-version=1.0.1 Automatic updates configuration has been updated. ``` -For Cloud clusters, `tools_version` will always be `X.Y.Z`, with the version +For Cloud clusters, `target_version` will always be `X.Y.Z`, with the version controlled by the Cloud team. The above configuration will then be available from the unauthenticated @@ -277,8 +279,10 @@ cache state, the last known version of the resources should be used for the resp ``` $ curl https://proxy.example.com/v1/webapi/find | jq . { - "tools_autoupdate": true, - "tools_version": "X.Y.Z", + "auto_update": { + "tools_mode": enabled, + "tools_version": "X.Y.Z", + } [...] } ``` From c906170e1dc2c7f32a4902c4ba81db2e29efb654 Mon Sep 17 00:00:00 2001 From: Vadym Popov Date: Fri, 1 Nov 2024 15:23:05 -0400 Subject: [PATCH 21/21] Drop watch command for autoupdate --- rfd/0144-client-tools-updates.md | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/rfd/0144-client-tools-updates.md b/rfd/0144-client-tools-updates.md index d15e7a83d6ad9..c38960427a747 100644 --- a/rfd/0144-client-tools-updates.md +++ b/rfd/0144-client-tools-updates.md @@ -192,20 +192,11 @@ to this cluster. #### Self-managed client tools updates Cluster administrators that want to self-manage client tools updates will be -able to get and watch for changes to client tools versions which can then be +able to get changes to client tools versions which can then be used to trigger other integrations (using MDM software like Jamf) to update the installed version of client tools on endpoints. -By defining the `proxy` flag, we can use the watch command without logging in. - -``` -$ tctl autoupdate client-tools watch --proxy proxy.example.com -{"target_version": "1.0.0"} -{"target_version": "1.0.1"} -{"target_version": "2.0.0"} - -[...] -``` +By defining the `proxy` flag, we can use the get command without logging in. ``` $ tctl autoupdate client-tools get --proxy proxy.example.com @@ -247,7 +238,7 @@ spec: [...] ``` ``` -$ tctl autoupdate client-tools update --set-mode=disabled +$ tctl autoupdate client-tools configure --set-mode=disabled Automatic updates configuration has been updated. ``` @@ -263,7 +254,7 @@ spec: target_version: X.Y.Z ``` ``` -$ tctl autoupdate client-tools update --set-target-version=1.0.1 +$ tctl autoupdate client-tools configure --set-target-version=1.0.1 Automatic updates configuration has been updated. ```