Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Credentials changes for v2.0.0 #1052

Merged
merged 11 commits into from
Dec 21, 2020
Merged

Credentials changes for v2.0.0 #1052

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
3be7d12
Remove load_config_file and support for KUBECONFIG environment variable
jrhouston Nov 2, 2020
5a16a65
Change config_path -> config_paths
jrhouston Nov 13, 2020
f82e5f2
Support config_path and config_paths
jrhouston Nov 18, 2020
70c644e
Use system path separator
jrhouston Nov 20, 2020
d7516bc
Update website/docs/index.html.markdown
jrhouston Nov 20, 2020
04fdfac
Add ConflictsWith for config_path
jrhouston Nov 23, 2020
b0bfaf8
Generate an error when the user does not supply any configuration
jrhouston Nov 25, 2020
975f492
Add better error message when provider is not configured
jrhouston Nov 27, 2020
75377d4
Add check for api token path
jrhouston Dec 18, 2020
b9bcdd2
Defer checking valid configuration to avoid progressive apply issue
jrhouston Dec 21, 2020
f16553e
Fix crash in kubernetes_job waiter
jrhouston Dec 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 64 additions & 46 deletions kubernetes/provider.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,12 @@ import (
"bytes"
"context"
"fmt"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"log"
"net/http"
"os"
"path/filepath"

"github.com/hashicorp/terraform-plugin-sdk/v2/diag"

"github.com/hashicorp/terraform-plugin-sdk/v2/helper/logging"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
Expand Down Expand Up @@ -67,16 +70,17 @@ func Provider() *schema.Provider {
DefaultFunc: schema.EnvDefaultFunc("KUBE_CLUSTER_CA_CERT_DATA", ""),
Description: "PEM-encoded root certificates bundle for TLS authentication.",
},
"config_paths": {
jrhouston marked this conversation as resolved.
Show resolved Hide resolved
Type: schema.TypeList,
Elem: &schema.Schema{Type: schema.TypeString},
Optional: true,
Description: "A list of paths to kube config files. Can be set with KUBE_CONFIG_PATHS environment variable.",
},
"config_path": {
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.MultiEnvDefaultFunc(
[]string{
"KUBE_CONFIG",
"KUBECONFIG",
},
"~/.kube/config"),
Description: "Path to the kube config file, defaults to ~/.kube/config",
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("KUBE_CONFIG_PATH", ""),
Description: "Path to the kube config file. Can be set with KUBE_CONFIG_PATH.",
Copy link
Contributor

@dak1n1 dak1n1 Nov 20, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Could we also add ConflictsWith = []string{"config_paths"}? And the equivalent for config_path, so that the SDK lets a user know if they've specified both? Otherwise we're selecting one of the two at random, and the user might find it unexpected. In my testing, it will yield this result:

$ terraform validate

Error: ConflictsWith

"config_paths": conflicts with config_path


Error: ConflictsWith

"config_path": conflicts with config_paths

However, if specified as environment variables, it doesn't give the user any error message:

$ KUBE_CONFIG_PATH=$KUBECONFIG KUBE_CONFIG_PATHS=$KUBECONFIG terraform init
$ KUBE_CONFIG_PATH=$KUBECONFIG KUBE_CONFIG_PATHS=$KUBECONFIG terraform validate
Success! The configuration is valid.

$ KUBE_CONFIG_PATH=$KUBECONFIG KUBE_CONFIG_PATHS=$KUBECONFIG terraform apply --auto-approve
kubernetes_pod.main: Creating...
kubernetes_pod.main: Creation complete after 6s [id=default/test-pod]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

That validation difference between the environment variables and the equivalent in HCL might be an SDK bug.

EDIT: I see, this is the same bug you linked in this PR. It affects setting the default, which affects ConflictsWith.

},
"config_context": {
Type: schema.TypeString,
Expand All @@ -101,12 +105,6 @@ func Provider() *schema.Provider {
DefaultFunc: schema.EnvDefaultFunc("KUBE_TOKEN", ""),
Description: "Token to authenticate an service account",
},
"load_config_file": {
Type: schema.TypeBool,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("KUBE_LOAD_CONFIG_FILE", true),
Description: "Load local kubeconfig.",
},
"exec": {
Type: schema.TypeList,
Optional: true,
Expand Down Expand Up @@ -270,40 +268,60 @@ func initializeConfiguration(d *schema.ResourceData) (*restclient.Config, error)
overrides := &clientcmd.ConfigOverrides{}
loader := &clientcmd.ClientConfigLoadingRules{}

if d.Get("load_config_file").(bool) {
log.Printf("[DEBUG] Trying to load configuration from file")
if configPath, ok := d.GetOk("config_path"); ok && configPath.(string) != "" {
path, err := homedir.Expand(configPath.(string))
configPaths := []string{}

if v, ok := d.Get("config_path").(string); ok && v != "" {
configPaths = []string{v}
} else if v, ok := d.Get("config_paths").([]interface{}); ok && len(v) > 0 {
for _, p := range v {
configPaths = append(configPaths, p.(string))
}
} else if v := os.Getenv("KUBE_CONFIG_PATHS"); v != "" {
// NOTE we have to do this here because the schema
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an open issue for this hashicorp/terraform-plugin-sdk#142

// does not yet allow you to set a default for a TypeList
configPaths = filepath.SplitList(v)
}

if len(configPaths) > 0 {
expandedPaths := []string{}
for _, p := range configPaths {
path, err := homedir.Expand(p)
if err != nil {
return nil, err
}
log.Printf("[DEBUG] Configuration file is: %s", path)
loader.ExplicitPath = path

ctxSuffix := "; default context"

kubectx, ctxOk := d.GetOk("config_context")
authInfo, authInfoOk := d.GetOk("config_context_auth_info")
cluster, clusterOk := d.GetOk("config_context_cluster")
if ctxOk || authInfoOk || clusterOk {
ctxSuffix = "; overriden context"
if ctxOk {
overrides.CurrentContext = kubectx.(string)
ctxSuffix += fmt.Sprintf("; config ctx: %s", overrides.CurrentContext)
log.Printf("[DEBUG] Using custom current context: %q", overrides.CurrentContext)
}

overrides.Context = clientcmdapi.Context{}
if authInfoOk {
overrides.Context.AuthInfo = authInfo.(string)
ctxSuffix += fmt.Sprintf("; auth_info: %s", overrides.Context.AuthInfo)
}
if clusterOk {
overrides.Context.Cluster = cluster.(string)
ctxSuffix += fmt.Sprintf("; cluster: %s", overrides.Context.Cluster)
}
log.Printf("[DEBUG] Using overidden context: %#v", overrides.Context)
log.Printf("[DEBUG] Using kubeconfig: %s", path)
expandedPaths = append(expandedPaths, path)
}

if len(expandedPaths) == 1 {
loader.ExplicitPath = expandedPaths[0]
} else {
loader.Precedence = expandedPaths
}

ctxSuffix := "; default context"

kubectx, ctxOk := d.GetOk("config_context")
authInfo, authInfoOk := d.GetOk("config_context_auth_info")
cluster, clusterOk := d.GetOk("config_context_cluster")
if ctxOk || authInfoOk || clusterOk {
ctxSuffix = "; overriden context"
if ctxOk {
overrides.CurrentContext = kubectx.(string)
ctxSuffix += fmt.Sprintf("; config ctx: %s", overrides.CurrentContext)
log.Printf("[DEBUG] Using custom current context: %q", overrides.CurrentContext)
}

overrides.Context = clientcmdapi.Context{}
if authInfoOk {
overrides.Context.AuthInfo = authInfo.(string)
ctxSuffix += fmt.Sprintf("; auth_info: %s", overrides.Context.AuthInfo)
}
if clusterOk {
overrides.Context.Cluster = cluster.(string)
ctxSuffix += fmt.Sprintf("; cluster: %s", overrides.Context.Cluster)
}
log.Printf("[DEBUG] Using overidden context: %#v", overrides.Context)
}
}

Expand Down
128 changes: 35 additions & 93 deletions kubernetes/provider_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ import (
"context"
"errors"
"fmt"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
"os"
"path/filepath"
"strings"
"testing"

"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"

gversion "github.com/hashicorp/go-version"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-plugin-sdk/v2/terraform"
Expand Down Expand Up @@ -72,7 +74,7 @@ func TestProvider_configure(t *testing.T) {
resetEnv := unsetEnv(t)
defer resetEnv()

os.Setenv("KUBECONFIG", "test-fixtures/kube-config.yaml")
os.Setenv("KUBE_CONFIG_PATH", "test-fixtures/kube-config.yaml")
os.Setenv("KUBE_CTX", "gcp")

rc := terraform.NewResourceConfigRaw(map[string]interface{}{})
Expand All @@ -86,91 +88,33 @@ func TestProvider_configure(t *testing.T) {
func unsetEnv(t *testing.T) func() {
e := getEnv()

if err := os.Unsetenv("KUBECONFIG"); err != nil {
t.Fatalf("Error unsetting env var KUBECONFIG: %s", err)
}
if err := os.Unsetenv("KUBE_CONFIG"); err != nil {
t.Fatalf("Error unsetting env var KUBE_CONFIG: %s", err)
}
if err := os.Unsetenv("KUBE_CTX"); err != nil {
t.Fatalf("Error unsetting env var KUBE_CTX: %s", err)
}
if err := os.Unsetenv("KUBE_CTX_AUTH_INFO"); err != nil {
t.Fatalf("Error unsetting env var KUBE_CTX_AUTH_INFO: %s", err)
}
if err := os.Unsetenv("KUBE_CTX_CLUSTER"); err != nil {
t.Fatalf("Error unsetting env var KUBE_CTX_CLUSTER: %s", err)
}
if err := os.Unsetenv("KUBE_HOST"); err != nil {
t.Fatalf("Error unsetting env var KUBE_HOST: %s", err)
}
if err := os.Unsetenv("KUBE_USER"); err != nil {
t.Fatalf("Error unsetting env var KUBE_USER: %s", err)
}
if err := os.Unsetenv("KUBE_PASSWORD"); err != nil {
t.Fatalf("Error unsetting env var KUBE_PASSWORD: %s", err)
}
if err := os.Unsetenv("KUBE_CLIENT_CERT_DATA"); err != nil {
t.Fatalf("Error unsetting env var KUBE_CLIENT_CERT_DATA: %s", err)
}
if err := os.Unsetenv("KUBE_CLIENT_KEY_DATA"); err != nil {
t.Fatalf("Error unsetting env var KUBE_CLIENT_KEY_DATA: %s", err)
}
if err := os.Unsetenv("KUBE_CLUSTER_CA_CERT_DATA"); err != nil {
t.Fatalf("Error unsetting env var KUBE_CLUSTER_CA_CERT_DATA: %s", err)
}
if err := os.Unsetenv("KUBE_INSECURE"); err != nil {
t.Fatalf("Error unsetting env var KUBE_INSECURE: %s", err)
}
if err := os.Unsetenv("KUBE_LOAD_CONFIG_FILE"); err != nil {
t.Fatalf("Error unsetting env var KUBE_LOAD_CONFIG_FILE: %s", err)
}
if err := os.Unsetenv("KUBE_TOKEN"); err != nil {
t.Fatalf("Error unsetting env var KUBE_TOKEN: %s", err)
envVars := map[string]string{
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mopped this up because it was tedious. 🧹

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I actually did the same thing here, but I removed the whole section. I found it wasn't being used, or at least, it had no visible effect when I removed it. That's ok though, I'll remove this from my PR and test again once yours merges.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be used in this one test that doesn't actually run as part of the acceptance tests: https://github.com/hashicorp/terraform-provider-kubernetes/blob/master/kubernetes/provider_test.go#L70

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I saw that it didn't run during acceptance tests and that nothing was calling it... but I think I understand its purpose now. Thanks for showing me. I was misinterpreting it as a "TestProvider" configure function. But it actually tests the "provider configure" code, which does serve a purpose. So that's worth keeping.

"KUBE_CONFIG_PATH": e.ConfigPath,
"KUBE_CONFIG_PATHS": filepath.Join(e.ConfigPaths...),
"KUBE_CTX": e.Ctx,
"KUBE_CTX_AUTH_INFO": e.CtxAuthInfo,
"KUBE_CTX_CLUSTER": e.CtxCluster,
"KUBE_HOST": e.Host,
"KUBE_USER": e.User,
"KUBE_PASSWORD": e.Password,
"KUBE_CLIENT_CERT_DATA": e.ClientCertData,
"KUBE_CLIENT_KEY_DATA": e.ClientKeyData,
"KUBE_CLUSTER_CA_CERT_DATA": e.ClusterCACertData,
"KUBE_INSECURE": e.Insecure,
"KUBE_TOKEN": e.Token,
}

for k, _ := range envVars {
if err := os.Unsetenv(k); err != nil {
t.Fatalf("Error unsetting env var %s: %s", k, err)
}
}

return func() {
if err := os.Setenv("KUBE_CONFIG", e.Config); err != nil {
t.Fatalf("Error resetting env var KUBE_CONFIG: %s", err)
}
if err := os.Setenv("KUBECONFIG", e.Config); err != nil {
t.Fatalf("Error resetting env var KUBECONFIG: %s", err)
}
if err := os.Setenv("KUBE_CTX", e.Ctx); err != nil {
t.Fatalf("Error resetting env var KUBE_CTX: %s", err)
}
if err := os.Setenv("KUBE_CTX_AUTH_INFO", e.CtxAuthInfo); err != nil {
t.Fatalf("Error resetting env var KUBE_CTX_AUTH_INFO: %s", err)
}
if err := os.Setenv("KUBE_CTX_CLUSTER", e.CtxCluster); err != nil {
t.Fatalf("Error resetting env var KUBE_CTX_CLUSTER: %s", err)
}
if err := os.Setenv("KUBE_HOST", e.Host); err != nil {
t.Fatalf("Error resetting env var KUBE_HOST: %s", err)
}
if err := os.Setenv("KUBE_USER", e.User); err != nil {
t.Fatalf("Error resetting env var KUBE_USER: %s", err)
}
if err := os.Setenv("KUBE_PASSWORD", e.Password); err != nil {
t.Fatalf("Error resetting env var KUBE_PASSWORD: %s", err)
}
if err := os.Setenv("KUBE_CLIENT_CERT_DATA", e.ClientCertData); err != nil {
t.Fatalf("Error resetting env var KUBE_CLIENT_CERT_DATA: %s", err)
}
if err := os.Setenv("KUBE_CLIENT_KEY_DATA", e.ClientKeyData); err != nil {
t.Fatalf("Error resetting env var KUBE_CLIENT_KEY_DATA: %s", err)
}
if err := os.Setenv("KUBE_CLUSTER_CA_CERT_DATA", e.ClusterCACertData); err != nil {
t.Fatalf("Error resetting env var KUBE_CLUSTER_CA_CERT_DATA: %s", err)
}
if err := os.Setenv("KUBE_INSECURE", e.Insecure); err != nil {
t.Fatalf("Error resetting env var KUBE_INSECURE: %s", err)
}
if err := os.Setenv("KUBE_LOAD_CONFIG_FILE", e.LoadConfigFile); err != nil {
t.Fatalf("Error resetting env var KUBE_LOAD_CONFIG_FILE: %s", err)
}
if err := os.Setenv("KUBE_TOKEN", e.Token); err != nil {
t.Fatalf("Error resetting env var KUBE_TOKEN: %s", err)
for k, v := range envVars {
if err := os.Setenv(k, v); err != nil {
t.Fatalf("Error resetting env var %s: %s", k, err)
}
}
}
}
Expand All @@ -187,14 +131,13 @@ func getEnv() *currentEnv {
ClientKeyData: os.Getenv("KUBE_CLIENT_KEY_DATA"),
ClusterCACertData: os.Getenv("KUBE_CLUSTER_CA_CERT_DATA"),
Insecure: os.Getenv("KUBE_INSECURE"),
LoadConfigFile: os.Getenv("KUBE_LOAD_CONFIG_FILE"),
Token: os.Getenv("KUBE_TOKEN"),
}
if cfg := os.Getenv("KUBE_CONFIG"); cfg != "" {
e.Config = cfg
if v := os.Getenv("KUBE_CONFIG_PATH"); v != "" {
e.ConfigPath = v
}
if cfg := os.Getenv("KUBECONFIG"); cfg != "" {
e.Config = cfg
if v := os.Getenv("KUBE_CONFIG_PATH"); v != "" {
e.ConfigPaths = filepath.SplitList(v)
}
return e
}
Expand All @@ -203,8 +146,7 @@ func testAccPreCheck(t *testing.T) {
ctx := context.TODO()
hasFileCfg := (os.Getenv("KUBE_CTX_AUTH_INFO") != "" && os.Getenv("KUBE_CTX_CLUSTER") != "") ||
os.Getenv("KUBE_CTX") != "" ||
os.Getenv("KUBECONFIG") != "" ||
os.Getenv("KUBE_CONFIG") != ""
os.Getenv("KUBE_CONFIG_PATH") != ""
hasUserCredentials := os.Getenv("KUBE_USER") != "" && os.Getenv("KUBE_PASSWORD") != ""
hasClientCert := os.Getenv("KUBE_CLIENT_CERT_DATA") != "" && os.Getenv("KUBE_CLIENT_KEY_DATA") != ""
hasStaticCfg := (os.Getenv("KUBE_HOST") != "" &&
Expand Down Expand Up @@ -440,7 +382,8 @@ func clusterVersionLessThan(vs string) bool {
}

type currentEnv struct {
Config string
ConfigPath string
ConfigPaths []string
Ctx string
CtxAuthInfo string
CtxCluster string
Expand All @@ -451,7 +394,6 @@ type currentEnv struct {
ClientKeyData string
ClusterCACertData string
Insecure string
LoadConfigFile string
Token string
}

Expand Down
15 changes: 3 additions & 12 deletions website/docs/index.html.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Use the navigation to the left to read about the available resources.

```hcl
provider "kubernetes" {
config_path = "~/.kube/config"
config_context = "my-context"
}

Expand Down Expand Up @@ -78,12 +79,6 @@ the provider will try to use the service account token from the `/var/run/secret
Detection of in-cluster execution is based on the sole availability both of the `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT` environment variables,
with non empty values.

```hcl
provider "kubernetes" {
load_config_file = "false"
}
```

If you have any other static configuration setting specified in a config file or static configuration, in-cluster service account token will not be tried.

### Statically defined credentials
Expand All @@ -92,8 +87,6 @@ Another way is **statically** define TLS certificate credentials:

```hcl
provider "kubernetes" {
load_config_file = "false"

host = "https://104.196.242.174"

client_certificate = "${file("~/.kube/client-cert.pem")}"
Expand All @@ -106,8 +99,6 @@ or username and password (HTTP Basic Authorization):

```hcl
provider "kubernetes" {
load_config_file = "false"

host = "https://104.196.242.174"

username = "username"
Expand All @@ -131,12 +122,12 @@ The following arguments are supported:
* `client_certificate` - (Optional) PEM-encoded client certificate for TLS authentication. Can be sourced from `KUBE_CLIENT_CERT_DATA`.
* `client_key` - (Optional) PEM-encoded client certificate key for TLS authentication. Can be sourced from `KUBE_CLIENT_KEY_DATA`.
* `cluster_ca_certificate` - (Optional) PEM-encoded root certificates bundle for TLS authentication. Can be sourced from `KUBE_CLUSTER_CA_CERT_DATA`.
* `config_path` - (Optional) Path to the kube config file. Can be sourced from `KUBE_CONFIG` or `KUBECONFIG`. Defaults to `~/.kube/config`.
* `config_path` - (Optional) A path to a kube config files. Can be sourced from `KUBE_CONFIG_PATH`.
jrhouston marked this conversation as resolved.
Show resolved Hide resolved
* `config_paths` - (Optional) A list of paths to the kube config files. Can be sourced from `KUBE_CONFIG_PATHS`.
* `config_context` - (Optional) Context to choose from the config file. Can be sourced from `KUBE_CTX`.
* `config_context_auth_info` - (Optional) Authentication info context of the kube config (name of the kubeconfig user, `--user` flag in `kubectl`). Can be sourced from `KUBE_CTX_AUTH_INFO`.
* `config_context_cluster` - (Optional) Cluster context of the kube config (name of the kubeconfig cluster, `--cluster` flag in `kubectl`). Can be sourced from `KUBE_CTX_CLUSTER`.
* `token` - (Optional) Token of your service account. Can be sourced from `KUBE_TOKEN`.
* `load_config_file` - (Optional) By default the local config (~/.kube/config) is loaded when you use this provider. This option at false disables this behaviour which is desired when statically specifying the configuration or relying on in-cluster config. Can be sourced from `KUBE_LOAD_CONFIG_FILE`.
* `exec` - (Optional) Configuration block to use an [exec-based credential plugin] (https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins), e.g. call an external command to receive user credentials.
* `api_version` - (Required) API version to use when decoding the ExecCredentials resource, e.g. `client.authentication.k8s.io/v1beta1`.
* `command` - (Required) Command to execute.
Expand Down