Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubectl does not support secret creation from .env files with multiline values #1610

Open
plusiv opened this issue Jun 8, 2024 · 19 comments · May be fixed by kubernetes/kubernetes#125430
Open
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature. sig/cli Categorizes an issue or PR as relevant to SIG CLI. triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@plusiv
Copy link

plusiv commented Jun 8, 2024

What happened?

kubectl currently does not support the creation of Kubernetes secrets from .env files containing multiline values. This limitation poses a challenge for users who need to store multiline environment variables as secrets, such as certificates or private keys.

Assuming the following .env file:

SECRET_ONE_LINE=Value one line

PASSPORT_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAx+4sI6wK3b7q2C4RjN02pH/sy9vwZ9Xbb1hjQmQY/V2aG5QQ
... (rest of the private key) ...
-----END RSA PRIVATE KEY-----"

PASSPORT_PUBLIC_KEY="-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAx+4sI6wK3b7q2C4RjN02
... (rest of the public key) ...
-----END PUBLIC KEY-----"

And running kubectl create secret generic test-secret --from-env-file=.env the following output is given by kubectl:

error: "MIIEpAIBAAKCAQEAx+4sI6wK3b7q2C4RjN02pH/sy9vwZ9Xbb1hjQmQY/V2aG5QQ" is not a valid key name: a valid environment variable name must consist of alphabetic characters, digits, '_', '-', or '.', and must not start with a digit (e.g. 'my.env-name',  or 'MY_ENV.NAME',  or 'MyEnvName1', regex used for validation is '[-._a-zA-Z][-._a-zA-Z0-9]*')

What did you expect to happen?

The secret should be created successfully with the multiline value stored.

How can we reproduce it (as minimally and precisely as possible)?

  1. Create a .env file with multiline values:
SECRET=Value
MULTILINE_SECRET="line1
line2
line3"
  1. Create a Generic Secret:
kubectl create secret generic test-secret --from-env-file=.env

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.9+rke2r1

Cloud provider

OS version

# On MacOs:
$ uname -a
Darwin 200.225.2.32-clientes-izzi.mx 23.4.0 Darwin Kernel Version 23.4.0: Wed Feb 21 21:44:06 PST 2024; root:xnu-10063.101.15~2/RELEASE_ARM64_T8103 arm64

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

@plusiv plusiv added the kind/bug Categorizes issue or PR as related to a bug. label Jun 8, 2024
@k8s-ci-robot k8s-ci-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Jun 8, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Jun 8, 2024
@plusiv
Copy link
Author

plusiv commented Jun 8, 2024

/sig cli

@k8s-ci-robot k8s-ci-robot added sig/cli Categorizes an issue or PR as relevant to SIG CLI. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 8, 2024
@mauri870
Copy link
Member

mauri870 commented Jun 9, 2024

Currently there is no support for parsing multi-line entries. We parse the .env file line by line.

I think you can get this to work by simply replacing the actual line breaks in the file with a \n, it should work when encased in quotation marks.

E.g:

PRIVATE_KEY="my\nprivate\nkey"

I'd consider this as a feature request, the dotenv JS library (probably the de-facto implementation) already supports entries with line breaks since v15.0.0 (Jan 2022).

@mauri870
Copy link
Member

mauri870 commented Jun 9, 2024

/sig support

@k8s-ci-robot
Copy link
Contributor

@mauri870: The label(s) sig/support cannot be applied, because the repository doesn't have them.

In response to this:

/sig support

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@mauri870
Copy link
Member

mauri870 commented Jun 9, 2024

Oops, mispelling

/kind support

@k8s-ci-robot k8s-ci-robot added the kind/support Categorizes issue or PR as a support question. label Jun 9, 2024
@mauri870
Copy link
Member

mauri870 commented Jun 9, 2024

Looking at the code, it seems that supporting multiline values would require a rewrite of the rather simple parser we have in cmdutil.AddFromEnvFile. I wonder if this is a good time to migrate to a library such as godotenv instead of implementing this ourselves.

@mauri870
Copy link
Member

mauri870 commented Jun 9, 2024

Noticed that we also rely on a rather unique behavior that is not supported on the majority of dotenv libraries. If we have only the name but no value and no equal sign, the variable name assumes the value from the environment (os.Getenv("B") in the example bellow:

A=a
B
C="c"

Since this is not a dotenv standard we cannot simply migrate to a library without introducing breaking changes.

@plusiv
Copy link
Author

plusiv commented Jun 10, 2024

Hi @mauri870 !

Thanks for your reply and intention to help, I was reviewing the code behind, what about just checking prefixes and suffixes of each value (in this case "")?. That's the approach of most dotenv libraries.

@mauri870
Copy link
Member

Yes, it looks rather simple. Given that you can use \n instead of actual line breaks in the values makes supporting this optional. Our current parser assumes each entry is a single line, whence why we need to rewrite it so it knows how to consume quoted values that span multiple lines. I'll wait on the team members to decide if this is worth supporting.

@HirazawaUi
Copy link
Contributor

I think this use case is useful, but I'd like to hear from the kubectl maintainers.
/remove-kind support
/transfer kubectl

@k8s-ci-robot k8s-ci-robot removed the kind/support Categorizes issue or PR as a support question. label Jun 10, 2024
@k8s-ci-robot k8s-ci-robot transferred this issue from kubernetes/kubernetes Jun 10, 2024
@plusiv
Copy link
Author

plusiv commented Jun 11, 2024

@mauri870 @HirazawaUi Hi guys!

The PR kubernetes/kubernetes#125430 could help!

@mauri870
Copy link
Member

/assign plusiv

@sftim
Copy link
Contributor

sftim commented Jun 12, 2024

/remove-kind bug
/kind feature

Early .env implementations didn't support multiline values with line breaks, so this feels like a feature request.

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/bug Categorizes issue or PR as related to a bug. labels Jun 12, 2024
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 10, 2024
@plusiv
Copy link
Author

plusiv commented Sep 10, 2024

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 10, 2024
@mpuckett159
Copy link
Contributor

/triage accepted
I think we should just implement what is being done in the motdotla/dotenv repo so we can fall more in line with what is expected to be the standard by others. Unfortunately because of the behavior described here we can not simply import an existing library over top of the existing code, but I think we should be able to relatively easily just implement the behavior from the JS repo.

We could also implement the mentioned godotenv library as a separate flag. If someone would like to take that on, we could implement that via the KEP process with a scheduled deprecation of the old behavior.

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Sep 11, 2024
@mpuckett159
Copy link
Contributor

Personally I am very pro doing things the way that people will expect them behave and would love to see the KEP process for migrating from the old behavior to the standard behavior done.

@plusiv
Copy link
Author

plusiv commented Sep 11, 2024

Hi @mpuckett159 ,

As mentioned in the comment below, I created a PR that solves this issue without using the dotenv package. The proposed solution aims to resolve the problem in the least intrusive way possible, without modifying any critical code.

@mauri870 @HirazawaUi Hi guys!

The PR kubernetes/kubernetes#125430 could help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. sig/cli Categorizes an issue or PR as relevant to SIG CLI. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
Status: Needs Triage
Development

Successfully merging a pull request may close this issue.

7 participants