Enable "helm upgrade --install" equivalent behavior #1247

n-oden · 2023-09-07T16:44:54Z

Description

This PR adds a (potentially) idempotent "upgrade mode" to the provider, mimicking the behavior of helm upgrade --install as defined in https://github.com/helm/helm/blob/main/cmd/helm/upgrade.go

To wit: an upgrade block is added to the resource attributes, consisting of two boolean values: enable and install.

If enabled is true, this causes the provider to create a *action.Upgrade client, and attempts to perform an upgrade on the named chart. If install is true, it will first create an *action.History client to determine if a release already exists; if it does not find one it creates an *action.Install client and attempts to install the release from scratch. If a release is found, an upgrade is performed. This allows the resource to potentially co-exist with, e.g., CI/CD systems that could potentially install the release out-of-band from terraform's viewpoint.

Acceptance tests

Have you added an acceptance test for the functionality being added?

Release Note

Release note for CHANGELOG:

ENHANCEMENT:
* `resource/helm_release`: add `upgrade` map attribute to enable idempotent release installation, addressing components of [GH-425](https://github.com/hashicorp/terraform-provider-helm/issues/425)

References

Error: cannot re-use a name that is still in use #425

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

hashicorp-cla · 2023-09-07T16:44:58Z

All committers have signed the CLA.

n-oden · 2023-09-17T15:56:24Z

As I noted over in the #425, I've released a test build of this for people to kick the tires on:

https://registry.terraform.io/providers/n-oden/helm/latest

n-oden · 2023-09-28T13:44:28Z

Bump: could someone re-approve the checks? I've updated the PR to address the failing ones.

helm/resource_release.go

n-oden · 2023-10-24T14:16:55Z

Updated, all tests passing locally.

n-oden · 2023-10-31T13:19:22Z

@bd-spl gentle nudge? :)

bd-spl · 2023-11-08T09:27:33Z

@bd-spl gentle nudge? :)

I welcome this change, albeit have no merging powers here. Please try reaching out the owners of this repository.
I think the 'copywrite' check is blocking this effort?

n-oden · 2023-11-09T19:42:34Z

@alexsomesan can I prevail on you to take a look here? I think this is ready to go.

n-oden · 2023-11-29T14:40:51Z

@alexsomesan nudge? :)

n-oden · 2023-12-13T16:12:29Z

@alexsomesan this is now rebased on top of the current main branch. Is there any chance this could be reviewed before EOY? I've got an internal project that's rather blocked by this and would like to avoid the temptation to use a forked version without a formal 👍 on the approach from your team.

n-oden · 2024-01-03T15:19:25Z

Happy New Year everyone! @alexsomesan my apologies for being a bother, but would it be possible to get the test workflows approved? The tests are passing locally and I believe absent any other feedback that this is ready for submission.

lodmfjord · 2024-01-25T10:53:58Z

Is there any news on this?

n-oden · 2024-01-26T14:54:27Z

@lodmfjord I'm honestly not sure what's going on here; we seem to have fallen into a bit of a hole. In the meantime if you'd like to kick the tires on this you can use https://registry.terraform.io/providers/n-oden/helm/0.0.2 which is a fork of the hashicorp helm provider with this PR cherry-picked on top. Obviously the caveat there is that there's no guarantee that they'll ever merge this back upstream, or hashicorp might ask for incompatible state schema changes before they do, so using it in production is very much an at your own risk proposition and I probably can't personally commit to keeping a fork up to date in perpetuity if this PR never gets accepted.

n-oden · 2024-02-20T15:57:38Z

@arybolovlev @SarahFrench @appilon @jrhouston my apologies for the spam, but I'm hoping to get someone to approve running the checks on this PR, which seems to have gotten a little lost.

This PR adds a (potentially) idempotent "upgrade mode" to the provider, mimicing the behavior of `helm upgrade --install` as defined in https://github.com/helm/helm/blob/main/cmd/helm/upgrade.go To wit: an `upgrade` block is added to the resource attributes, consisting of tool boolean values: `enable` and `install`. If `enable` is true, this causes the provider to create a `*action.Upgrade` client, and attempts to perform an upgrade on the named chart. If `install` is true, it will first create an `*action.History` client to determine if a release already exists; if it does not find one it creates an `*action.Install` client and attempts to install the release from scratch. If a release _is_ found, an upgrade is performed. This allows the resource to potentially co-exist with, e.g., CI/CD systems that could potentially install the release out-of-band from terraform's viewpoint.

n-oden · 2024-08-09T21:20:24Z

The failure in acc_test (v0.15.5) does not look like it's in any way obviously relevant to this change; can you try re-running that job?

BBBmau · 2024-08-09T21:23:20Z

The failure in acc_test (v0.15.5) does not look like it's in any way obviously relevant to this change; can you try re-running that job?

Yeah often times the tests can fail due to a network issue.

We'll want to add in retries to prevent us from needing to rerun the jobs all the time @JaylonmcShan03

BBBmau · 2024-08-09T23:03:39Z

@n-oden did the same manual test i did before and I see that it uses the latest version rather than the chart version already being used on the chart:

when running TF_LOG=debug terraform apply:

32 [DEBUG] [getInstalledReleaseVersion: mau-test-pr] Chart mau-test-pr is installed as release 19.6.1: timestamp=2024-08-09T15:58:32.743-0700
2024-08-09T15:58:33.849-0700 [INFO]  provider.terraform-provider-helm_9.9.9_darwin_arm64: 2024/08/09 15:58:33 [DEBUG] [resourceDiff: mau-test-pr] Got chart redis version 19.6.1: timestamp=2024-08-09T15:58:33.849-0700
2024-08-09T15:58:33.850-0700 [INFO]  provider.terraform-provider-helm_9.9.9_darwin_arm64: 2024/08/09 15:58:33 [DEBUG] [resourceDiff: mau-test-pr] Release validated: timestamp=2024-08-09T15:58:33.850-0700
2024-08-09T15:58:33.850-0700 [INFO]  provider.terraform-provider-helm_9.9.9_darwin_arm64: 2024/08/09 15:58:33 [DEBUG] [resourceDiff: mau-test-pr] setting version to installed version 19.6.1: timestamp=2024-08-09T15:58:33.850-0700

This appears with this tfconfig where tfstate is empty as well as the release being installe with version being 19.4.0

tfconfig:

resource "helm_release" "test" {
  name = "mau-test-pr"
  chart = "bitnami/redis"
  # version = "19.4.0"
  upgrade_install = true
}

from helm list:

└─(16:00:30)──> helm list                               ──(Fri,Aug09)─┘
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
mau-test-pr     default         18              2024-08-09 15:58:07.823949 -0700 PDT    deployed        redis-19.4.0    7.2.5

n-oden · 2024-08-12T13:24:56Z

@BBBmau okay that's weird -- I would think that the test case in TestAccResourceRelease_upgrade_with_install_warmstart_no_version would cover this exact scenario: we preinstall test-chart v1.2.3, apply the upgrade_install plan and then verify that v1.2.3 is still installed.

I'll try to figure out what's going on, but can you please attach the full debug log of your tf plan/apply sessions as well as the helm cli commands your ran and their output in a gist?

n-oden · 2024-08-12T17:19:15Z

@BBBmau I don't seem to be able to reproduce this? Take a look at this gist: https://gist.github.com/n-oden/d54d278d3a2ffbc5e2fb81d6f00d3092

The upshot is:

manually helm install redis-test bitnami/redis --version 19.4.0; helm list confirms that chart version 19.4.0 is installed at revision 1
terraform plan/apply with upgrade_install=true and version=null
helm list confirms that revision 2 has been installed but still chart vresion 19.4.0
terraform plan confirms no changes at this point
the string 19.4.1 does not appear in any of the log files

Can you double-check this on your side?

n-oden · 2024-08-12T17:34:13Z

One thing that's probably worth explicitly noting is that this does create a new revision, which means that if there are deployments or statefulsets in your release, it will create a kubernetes rollout event, with the associated pod restarts. It's the moral equivalent of running:

$ helm install  --version 19.4.0 myredis bitnami/redis
$ helm upgrade --version 19.4.0 myredis bitnami/redis

...which if you do it locally you will observe does create rollout events for both the deploy and sts resources.

I'm going to assert strongly that this is expected behavior and I'll add a note about it to the documentation.

also use the test repo in the tests for upgrade_install scenarios

BBBmau

When testing on my machine I was unable to get the behavior despite being able to run the tests and getting them to pass. @jrhouston was able to run manual tests on this PR and it all looks to be running as expected.

@n-oden We really appreciate your contribution! Just apply the documentation changes and we'll get this approved and merged 🎉

docs/resources/release.md

templates/resources/release.md.tmpl

BBBmau · 2024-08-13T00:56:36Z

docs/index.md

-* `command` - (Required) Command to execute.
-* `args` - (Optional) List of arguments to pass when executing the plugin.
-* `env` - (Optional) Map of environment variables to set when executing the plugin.
+  * `api_version` - (Required) API version to use when decoding the ExecCredentials resource, e.g. `client.authentication.k8s.io/v1beta1`.


These extra spaces should be removed.

I agree! But tfplugindocs generate really wants them to be there, and fixing this breaks the documentation check build. I'll revert them out manually but someone should probably figure out why tfplugndocs keeps applying them.

yeah for right now we've been ignoring the documentation workflow due to it generating documentation that's just flat out wrong. Thanks for at least attempting to use it though!

BBBmau

This is a big contribution that the community has been asking for for quite sometime. You're more than welcome to pick up other issues that catch your eye if you're interested. Thank you again! 🎉

n-oden · 2024-08-13T14:36:33Z

@BBBmau thank you for all of your help along this long and winding road! Now... ETA to the next release of the provider? 🤣

BBBmau · 2024-08-14T19:07:45Z

@n-oden Thank you for your contribution 😄

https://github.com/hashicorp/terraform-provider-helm/releases/tag/v2.15.0

n-oden requested a review from a team as a code owner September 7, 2023 16:44

github-actions bot added documentation size/XL labels Sep 7, 2023

n-oden force-pushed the upgrade-mode branch from 0decfbf to a8d9938 Compare September 7, 2023 17:26

n-oden mentioned this pull request Sep 7, 2023

Error: cannot re-use a name that is still in use #425

Closed

n-oden force-pushed the upgrade-mode branch from a8d9938 to 72b261e Compare September 22, 2023 18:40

alexsomesan reviewed Oct 11, 2023

View reviewed changes

helm/resource_release.go Outdated Show resolved Hide resolved

n-oden force-pushed the upgrade-mode branch from cd53b57 to b72ab47 Compare October 14, 2023 19:15

n-oden requested a review from alexsomesan October 14, 2023 19:15

This comment was marked as resolved.

Sign in to view

bd-spl reviewed Oct 18, 2023

View reviewed changes

helm/resource_release.go Outdated Show resolved Hide resolved

n-oden requested a review from bd-spl October 24, 2023 14:16

n-oden force-pushed the upgrade-mode branch from 607c23d to b723cf8 Compare December 13, 2023 15:49

n-oden force-pushed the upgrade-mode branch from b723cf8 to 06110fe Compare January 3, 2024 15:18

n-oden force-pushed the upgrade-mode branch 2 times, most recently from 3c6f6e8 to 599bca0 Compare January 17, 2024 20:02

n-oden changed the title ~~Proposal: upgrade mode~~ Enable "helm upgrade --install" equivalent behavior Jan 17, 2024

n-oden added 2 commits August 9, 2024 17:05

simplify configuration to a single boolean attr

9d4fb27

n-oden force-pushed the upgrade-mode branch from 6dd3fb3 to 17fc4fd Compare August 9, 2024 21:06

github-actions bot added size/XL and removed size/L labels Aug 9, 2024

n-oden force-pushed the upgrade-mode branch from 17fc4fd to 0470be1 Compare August 9, 2024 21:08

n-oden requested a review from BBBmau August 9, 2024 21:20

n-oden force-pushed the upgrade-mode branch from 0470be1 to fbaf170 Compare August 12, 2024 17:58

do not auto-update when version is unset

ffc2134

also use the test repo in the tests for upgrade_install scenarios

n-oden force-pushed the upgrade-mode branch from fbaf170 to ffc2134 Compare August 12, 2024 18:03

BBBmau requested changes Aug 13, 2024

View reviewed changes

docs update

eaf6230

n-oden requested a review from BBBmau August 13, 2024 13:24

BBBmau added the no-documentation This issue or pull request does not have documentation changes. label Aug 13, 2024

BBBmau approved these changes Aug 13, 2024

View reviewed changes

BBBmau merged commit 723cb76 into hashicorp:main Aug 13, 2024
21 of 22 checks passed

n-oden deleted the upgrade-mode branch August 13, 2024 14:35

mouellet mentioned this pull request Sep 6, 2024

fix: Missing postrender args #1477

Open

1 task

n-oden mentioned this pull request Sep 19, 2024

Handle errors during diff stage when using upgrade_mode #1489

Closed

n-oden mentioned this pull request Oct 2, 2024

upgrade_mode fails if the cluster is being built #1495

Open

This was referenced Nov 14, 2024

postrender ignores args since v2.15.0 #1533

Closed

🐛 Fix missed postrender binary path arguments parsing #1534

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable "helm upgrade --install" equivalent behavior #1247

Enable "helm upgrade --install" equivalent behavior #1247

n-oden commented Sep 7, 2023 •

edited

Loading

hashicorp-cla commented Sep 7, 2023 •

edited

Loading

n-oden commented Sep 17, 2023

n-oden commented Sep 28, 2023

This comment was marked as resolved.

n-oden commented Oct 24, 2023

n-oden commented Oct 31, 2023

bd-spl commented Nov 8, 2023

n-oden commented Nov 9, 2023 •

edited

Loading

n-oden commented Nov 29, 2023

n-oden commented Dec 13, 2023

n-oden commented Jan 3, 2024

lodmfjord commented Jan 25, 2024

n-oden commented Jan 26, 2024

n-oden commented Feb 20, 2024

n-oden commented Aug 9, 2024

BBBmau commented Aug 9, 2024

BBBmau commented Aug 9, 2024

n-oden commented Aug 12, 2024

n-oden commented Aug 12, 2024

n-oden commented Aug 12, 2024

BBBmau left a comment

BBBmau Aug 13, 2024

n-oden Aug 13, 2024

BBBmau Aug 13, 2024

BBBmau left a comment

n-oden commented Aug 13, 2024

BBBmau commented Aug 14, 2024

Enable "helm upgrade --install" equivalent behavior #1247

Enable "helm upgrade --install" equivalent behavior #1247

Conversation

n-oden commented Sep 7, 2023 • edited Loading

Description

Acceptance tests

Release Note

References

Community Note

hashicorp-cla commented Sep 7, 2023 • edited Loading

n-oden commented Sep 17, 2023

n-oden commented Sep 28, 2023

This comment was marked as resolved.

n-oden commented Oct 24, 2023

n-oden commented Oct 31, 2023

bd-spl commented Nov 8, 2023

n-oden commented Nov 9, 2023 • edited Loading

n-oden commented Nov 29, 2023

n-oden commented Dec 13, 2023

n-oden commented Jan 3, 2024

lodmfjord commented Jan 25, 2024

n-oden commented Jan 26, 2024

n-oden commented Feb 20, 2024

n-oden commented Aug 9, 2024

BBBmau commented Aug 9, 2024

BBBmau commented Aug 9, 2024

n-oden commented Aug 12, 2024

n-oden commented Aug 12, 2024

n-oden commented Aug 12, 2024

BBBmau left a comment

Choose a reason for hiding this comment

BBBmau Aug 13, 2024

Choose a reason for hiding this comment

n-oden Aug 13, 2024

Choose a reason for hiding this comment

BBBmau Aug 13, 2024

Choose a reason for hiding this comment

BBBmau left a comment

Choose a reason for hiding this comment

n-oden commented Aug 13, 2024

BBBmau commented Aug 14, 2024

n-oden commented Sep 7, 2023 •

edited

Loading

hashicorp-cla commented Sep 7, 2023 •

edited

Loading

n-oden commented Nov 9, 2023 •

edited

Loading