command/init: Add a new flag `-lockfile=readonly` #27630

minamijoyo · 2021-01-29T13:11:37Z

Add a new flag -lockfile=readonly to terraform init. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the -lockfile flag is string rather than bool, leaving room for future extensions to other behavior variants.

The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the -upgrade flag.

Note: In the original use-case described in #27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing.

codecov · 2021-01-29T13:15:12Z

Codecov Report

Merging #27630 (f241d60) into main (98899df) will increase coverage by 0.03%.
The diff coverage is 85.71%.

Impacted Files	Coverage Δ
internal/depsfile/locks.go	`55.73% <66.66%> (+1.19%)`	⬆️
command/init.go	`56.39% <90.90%> (+2.56%)`	⬆️
states/statefile/version4.go	`58.19% <0.00%> (+0.23%)`	⬆️
terraform/node_resource_plan.go	`98.05% <0.00%> (+1.94%)`	⬆️

wyardley · 2021-02-01T21:22:36Z

What about a flag to ignore it vs. not update it?

richardj-bsquare · 2021-02-04T09:37:02Z

An option to 'ignore' would be ideal for CI/CD systems as it somewhat avoids the multi-arch issue...

#27506 (comment)

mumoshu · 2021-02-05T01:55:56Z

@wyardley @richardj-bsquare Sorry for the plug but I believe you should submit separate feature requests/pull requests your yours. Let's make separate things separate. Also, changes in this PR should be awesome foundations for your future enhancements.

I find this feature pretty solid and valid on its own 👍

minamijoyo · 2021-02-08T13:53:59Z

@apparentlymart Could you review this?

alisdair

Thanks for following up on this, @minamijoyo! Sorry for the delay in reviewing.

This looks good to me, with some minor notes inline from reading the code. I'd like to test it out locally and get a second opinion from someone else on @hashicorp/terraform-core.

alisdair · 2021-03-03T15:52:12Z

command/init.go

@@ -767,6 +773,12 @@ func (c *InitCommand) getProviders(config *configs.Config, state *states.State,
 	// it's the smallest change relative to what came before it, which was
 	// a hidden JSON file specifically for tracking providers.)
 	if !newLocks.Equal(previousLocks) {
+		// if readonly mode, suppress changes
+		if flagLockfile == "readonly" {
+			log.Println("[DEBUG] init: detected changing dependencies, but suppressed by readonly mode")


I think we may want to output a warning to the UI here rather than a debug log. Maybe returning a warning diagnostic? For example:

diags = diags.Append(tfdiags.Sourceless( tfdiags.Warning, "Provider lock file not updated", "Changes to the provider selections were detected, but not saved in the .terraform.lock.hcl file. To record these selections, run "terraform init" without the "-lockfile=readonly" flag., ))

alisdair · 2021-03-03T15:53:28Z

website/docs/cli/commands/init.html.md

@@ -157,6 +157,7 @@ You can modify `terraform init`'s plugin behavior with the following options:
  You can use `-plugin-dir` as a one-time override for exceptional situations,
  such as if you are testing a local build of a provider plugin you are
  currently developing.
+- `-lockfile=MODE` Set dependency lockfile mode. Currently only "readonly" is valid.


It would be great to get some more detail on why you'd want to use this flag, and what the consequences are.

alisdair

I tested this locally, and while the happy path worked, I found one problem. From Martin's comment on the original issue:

I think the main requirement for that option is that installation can succeed as long as all packages can be verified with information already recorded in the lock file. The option would disable Terraform from updating the file to record any new information it learned (such as a hash using a new scheme) but Terraform would still rely on and check against the information already recorded.

I don't think this requirement is met with this PR. Here's what I did:

Initialized a configuration depending on only hashicorp/aws, and committed the lockfile
Deleted the .terraform cache directory
Added a requirement for hashicorp/random to the configuration
terraform init -lockfile=readonly

Expected: error due to missing lockfile entry for hashicorp/random
Actual: initialization succeeded without error

Unless I'm misunderstanding that requirement, I think we'd need to fix this issue before merge, so I'm marking this as "request changes" for now.

Fixes hashicorp#27506 Add a new flag `-lockfile=readonly` to `terraform init`. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the `-lockfile` flag is string rather than bool, leaving room for future extensions to other behavior variants. The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the `-upgrade` flag. Note: In the original use-case described in hashicorp#27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing.

minamijoyo · 2021-03-08T15:43:19Z

@alisdair Thank you for reviewing!
I added a check if required provider dependencies are changed.

alisdair

Thanks @minamijoyo, this looks good to me! I made a quick copy edit to one of the diagnostics to keep with our normal UI text house style, just to save another review round-trip.

Fixes #27506 Add a new flag `-lockfile=readonly` to `terraform init`. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the `-lockfile` flag is string rather than bool, leaving room for future extensions to other behavior variants. The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the `-upgrade` flag. Note: In the original use-case described in #27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing. Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>

minamijoyo · 2021-03-10T00:16:12Z

@alisdair Thanks!

Fixes hashicorp#27506 Add a new flag `-lockfile=readonly` to `terraform init`. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the `-lockfile` flag is string rather than bool, leaving room for future extensions to other behavior variants. The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the `-upgrade` flag. Note: In the original use-case described in hashicorp#27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing. Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>

command/init: Add a new flag `-lockfile=readonly` (v0.15 backport of #27630)

ghost · 2021-04-09T01:50:17Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

minamijoyo force-pushed the init-lockfile-readonly branch from edc341a to 755a48d Compare February 2, 2021 02:40

mildwonkey added the enhancement label Feb 8, 2021

wyardley mentioned this pull request Feb 23, 2021

provide method (env var or flag) to restore old locking behavior #27888

Closed

Base automatically changed from master to main February 24, 2021 18:01

alisdair requested a review from a team March 2, 2021 20:10

alisdair approved these changes Mar 3, 2021

View reviewed changes

alisdair suggested changes Mar 3, 2021

View reviewed changes

minamijoyo added 4 commits March 9, 2021 00:20

Reject to change required provider dependences in the readonly mode

699ee3e

Warn if provider selections are changed in the readonly mode

f7953e8

Add an explanation about the readonly mode

e1af3d6

minamijoyo force-pushed the init-lockfile-readonly branch from a9a8201 to e1af3d6 Compare March 8, 2021 15:30

Update diagnostic message

f241d60

alisdair approved these changes Mar 9, 2021

View reviewed changes

alisdair merged commit 31a5aa1 into hashicorp:main Mar 9, 2021

alisdair mentioned this pull request Mar 9, 2021

command/init: Add a new flag -lockfile=readonly (v0.15 backport of #27630) #28026

Merged

minamijoyo mentioned this pull request Mar 10, 2021

command/init: Add a new flag -lockfile=readonly (v0.14 backport of #27630) #28033

Closed

alisdair added a commit that referenced this pull request Mar 11, 2021

Merge pull request #28026 from hashicorp/alisdair/backport-27630-v0.15

68a4210

command/init: Add a new flag `-lockfile=readonly` (v0.15 backport of #27630)

minamijoyo mentioned this pull request Mar 14, 2021

How to update .terraform.lock.hcl introduced in Terraform v0.14 minamijoyo/tfupdate#32

Closed

alisdair mentioned this pull request Mar 29, 2021

cli: Only rewrite provider locks file if changed #28230

Merged

teamterraform mentioned this pull request Mar 30, 2021

Backport of cli: Only rewrite provider locks file if changed into v0.14 #28235

Merged

teamterraform mentioned this pull request Mar 30, 2021

Backport of cli: Only rewrite provider locks file if changed into v0.15 #28236

Merged

ghost locked as resolved and limited conversation to collaborators Apr 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

command/init: Add a new flag `-lockfile=readonly` #27630

command/init: Add a new flag `-lockfile=readonly` #27630

minamijoyo commented Jan 29, 2021

codecov bot commented Jan 29, 2021 •

edited

Loading

wyardley commented Feb 1, 2021

richardj-bsquare commented Feb 4, 2021

mumoshu commented Feb 5, 2021

minamijoyo commented Feb 8, 2021

alisdair left a comment

alisdair Mar 3, 2021

minamijoyo Mar 8, 2021

alisdair Mar 3, 2021

minamijoyo Mar 8, 2021

alisdair left a comment

minamijoyo commented Mar 8, 2021

alisdair left a comment

minamijoyo commented Mar 10, 2021

ghost commented Apr 9, 2021

command/init: Add a new flag -lockfile=readonly #27630

command/init: Add a new flag -lockfile=readonly #27630

Conversation

minamijoyo commented Jan 29, 2021

codecov bot commented Jan 29, 2021 • edited Loading

Codecov Report

wyardley commented Feb 1, 2021

richardj-bsquare commented Feb 4, 2021

mumoshu commented Feb 5, 2021

minamijoyo commented Feb 8, 2021

alisdair left a comment

Choose a reason for hiding this comment

alisdair Mar 3, 2021

Choose a reason for hiding this comment

minamijoyo Mar 8, 2021

Choose a reason for hiding this comment

alisdair Mar 3, 2021

Choose a reason for hiding this comment

minamijoyo Mar 8, 2021

Choose a reason for hiding this comment

alisdair left a comment

Choose a reason for hiding this comment

minamijoyo commented Mar 8, 2021

alisdair left a comment

Choose a reason for hiding this comment

minamijoyo commented Mar 10, 2021

ghost commented Apr 9, 2021

command/init: Add a new flag `-lockfile=readonly` #27630

command/init: Add a new flag `-lockfile=readonly` #27630

codecov bot commented Jan 29, 2021 •

edited

Loading