Add ability to reorder var and var-file options

[chilledornaments]

Description

This adds a SetVarsAfterVarFiles field to the Options struct. When set to true, FormatArgs will place -var options after -var-file options. When false, FormatArgs will maintain its current behavior. This is a similar idea to #256.

This will increase terratest's flexibility without introducing a breaking changes 🙂

TODOs

Read the Gruntwork contribution guidelines.

• Update the docs.
• Run the relevant tests successfully, including pre-commit checks.
• Ensure any 3rd party code adheres with our license policy or delete this line if its not applicable.
• Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Added ability to control order of -var and -var-file options passed to Terraform

Migration Guide

[denis256]

Hi,
consecutive runs of TestFormatSetVarsAfterVarFilesFormatsCorrectly periodically fail with error on the assertion

=== CONT  TestFormatSetVarsAfterVarFilesFormatsCorrectly
    format_test.go:298: 
        	Error Trace:	format_test.go:298
        	Error:      	Not equal: 
        	            	expected: []string{"plan", "-var-file", "test.tfvars", "-var", "foo=bar", "-var", "hello=world", "-lock=false"}
        	            	actual  : []string{"plan", "-var-file", "test.tfvars", "-var", "hello=world", "-var", "foo=bar", "-lock=false"}
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -5,5 +5,5 @@
        	            	  (string) (len=4) "-var",
        	            	+ (string) (len=11) "hello=world",
        	            	+ (string) (len=4) "-var",
        	            	  (string) (len=7) "foo=bar",
        	            	- (string) (len=4) "-var",
        	            	- (string) (len=11) "hello=world",
        	            	  (string) (len=11) "-lock=false"
        	Test:       	TestFormatSetVarsAfterVarFilesFormatsCorrectly
--- FAIL: TestFormatSetVarsAfterVarFilesFormatsCorrectly (0.00s)

[chilledornaments]

HI @denis256,

Just pushed a commit to fix that! I didn't catch that failure locally because I hadn't cleaned my cache. Working locally now 🙂

[14:40:01 ~/repos/chilledornaments/terratest/modules/terraform] > go clean -cache
[14:40:05 ~/repos/chilledornaments/terratest/modules/terraform] > go test -timeout 30s -run ^TestFormatSetVarsAfterVarFilesFormatsCorrectly$ 
PASS
ok      github.com/gruntwork-io/terratest/modules/terraform     0.275s

[denis256]

Hello,
test case is still failing periodically:

=== CONT  TestFormatSetVarsAfterVarFilesFormatsCorrectly
    format_test.go:299: 
        	Error Trace:	format_test.go:299
        	Error:      	Not equal: 
        	            	expected: []string{"plan", "-var", "foo=bar", "-var", "hello=world", "-var-file", "test.tfvars", "-lock=false"}
        	            	actual  : []string{"plan", "-var", "hello=world", "-var", "foo=bar", "-var-file", "test.tfvars", "-lock=false"}
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,5 +3,5 @@
        	            	  (string) (len=4) "-var",
        	            	+ (string) (len=11) "hello=world",
        	            	+ (string) (len=4) "-var",
        	            	  (string) (len=7) "foo=bar",
        	            	- (string) (len=4) "-var",
        	            	- (string) (len=11) "hello=world",
        	            	  (string) (len=9) "-var-file",
        	Test:       	TestFormatSetVarsAfterVarFilesFormatsCorrectly
--- FAIL: TestFormatSetVarsAfterVarFilesFormatsCorrectly (0.00s)


Expected :[]string{"plan", "-var", "foo=bar", "-var", "hello=world", "-var-file", "test.tfvars", "-lock=false"}
Actual   :[]string{"plan", "-var", "hello=world", "-var", "foo=bar", "-var-file", "test.tfvars", "-lock=false"}

I think should be implemented more smart verification than assert.Equal()

[chilledornaments]

Hi @denis256 ,

I implemented logic in the test to use the checkResultWithRetry() function based on this comment. The test failures are caused by Go not caring about the order of keys when iterating over a map, not a problem with the changes to FormatArgs() (the order of -var and -var-file flags is always correct, but the order of -var arguments may change between runs, which is causing the test failures)

I was able to reproduce the test failures by running the test I added multiple times. I was also able to verify that the tests will pass given enough loops:

$ for i in {0..10}; do go clean -cache; go test -timeout 30s -run ^TestFormatSetVarsAfterVarFilesFormatsCorrectly$ -v ; done

[TRUNCATED] 

=== RUN   TestFormatSetVarsAfterVarFilesFormatsCorrectly
=== PAUSE TestFormatSetVarsAfterVarFilesFormatsCorrectly
=== CONT  TestFormatSetVarsAfterVarFilesFormatsCorrectly
--- PASS: TestFormatSetVarsAfterVarFilesFormatsCorrectly (0.00s)
PASS
ok      github.com/gruntwork-io/terratest/modules/terraform     0.266s
=== RUN   TestFormatSetVarsAfterVarFilesFormatsCorrectly
=== PAUSE TestFormatSetVarsAfterVarFilesFormatsCorrectly
=== CONT  TestFormatSetVarsAfterVarFilesFormatsCorrectly
    format_test.go:149: Retry 0 of FormatArgs(1) failed: expected [plan -var-file test.tfvars -var foo=bar -var hello=world -lock=false], got [plan -var-file test.tfvars -var hello=world -var foo=bar -lock=false]
    format_test.go:149: Retry 0 of FormatArgs(2) failed: expected [plan -var foo=bar -var hello=world -var-file test.tfvars -lock=false], got [plan -var hello=world -var foo=bar -var-file test.tfvars -lock=false]
--- PASS: TestFormatSetVarsAfterVarFilesFormatsCorrectly (0.00s)
PASS
ok      github.com/gruntwork-io/terratest/modules/terraform     0.249s
=== RUN   TestFormatSetVarsAfterVarFilesFormatsCorrectly
=== PAUSE TestFormatSetVarsAfterVarFilesFormatsCorrectly
=== CONT  TestFormatSetVarsAfterVarFilesFormatsCorrectly
--- PASS: TestFormatSetVarsAfterVarFilesFormatsCorrectly (0.00s)
PASS
ok      github.com/gruntwork-io/terratest/modules/terraform     0.275s

[TRUNCATED] 

[denis256]

Hello,
approach using checkResultWithRetry is confusing, it makes to think that options formatting may not work each time.

A simplified approach may be to compare index of -var and -var-file arguments, which one should be before of others

[chilledornaments]

Hi @denis256 ,

Thanks for the suggestion 🙂 I've updated the test to do just that!

[denis256]

Failing tests aren't related to implemented changes