Remove some destroy cycles #22976
Merged
Remove some destroy cycles #22976
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Destroy-time references are not correctly or fully inverted when crossing module boundaries, causing cycle during apply.
Destroy nodes do not need to be connected to the resource (prepare state) node when adding them to the graph. Destroy nodes already have a complete state in the graph (which is being destroyed), any references will be added in the ReferenceTransformer, and the proper connection to the create node will be added in the DestroyEdgeTransformer. Under normal circumstances this makes no difference, as create and destroy nodes always have an dependency, so having the prepare state handled before both only linearizes the operation slightly in the normal destroy-then-create scenario. However if there is a dependency on a resource being replaced in another module, there will be a dependency between the destroy nodes in each module (to complete the destroy ordering), while the resource node will depend on the variable->output->resource chain. If both the destroy and create nodes depend on the resource node, there will be a cycle.
after the previous commit the cycle is broken, but we can't evaluate resource counts that depends on data sources being destroyed.
If a resource is only destroying instances, there is no reason to prepare the state and we can remove the Resource (prepare state) nodes. They normally have pose no issue, but if the instances are being destroyed along with their dependencies, the resource node may fail to evaluate due to the missing dependencies (since destroy happens in the reverse order). These failures were previously blocked by there being a cycle when the destroy nodes were directly attached to the resource nodes.
ghost
added
the
sdkv1
pselle
reviewed
Oct 2, 2019
| @@ -10527,3 +10527,225 @@ func TestContext2Apply_issue19908(t *testing.T) { | |||
| t.Errorf("test.foo attributes JSON doesn't contain %s after apply\ngot: %s", want, got) | |||
| } | |||
| } | |||
|
|
|||
| func TestContext2Apply_moduleReplaceCycle(t *testing.T) { | |||
| for _, mode := range []string{"normal", "cbd"} { | |||
There was a problem hiding this comment.
huh! interesting approach to help with the super-bloat of this test file. since this string is only used for the switch, I feel like it would reduce my cognitive dissonance to see "create before destroy" vs cbd (fine in the file name below) just so I don't have to take that extra brain cycle not to think of hemp.
|
Merging into the other PR branch first, rather than rebasing |
appilon
removed
the
sdkv1
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
ghost
locked and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR removes extra connections from the graph which are unneeded, however only cause problems in certain destroy scenarios. This solves a number of common cycles issues, while simplifying the graph itself. It does not solve all cycles, as destroy nodes still need their references evaluated in order for destroy-provisioners to function.
Destroy nodes do not need to be connected to the resource (prepare
state) node when adding them to the graph. Destroy nodes already have a
complete state in the graph (which is being destroyed), any references
will be added in the ReferenceTransformer, and the proper
connection to the create node will be added in the
DestroyEdgeTransformer.
Under normal circumstances this makes no difference, as create and
destroy nodes always have a direct dependency, so having the
prepare state handled before both only linearizes the operation slightly
in the normal destroy-then-create scenario.
However if there is a dependency on a resource being replaced in another
module, there will be a dependency between the destroy nodes in each
module (to complete the destroy ordering), while the resource node will
depend on the variable->output->resource chain. If both the destroy and
create nodes depend on the resource node, there will be a cycle.
If a resource is only destroying instances, there is no reason to
prepare the state and we can remove the Resource (prepare state) nodes.
They normally have pose no issue, but if the instances are being
destroyed along with their dependencies, the resource node may fail to
evaluate due to the missing dependencies (since destroy happens in the
reverse order).
These failures were previously blocked by there being a cycle when the
destroy nodes were directly attached to the resource nodes.
This depends on jbardin/cbd-modules, and can be merged into that first, or re-targeted to later to master.
Fixes #22502
Does not close #21662, but does fix some of the in-line examples that have been provided.