-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(ec2): 'encoded list token' error using Vpc imported from deploy-time lists #12040
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adding a test.
…ime lists Even though using `Vpc.fromVpcAttributes()` using deploy-time lists like from `Fn.importValue()`s and `CfnParameters` was never really supposed to work, it accidentally did. The reason for that is: ```ts // Encoded list token const subnetIds = Token.asList(Fn.importValue('someValue')) // [ '#{Token[1234]}' ] // Gets parsed to a singleton `Subnet` list: const subnets = subnetIds.map(s => Subnet.fromSubnetAttributes({ subnetId: s })); // [ Subnet({ subnetId: '#{Token[1234]}' }) ] // This 'subnetId' is illegal by itself, and if yould try to use it for, // say, an ec2.Instance it would fail. However, if you treat this single // subnet as a GROUP of subnets: new CfnAutoScalingGroup({ subnetIds: subnets.map(s => s.subnetId) }) // [ '#{Token[1234]}' ] // And this resolves back to: resolve(cfnSubnetIds) // SubnetIds: { Fn::ImportValue: 'someValue' } ``` -------- We introduced an additional check in #11899 to make sure that the list-element token that represents an encoded list (`'#{Token[1234]}'`) never occurs in a non-list context, because it's illegal there. However, because: * `Subnet.fromSubnetAttributes()` logs the subnetId as a *warning* to its own metadata (which will log a string like `"there's something wrong with '#{Token[1234]}' ..."`). * All metadata is resolved just the same as the template expressions are. The `resolve()` function encounters that orphaned list token in the metadata and throws. -------- The *proper* solution would be to handle unparseable list tokens specially to never get into this situation, but doing that requires introducing classes and new concepts that will be a large effort and not be backwards compatible. Tracking in #4118. Another possible solution is to stop resolving metadata. I don't know if we usefully use this feature; I think we don't. However, we have tests enforcing that it is being done, and I'm scared to break something there. The quick solution around this for now is to have `Subnet.fromSubnetAttributes()` recognize when it's about to log a problematic identifier to metadata, and don't do it. Fixes #11945.
github-actions
bot
added
the
@aws-cdk/aws-ec2
Related to Amazon Elastic Compute Cloud
label
Dec 12, 2020
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
eladb
approved these changes
Dec 14, 2020
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
flochaz
pushed a commit
to flochaz/aws-cdk
that referenced
this pull request
Jan 5, 2021
…ime lists (aws#12040) Even though using `Vpc.fromVpcAttributes()` using deploy-time lists like from `Fn.importValue()`s and `CfnParameters` was never really supposed to work, it accidentally did. The reason for that is: ```ts // Encoded list token const subnetIds = Token.asList(Fn.importValue('someValue')) // [ '#{Token[1234]}' ] // Gets parsed to a singleton `Subnet` list: const subnets = subnetIds.map(s => Subnet.fromSubnetAttributes({ subnetId: s })); // [ Subnet({ subnetId: '#{Token[1234]}' }) ] // This 'subnetId' is illegal by itself, and if yould try to use it for, // say, an ec2.Instance it would fail. However, if you treat this single // subnet as a GROUP of subnets: new CfnAutoScalingGroup({ subnetIds: subnets.map(s => s.subnetId) }) // [ '#{Token[1234]}' ] // And this resolves back to: resolve(cfnSubnetIds) // SubnetIds: { Fn::ImportValue: 'someValue' } ``` -------- We introduced an additional check in aws#11899 to make sure that the list-element token that represents an encoded list (`'#{Token[1234]}'`) never occurs in a non-list context, because it's illegal there. However, because: * `Subnet.fromSubnetAttributes()` logs the subnetId as a *warning* to its own metadata (which will log a string like `"there's something wrong with '#{Token[1234]}' ..."`). * All metadata is resolved just the same as the template expressions are. The `resolve()` function encounters that orphaned list token in the metadata and throws. -------- The *proper* solution would be to handle unparseable list tokens specially to never get into this situation, but doing that requires introducing classes and new concepts that will be a large effort and not be backwards compatible. Tracking in aws#4118. Another possible solution is to stop resolving metadata. I don't know if we usefully use this feature; I think we don't. However, we have tests enforcing that it is being done, and I'm scared to break something there. The quick solution around this for now is to have `Subnet.fromSubnetAttributes()` recognize when it's about to log a problematic identifier to metadata, and don't do it. Fixes aws#11945. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
rix0rrr
added a commit
that referenced
this pull request
Jan 18, 2021
This PR has a fix similar to #12040, and introduces more explanations and safeguards around the mechanism of importing a VPC using `fromVpcAttributes()`. This is not the recommended way of importing VPCs, but many users really don't want to use lookups so we'd better make it a little safer. This PR contains: * A fix in the EKS library to have it stop logging subnet IDs to metadata if it looks like the subnet ID will lead to the aforementioned synthesis error (similar to the fix in the linked PR). * A validation in the EKS library to stop a similar-but-different error from occurring if people select multiple VPC subnet groups from a VPC imported from token lists; this can never work and we might as well tell them directly. * A metadata warning added to VPCs imported using `Vpc.fromVpcAttributes()`, to inform users that their VPC imported in this way has a good chance of not working in all cases they expect it to. * A mechanism to specify the length of deploy-time lists at synthesis time, by passing an `assumedLength` parameter to `Fn.split()`. This will produce a list that is safe for manipulation. * A note on how to use `Vpc.fromVpcAttributes()` in the `ec2` README. Fixes #12160.
mergify bot
pushed a commit
that referenced
this pull request
Jan 19, 2021
This PR has a fix similar to #12040, and introduces more explanations and safeguards around the mechanism of importing a VPC using `fromVpcAttributes()`. This is not the recommended way of importing VPCs, but many users really don't want to use lookups so we'd better make it a little safer. This PR contains: * A fix in the EKS library to have it stop logging subnet IDs to metadata if it looks like the subnet ID will lead to the aforementioned synthesis error (similar to the fix in the linked PR). * A validation in the EKS library to stop a similar-but-different error from occurring if people select multiple VPC subnet groups from a VPC imported from token lists; this can never work and we might as well tell them directly. * A metadata warning added to VPCs imported using `Vpc.fromVpcAttributes()`, to inform users that their VPC imported in this way has a good chance of not working in all cases they expect it to. * A mechanism to specify the length of deploy-time lists at synthesis time, by passing an `assumedLength` parameter to `Fn.split()`. This will produce a list that is safe for manipulation, and removes the limitations addressed by the previous bullets. * A note on how to use `Vpc.fromVpcAttributes()` in the `ec2` README. Fixes #12160. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
mohanrajendran
pushed a commit
to mohanrajendran/aws-cdk
that referenced
this pull request
Jan 24, 2021
This PR has a fix similar to aws#12040, and introduces more explanations and safeguards around the mechanism of importing a VPC using `fromVpcAttributes()`. This is not the recommended way of importing VPCs, but many users really don't want to use lookups so we'd better make it a little safer. This PR contains: * A fix in the EKS library to have it stop logging subnet IDs to metadata if it looks like the subnet ID will lead to the aforementioned synthesis error (similar to the fix in the linked PR). * A validation in the EKS library to stop a similar-but-different error from occurring if people select multiple VPC subnet groups from a VPC imported from token lists; this can never work and we might as well tell them directly. * A metadata warning added to VPCs imported using `Vpc.fromVpcAttributes()`, to inform users that their VPC imported in this way has a good chance of not working in all cases they expect it to. * A mechanism to specify the length of deploy-time lists at synthesis time, by passing an `assumedLength` parameter to `Fn.split()`. This will produce a list that is safe for manipulation, and removes the limitations addressed by the previous bullets. * A note on how to use `Vpc.fromVpcAttributes()` in the `ec2` README. Fixes aws#12160. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
@aws-cdk/aws-ec2
Related to Amazon Elastic Compute Cloud
contribution/core
This is a PR that came from AWS.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Even though using
Vpc.fromVpcAttributes()
using deploy-time lists likefrom
Fn.importValue()
s andCfnParameters
was never really supposedto work, it accidentally did.
The reason for that is:
We introduced an additional check in #11899 to make sure that the
list-element token that represents an encoded list (
'#{Token[1234]}'
)never occurs in a non-list context, because it's illegal there.
However, because:
Subnet.fromSubnetAttributes()
logs the subnetId as a warningto its own metadata (which will log a string like
"there's something wrong with '#{Token[1234]}' ..."
).are.
The
resolve()
function encounters that orphaned list token in themetadata and throws.
The proper solution would be to handle unparseable list tokens
specially to never get into this situation, but doing that requires
introducing classes and new concepts that will be a large effort and
not be backwards compatible. Tracking in #4118.
Another possible solution is to stop resolving metadata. I don't
know if we usefully use this feature; I think we don't. However,
we have tests enforcing that it is being done, and I'm scared
to break something there.
The quick solution around this for now is to have
Subnet.fromSubnetAttributes()
recognize when it's about to loga problematic identifier to metadata, and don't do it.
Fixes #11945.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license