-
Notifications
You must be signed in to change notification settings - Fork 9.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't index from schema.Set #9693
Comments
It appears that you can get at them with the hashes for the block_device_mappings: output "test4" {
value = "${data.aws_ami.test.block_device_mappings.2547816212.ebs.snapshot_id}"
} That hash is calculated here: https://github.com/hashicorp/terraform/blob/v0.7.8/builtin/providers/aws/data_source_aws_ami.go#L453-L478 But this doesn't really seem like an acceptable way to access this stuff. I want to have the There must be some correct way in HCL to access items in Sets. |
Yes, it's interesting that the hash lookup succeeds, but not particularly useful. |
So I think the issue here is that I think what would be better is if our output was perhaps a computed map and the key was the device name. Unfortunately we don't support maps with complex values at the moment, though. :( |
What would be the consequences of having |
That is saying that the uniqueness of the entire value is determined by only the device name. That's a bit out of my understanding since I think the uniqueness is determined by a bit more but I'll lean on the AWS provider folks for that. |
It seems like sets get interpolated like lists. I just noticed a list_of_map attribute was added to test_resource so here's another example (using 0.7.10): Config
Apply run
Log
|
@jeremy-asher I believe that's only if you manually set the set since then we can assume the ordering you want is what you explicitly provided. I am actually surprised this worked though. |
@mitchellh Is resolving the list of maps type case in the works at all? I'd be willing to look into it, but it's a reasonably deep issue from what I've seen looking through the interpolation code. In the short term, do you know of any workarounds for this that would allow #9891 to work? |
@jbardin thanks for the fix, though it doesn't seem to fully cover the original case in the description. I did some testing on master and it looks like the fact that |
Sorry, this auto-closed but I agree there's a couple of related issues. The first being Being able to chain multiple index operations is a separate issue for HIL. |
I ran into this issue today and +1 for making the
Additional Request:
~ terraform version
Terraform v0.8.1 |
FYI, @jbardin #11042 does lead to a bit of progress here. I've got an additional test case that now works which probably indicates things will work once some HIL changes are made. With this output added to the code in the description:
On 0.8.2 results in:
On 0.8.3 results in:
This means the string value in a set of map can now be interpolated. The map value probably only fails due to the flat map restriction. Adding support for multiple square bracket lookups in series would probably also work. |
Thanks for your work on hashicorp/hil#42 @apparentlymart. That looks like the HIL side of the fix for this issue. |
Looks like the above HIL PR was not merged. @apparentlymart have there been any other changes to allow the index operator to work for this kind of case? |
Hi @jeremy-asher, That PR did not directly address the issue described here as far as I can tell, because the problem still remains that a set element has no "index" or "key" with which to look it up. I don't have a quick short-term fix for this one, but we've been working on some early design/discovery for various improvements to the configuration language, and I think some of the stuff we have in there would get to what you need here, by allowing filtering down to a single-element set based on some condition (e.g |
I'm not sure if it's intended, but I have been able to index into the set like a array using the square bracket notation (see my "test4" example above). This attribute may or may not make sense to be implemented as a set, but even if it wasn't a set, there's no supported language feature to get 3 levels deep to the data I want (see "test3") as far as I know which was why I originally opened this issue. |
Hi again, @jeremy-asher! The fact that there's no predictable ordering for sets makes that risky even if it is working, but I suppose as long as you only have a one-element set it's fine... I worry a little that this might get broken by future language changes, since that operation is only really working by accident now. 😖 Being able to index multiple times, like Unfortunately I have to just ask for some patience while we work through this, since the existing interpolation language has some assumptions deep in its design that make this tricky, and so we need to do more foundational work before we can get there. |
@apparentlymart okay thanks for the update. I appreciate that it's a pretty big change, but I wanted to check whether it was still on the agenda. Agreed that the set indexing is probably a bad idea regardless. Is there any reason why these various attributes are sets in the first place? I suspect AWS treats them as having ordering, but I could be mistaken. |
Indeed it's likely a historical error that the ebs volumes on the data sources are sets, originating from the fact that the data source schemata were based on the resource schemata, and AWS apparently doesn't apply ordering to these things. (There is an implied ordering by the One thing we probably could do here is to make the data sources sort by something dependable -- in this case, the At this point it feels like this issue is representing a few different things. We should probably split it out into a few different issues since they are all legitimate but different levels of effort. If the answer is to change the data sources to use lists then that's something we'd do over in the I'm not sure which one of these to consider this issue to be. If you have a preference then let me know and we can make the other two. 😀 |
Yes, that was a bit off topic, but it could help a bit. I'm more concerned about the core language changes which I'm sure will take longer to implement. It sounds like that's not quite at the stage of opening a GitHub issue, though. Was there another component to this issue you were referring to? Regarding the attribute types, there are a number of other AMI related (non-data) resources with similar attributes. I initially encountered this issue with the |
I found this issue while trying to work out how to parse The closest I've come so far is by returning [{
"device_name": "/dev/xvda",
"ebs": {
"delete_on_termination": "1",
"encrypted": "0",
"iops": "0",
"snapshot_id": "snap-05195bed51380c7b1",
"volume_size": "8",
"volume_type": "gp2"
},
"no_device": "",
"virtual_name": ""
}, {
"device_name": "/dev/xvdcz",
"ebs": {
"delete_on_termination": "1",
"encrypted": "0",
"iops": "0",
"snapshot_id": "snap-0046522873778f9e0",
"volume_size": "1024",
"volume_type": "gp2"
},
"no_device": "",
"virtual_name": ""
}] This proves that the data is there. The value I want is the
Is there any way for me to get the |
All this information is easily obtained from the AWS cli, so if you have that available you could have a script run aws cli to look up your AMI, get the block device mappings and format it with jq. Then just use that in your terraform with a #!/usr/bin/env bash
set -euo pipefail
aws ec2 describe-images --image-id "$1" | jq -r '[.Images[0].BlockDeviceMappings[] | {(.DeviceName) : .Ebs.SnapshotId}] | add' and then in your terraform: data "aws_ami" "foo" {
# .. some stuff to find the right AMI
}
data "external" "ami_snapshots" {
program = ["${path.module}/files/ami_snapshots.sh", "${data.aws_ami.foo.id}"]
}
resource "aws_instance" "bar" {
ami = "${data.aws_ami.foo.id}"
ebs_block_device {
device_name = "/dev/xvdb"
snapshot_id = "${data.external.cache_snapshots.result["/dev/xvdb"]}"
}
} Or something along those lines. It's a hacky workaround until such time as |
@b-dean that is ingenious! I'm now using something based on your solution. I wish I knew about the |
Trying to do the exact same thing as @evansj |
Hi all! There are some changes coming that will eventually address this in a few different ways. The change that will arrive first is the introduction of # details may change before release
snapshot_id = [for m in aws_ami.block_device_mappings: m if m.device_name == "/dev/xvdb"][0] The above iterates over all of the set elements, filters to just the one that has the given device name and producing a list, and then taking the first (and only) element of that list. This above will at least make it possible to access a particular element in a set, though I'll be the first to admit that it's pretty convoluted. For the longer term, we're also introducing some changes to the provider API in 0.12 which will allow providers to support maps with complex element types, as Mitchell hinted at earlier in the thread. This will not have any immediate effect for 0.12 because the plugin SDK and the providers themselves must first also be updated to make use of this new capability, but in some subsequent release of the AWS provider (determined by the AWS provider team, once the SDK is updated) we could see these exposed as a map with the device name as a key. The pattern of returning "computed" values (values defermined by the provider) in complex-typed sets will be deprecated and eventually phased out across all of the providers, since sets were introduced as a construct for representing values from configuration, and are not well-suited to exporting values to be referenced elsewhere for the reasons discussed above. As I'm sure you can imagine, this will be a gradual process as each provider team figures out the best way to update their resource schemas, so the pattern of filtering a set using a |
Hello All! The aforementioned Since the original issues here related to implementation details of the legacy SDK, which is no longer used in Terraform core itself, there is nothing else to be done here and we can close the issue. Thanks! |
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
Terraform Version
Terraform v0.7.7
Affected Resource(s)
Any resources with list or map attributes containing list or map values. This is particularly bad for the aws_ami data source's block_device_mappings attribute.
Terraform Configuration Files
Debug Output
https://gist.github.com/jeremy-asher/ff97ae0c053cc3b6134b8a8485717595
Expected Behavior
The block_device_mappings attribute is processed as a nested list of maps with values being a mixture of strings and maps. The outputs are returned as follows:
Actual Behavior
The interpolation of the attribute produces an empty list and no outputs are produced.
Steps to Reproduce
terraform apply
The text was updated successfully, but these errors were encountered: