Troposphere types

[mwildehahn]

Adds a new custom variable type: TroposphereType. This builds off @phobologic's improvements to from_dict within Troposphere. TroposphereType lets you pass through the yaml from a config directly to troposphere types which saves you the boilerplate of specifying variables within the blueprint or instantiating the nested types yourself.

Open to suggestions about naming or other syntax.

An example:

config file:

- name: myBuckets
  class_path: stacker_blueprints.s3.Buckets
  variables:
    Buckets:
      - BucketName: my-bucket
      - BucketName: my-other-bucket

stacker_blueprints.s3.Buckets:

from stacker.blueprints.base import Blueprint
from stacker.blueprints.variables.types import TroposphereType
from troposphere import s3


class Buckets(Blueprint):

    VARIABLES = {
        "Buckets": {
            "type": TroposphereType([s3.Bucket]),
            "description": "s3 buckets to create",
        },
    }

    def create_template(self):
        t = self.template
        variables = self.get_variables()
        [t.add_resource(bucket) for bucket in variables["Buckets"]]

[phobologic]

I'd just move this into the 1.0.0a1 section - I plan to just update that to be the 1.0.0 release when we're ready to go.

[phobologic]

Is this, and a single type, the only applicable syntax for TroposphereType? If so, probably worth documenting both.

[phobologic]

Rather than this, what about making it so that every resource should have a title attribute that is required? That's the name of the variable that is used in troposphere. Another option is that we could do away with both the list & plain type syntax, and instead replace it with a single format that uses dictionaries like I did with the SecurityGroup blueprint: remind101/stacker_blueprints@ab094c3#diff-e1bf8fc9a0d5fa2e2b507057d3fcd3aaR29

[phobologic]

I think the way I did it in the security group blueprint actually makes a lot of sense. You have to give unique names to all resources in a blueprint, so you might as well make it a requirement in the config. Going to hold off on further review till you reply to this.

[mwildehahn]

good call

[mwildehahn]

@phobologic updated based on your feedback

[phobologic]

Honestly, at this point, my only concern is the fact we call it a list type, but when you define it in the config its actually a dict. Not sure if this is a big deal or not - dict makes more sense from the config perspective, but maybe making it a list of single item dicts would bridge the gap:

Buckets:
    - FirstBucket:
        BucketName: my-first-bucket
    - SecondBucket:
        BucketName: my-second-bucket

What do you think?

[phobologic]

Hmm, the other option is to step away from the python types altogether and instead go with our own syntax. TroposphereType ManyTroposphereType or something?

[mwildehahn]

We talked about this on slack, but we decided we should do TroposphereType(<type>, many=True)

[mwildehahn]

@phobologic i think this is good to go

[phobologic]

LGTM - only one potential enhancement I'd like to see. Thanks!

[phobologic]

Is there anyway we can make this error more helpful in determining where the issue is? Maybe we need to catch this higher and point out what caused this?

[phobologic]

NM, just realized this is caught and re-raised as a ValidatorError, which will point out where the issue stems from. Awesome, going to go ahead and merge this.