A semi-interactive tool for helping to create and update AWS CloudFormation stacks.
This tool already exists (I wrote about it here: https://medium.com/@rvedotrc/managing-aws-cloudformation-templates-using-stack-fetcher-4d798d406fd0) but it has, shall we say, "evolved". It would benefit from a rethink and a rewrite before being opened up to the world. So this is that rewrite.
For now at least, during development, it is called spud
instead of
stack-fetcher
.
You'll need a working modern Ruby, and bundler.
bundle install
rspec
gem build spud.gemspec
gem install *.gem
First, install spud. Also, ensure your environment has AWS credentials,
and (if required) $https_proxy
.
In this example, we'll be working with a single stack, labelled the "main" stack. This is referred to as the "stack type".
Starting in the spud
source directory,
cd examples/single-stack
cp src/main/template.1-topic-only.json src/main/template.default.json
spud prepare
You will be prompted for the name and the region of the "main" stack. Enter a valid stack name, e.g. "MySpudTestStack", and pick a region (e.g. eu-west-1).
The output will then look a bit like this (don't worry if it's not absolutely identical):
Retrieving stack "MySpudTestStack"
Generating target stacks
Normalising
main : NEW
You should now edit the "next" files to suit, for example using the following
commands:
vimdiff tmp/templates/template-main.{current,generated,next}.json ; vim tmp/templates/description-main.next.json
then run spud "apply".
For each stack, spud
always works with three sets of templates:
- "current": whatever is actually in AWS CloudFormation
- "generated": whatever your source code says you want the stack to look like
- "next": what is about to be pushed to AWS CloudFormation, if you run
spud apply
For a new stack, "current" will basically be blank, since there is no existing stack.
After running spud prepare
, "next" will always be exactly the same as
"current" – i.e. by default, nothing will be changed, unless you ask for it.
The next step is to change the "next" file to include whatever changes you
want to be in CloudFormation – in this case, we can simply copy the
"generated" file over before running spud apply
:
cp tmp/templates/template-main.generated.json tmp/templates/template-main.next.json
spud apply
You will be asked for confirmation that you want to create the new stack.
Enter "Y", and the stack should be created. As CloudFormation does its work,
spud
shows the stack events, and waits for the creation to finish:
2017-06-30T10:21:21Z AWS::CloudFormation::Stack CREATE_IN_PROGRESS MySpudTestStack arn:aws:cloudformation:eu-west-1:123456789012:stack/MySpudTestStack/dcd262c0-5d7d-11e7-9d9e-50a686335cd2 User Initiated
2017-06-30T10:21:26Z AWS::SNS::Topic CREATE_IN_PROGRESS MyTopic
2017-06-30T10:21:27Z AWS::SNS::Topic CREATE_IN_PROGRESS MyTopic arn:aws:sns:eu-west-1:123456789012:MySpudTestStack-MyTopic-REMC59UM6J8V Resource creation Initiated
2017-06-30T10:21:38Z AWS::SNS::Topic CREATE_COMPLETE MyTopic arn:aws:sns:eu-west-1:123456789012:MySpudTestStack-MyTopic-REMC59UM6J8V
2017-06-30T10:21:40Z AWS::CloudFormation::Stack CREATE_COMPLETE MySpudTestStack arn:aws:cloudformation:eu-west-1:123456789012:stack/MySpudTestStack/dcd262c0-5d7d-11e7-9d9e-50a686335cd2
Congratulations! You've now created your first stack with spud.
If you now re-run spud prepare
, you'll notice that (a) you aren't asked for
the stack name, and (b) the summary now shows "same", whereas before it said
"NEW".
Now let's do an update.
First, let's give spud
a new template, then re-run prepare
:
cp ./src/main/template.2-topic-and-queue.json ./src/main/template.default.json
spud prepare
This time, the summary shows that the "main" stack has differences (i.e. that "current" and "generated" are different):
main : DIFFERENT (hunks: 1, lines: 3)
You can use whatever tool you like for viewing and resolving these
differences, but as an example, you can view the change with diff
:
diff -u tmp/templates/template-main.current.json tmp/templates/template-main.generated.json
and, assuming you have "vim" installed, you can use vimdiff
to look compare
current / generated / next, and (ideally) make "next" be the same as
"generated".
For now let's just copy the generated file over, before running "apply" (note that these are exactly the same commands we used earlier):
cp tmp/templates/template-main.generated.json tmp/templates/template-main.next.json
spud apply
This time, spud
shows the changes we're about to apply, before asking if we want to proceed:
--- tmp/templates/template-main.current.json 2017-06-30 11:27:50.000000000 +0100
+++ tmp/templates/template-main.next.json 2017-06-30 11:31:49.000000000 +0100
@@ -2,6 +2,9 @@
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Trivial example stack",
"Resources": {
+ "MyQueue": {
+ "Type": "AWS::SQS::Queue"
+ },
"MyTopic": {
"Type": "AWS::SNS::Topic"
}
<then a CloudFormation change set is prepared, and displayed>...
INFO: Template has no parameters
Update the main stack MySpudTestStack? [y/N]:
Again, if we say yes, then spud applies the change for us, showing us the stack events:
Update the main stack MySpudTestStack? [y/N]: y
Pushing stack arn:aws:cloudformation:eu-west-1:123456789012:stack/MySpudTestStack/dcd262c0-5d7d-11e7-9d9e-50a686335cd2 using update_stack via region "eu-west-1"
2017-06-30T10:33:15Z AWS::CloudFormation::Stack UPDATE_IN_PROGRESS MySpudTestStack arn:aws:cloudformation:eu-west-1:123456789012:stack/MySpudTestStack/dcd262c0-5d7d-11e7-9d9e-50a686335cd2 User Initiated
2017-06-30T10:33:20Z AWS::SQS::Queue CREATE_IN_PROGRESS MyQueue
2017-06-30T10:33:21Z AWS::SQS::Queue CREATE_IN_PROGRESS MyQueue https://sqs.eu-west-1.amazonaws.com/123456789012/MySpudTestStack-MyQueue-1QL7IAM2WMJTY Resource creation Initiated
2017-06-30T10:33:21Z AWS::SQS::Queue CREATE_COMPLETE MyQueue https://sqs.eu-west-1.amazonaws.com/123456789012/MySpudTestStack-MyQueue-1QL7IAM2WMJTY
2017-06-30T10:33:24Z AWS::CloudFormation::Stack UPDATE_COMPLETE_CLEANUP_IN_PROGRESS MySpudTestStack arn:aws:cloudformation:eu-west-1:123456789012:stack/MySpudTestStack/dcd262c0-5d7d-11e7-9d9e-50a686335cd2
2017-06-30T10:33:25Z AWS::CloudFormation::Stack UPDATE_COMPLETE MySpudTestStack arn:aws:cloudformation:eu-west-1:123456789012:stack/MySpudTestStack/dcd262c0-5d7d-11e7-9d9e-50a686335cd2
So the basic pattern is:
- make
src/*/template.default.json
contain the desired stack template - "spud prepare"
- Use vimdiff or whatever to resolve the differences between current / generated / next, making "next" contain whatever you want to push to CloudFormation
- "spud apply"
In the walkthrough above, we used a single stack, referred to as the "main" stack. If you need to work with more stacks, just create more directories in parallel. For example, here's a "blue" and a "green" stack:
./src/blue/template.default.json
./src/green/template.default.json
spud
will process all of the stacks. spud
always asks before making any
changes to CloudFormation.
stack_names.json
includes the region of each stack:
{
"default": {
"main": {
"account_alias": null,
"region": "eu-west-1",
"stack_name": "MySpudTestStack"
}
}
}
spud
switches region to deal with each stack as required.
There is a space in stack_names.json
for the account alias associated with
each stack, but this has no effect, unless you extend spud (see below).
spud is designed to be extended in a few specific areas, referred to as the "extensions":
- the generator is responsible for generating the desired stack template JSON
- the stack name suggester is responsible for suggesting names for new stacks
- the puller is responsible for retrieving the existing stack details from AWS CloudFormation
- the pusher is responsible for applying the "next" stack details to AWS CloudFormation
The intended way of extending spud is to create your own executable (e.g.
my-custom-spud
) which would construct a Spud::Context
object, override the
default extentions, then run Spud::Main.new(args, context).run
.
# You don't have to override everything!
context = Spud::Context.new
context.extensions.stack_name_suggester = MyCustomSpud::NameSuggester.new
context.extensions.generator = MyCustomSpud::Generator.new
context.extensions.puller = MyCustomSpud::Puller.new
context.extensions.pusher = MyCustomSpud::Pusher.new
Spud::Main.new(ARGV, context).run
The extensions have to implement the same interface as the default, of course;
see lib/spud/ext/default*
for a guide.
What it does:
- Handle creation of new stacks
- Handle updates to existing stacks
Changes I'm considering:
- Delegate finding the list of stack types to an extension
- This would allow much more easily for cases where an entire stack is not required in some contexts (e.g. a "test support" stack is not required in live)
- Default implementation: effectively
ls src/
- Delegate the current function of the
stack_names.json
file to an extension- This would allow the site-local usage to be as simple or complex as required, e.g. store information about how to acquire credentials / how to select the region for each stack type
- This would render the
--config-set=KEY
option redundant - Default implementation: exactly as
stack_names.json
currently is in the core
What it doesn't do, and probably never will:
- Handle deletion of stacks
- Any knowledge of how to acquire AWS credentials. To handle this, extend spud, overriding the puller and pusher.