This is an open-source tagging solution for AWS. Deploy AutoTag to Lambda using CloudTrail consumed through CloudWatch Events and have each of your resources tagged with the ARN of who created it. Optionally, resources can be tagged with when it was created and which AWS service invoked the request if one is provided. It was written by GorillaStack.
Read a blog post about the project.
Also see retro-tag for a solution to retrospectively tagging your resources using CloudTrail data.
Automatically tagging resources can greatly improve the ease of cost allocation and governance.
CloudWatch events delivers a near real-time stream of CloudTrail events as soon as a supported resource type is created. CloudWatch event rules triggers our AutoTag code to tag the resource. In this configuration the Lambda function is executed once each time it is triggered by the CloudWatch Event Rule (one event at a time). The CloudWatch Event Rule includes a pattern filter so it is only triggered by the supported events, meaning fewer Lambda invocations and lower operational costs.
The infrastructure consists of:
- S3 Bucket
- Main CloudFormation Stack (1 AWS region)
- Lambda Function
- IAM Role
- Collector CloudFormation Stack (All active AWS regions)
- CloudWatch Events Rule
- SNS Topic
You will need at least 1 AWS Account, and CloudTrail should be enabled.
We have documented two different ways to deploy the infrastructure to an AWS account. Since there are CloudFormation stacks that need to be deployed in multiple regions we've provided a script that uses the AWS CLI to deploy everything for you. The other deployment method has more steps and uses CloudFormation StackSets to deploy across multiple regions.
This deploy script deploy_autotag.sh will create, delete, or update all of the AutoTag infrastructure for a single AWS account.
The script will attempt to auto-install its own dependencies: aws-cli, jq, npm, git, zip
The create command will start by creating a dedicated AutoTag S3 Bucket for storing code deployment packages in your AWS account. Then it will download or build the code package, and create both the main CloudFormation stack and the collector CloudFormation stacks. When executing the delete command all resources will be removed except the S3 bucket.
The deploy script can use all of the credential providers that the AWS CLI allows, see Configure AWS CLI and take a look at the examples below. A separate set of CLI credentials can be provided by the argument --s3-profile for utilizing a single S3 bucket when deploying infrastructure across multiple AWS accounts. The script will also secure the S3 bucket by blocking all public access configuration, and add the required S3 bucket-policy statement to allow the cross-account GetObject access if necessary.
The script needs at minimum the IAM permissions described in this policy: deploy_iam_policy.json
Before using this IAM policy replace the 2 occurrences of my-autotag-bucket with the name of your actual AutoTag S3 bucket.
Usage: deploy_autotag.sh [options] <command>
Commands:
create Create the AutoTag infrastructure
delete Delete the AutoTag infrastructure
update-release Update the AutoTag infrastructure with a specific release version
update-master Update the AutoTag infrastructure with the latest from the master branch
update-local Update the AutoTag infrastructure with the local source code
Options:
-h --help Show this screen
-r --region The primary AWS region where the main CloudFormation stack will be deployed
-p --profile The main AWS credential profile
-s3bu --s3-bucket The S3 bucket where the code package will be uploaded
-s3pr --s3-profile A separate AWS credential profile to upload code packages to the S3 Bucket
-rv --release-version The release version to deploy, e.g. '0.5.2' or 'latest'
-lr --log-retention-days The number of days to retain the Lambda Function's logs (default: 90)
-ld --log-level-debug Enable the debug logging for the Lambda Function
-ct --disable-create-time Disable the 'CreateTime' tagging for all AWS resources
-ib --disable-invoked-by Disable the 'InvokedBy' tagging for all AWS resources
Follow these steps to prepare to run the create command.
- Select a primary AWS
--regionfor the S3 bucket and the Main CloudFormation stack - Pick a dedicated AutoTag
--s3-bucketname, e.g. 'acme-autotag' - Configure AWS credentials for the AWS CLI, see Configure AWS CLI
Download the latest version of deploy_autotag.sh, or find it in the root of the repository.
curl -LO https://raw.githubusercontent.com/GorillaStack/auto-tag/master/deploy_autotag.sh
chmod +x deploy_autotag.sh
Create the infrastructure with the latest release using either the default, $AWS_PROFILE, or instance AWS credentials profile.
./deploy_autotag.sh --region us-west-2 --s3-bucket my-autotag-bucket --release-version latest createCreate the infrastructure with the latest release using a named AWS credentials profile.
./deploy_autotag.sh --region us-west-2 --s3-bucket my-autotag-bucket --release-version latest --profile dev-acct createCreate the infrastructure using $AWS_ACCESS_KEY_ID and $AWS_SECRET_ACCESS_KEY.
export AWS_ACCESS_KEY_ID=XXX
export AWS_SECRET_ACCESS_KEY=YYY
./deploy_autotag.sh --region us-west-2 --s3-bucket my-autotag-bucket createCreate the infrastructure using a named AWS credentials profile (--profile), but with the S3 Bucket operations utilizing a separate AWS credential profile (--s3-profile). Use this feature to deploy across multiple accounts using a single S3 bucket.
./deploy_autotag.sh --region us-west-2 --s3-bucket my-autotag-bucket --profile dev-acct --s3-profile s3-acct createUpdate the infrastructure to the bleeding edge (master).
./deploy_autotag.sh --region us-west-2 --s3-bucket my-autotag-bucket update-masterUpdate the infrastructure to the latest git release.
./deploy_autotag.sh --region us-west-2 --s3-bucket my-autotag-bucket --release-version latest update-releaseUpdate the infrastructure to a specific git release - only works for releases >= 0.5.1.
./deploy_autotag.sh --region us-west-2 --s3-bucket my-autotag-bucket --release-version 0.5.2 update-releaseUpdate the infrastructure to the local git folder's current state.
git clone https://github.com/GorillaStack/auto-tag.git
cd auto-tag
./deploy_autotag.sh --region us-west-2 --s3-bucket my-autotag-bucket update-localDelete the infrastructure.
./deploy_autotag.sh --region us-west-2 deleteCloudFormation StackSet Deployment Method
Currently Auto-Tag, supports the following AWS resource types:
Tags Applied: C=Creator, T=Create Time, I=Invoked By
| Technology | Event Name | Tags Applied | IAM Deny Tag Support |
|---|---|---|---|
| AutoScaling Group | CreateAutoScalingGroup | C, T, I | Yes |
| ASG Instances w/ENI & Vol | RunInstances | C, T, I | Yes |
| Data Pipeline | CreatePipeline | C, T, I | No |
| DynamoDB Table | CreateTable | C, T, I | No |
| CloudWatch Alarm ? | PutMetricAlarm | C, T, I | ? |
| CloudWatch Events Rule ? | PutRule | C, T, I | ? |
| CloudWatch Log Group ? | CreateLogGroup | C, T, I | ? |
| EBS Volume | CreateVolume | C, T, I | Yes |
| EC2 AMI w/Snapshot * | CreateImage | C, T, I | Yes |
| EC2 AMI w/Snapshot * | CopyImage | C, T, I | Yes |
| EC2 AMI * | RegisterImage | C, T, I | Yes |
| EC2 Customer Gateway ? | CreateCustomerGateway | C, T, I | ? |
| EC2 DHCP Options ? | CreateDhcpOptions | C, T, I | ? |
| EC2 Elastic IP | AllocateAddress | C, T, I | Yes |
| EC2 ENI | CreateNetworkInterface | C, T, I | Yes |
| EC2 Instance w/ENI & Volume | RunInstances | C, T, I | Yes |
| EC2 / VPC Security Group | CreateSecurityGroup | C, T, I | Yes |
| EC2 Snapshot * | CreateSnapshot | C, T, I | Yes |
| EC2 Snapshot * | CopySnapshot | C, T, I | Yes |
| EC2 Snapshot * | ImportSnapshot | C, T, I | Yes |
| Elastic LB (v1 & v2) | CreateLoadBalancer | C, T, I | No |
| EMR Cluster | RunJobFlow | C, T, I | No |
| IAM Role | CreateRole | C, T, I | ? |
| IAM User | CreateUser | C, T, I | ? |
| Lambda Function ? | CreateFunction20150331 | C, T, I | ? |
| Lambda Function ? | CreateFunction20141111 | C, T, I | ? |
| OpsWorks Stack | CreateStack | C | No |
| OpsWorks Clone Stack * | CloneStack | C | No |
| OpsWorks Instances w/ENI & Vol | RunInstances | C, T, I | Yes |
| RDS Instance | CreateDBInstance | C, T, I | No |
| S3 Bucket | CreateBucket | C, T, I | No |
| NAT Gateway | CreateNatGateway | C, T, I | Yes |
| VPC | CreateVpc | C, T, I | Yes |
| VPC Internet Gateway | CreateInternetGateway | C, T, I | Yes |
| VPC Network ACL | CreateNetworkAcl | C, T, I | Yes |
| VPC Peering Connection | CreateVpcPeeringConnection | C, T, I | Yes |
| VPC Route Table | CreateRouteTable | C, T, I | Yes |
| VPC Subnet | CreateSubnet | C, T, I | Yes |
| VPN Connection | CreateVpnConnection | C, T, I | Yes |
| VPN Gateway ? | CreateVpnGateway | C, T, I | ? |
*=not tested by the test suite
NOTE: When tag-able resources are created using CloudFormation StackSets the "Creator" tag is NEVER populated with the ARN of the user who executed the StackSet, instead it is tagged with the less useful CloudFormation StackSet Execution Role's "assumed-role" ARN.
Use the following IAM policy to deny a user or role the ability to create, delete, and edit any tag starting with 'AutoTag_'. The ec2:CreateAction condition allows users to create EC2 instances with tags starting with 'AutoTag_', this enables the 'Launch More Like This' feature, in that case the tags will be overwritten after the instance is created.
{
"Sid": "DenyAutoTagPrefix",
"Effect": "Deny",
"Action": [
"ec2:CreateTags",
"ec2:DeleteTags",
"autoscaling:CreateOrUpdateTags",
"autoscaling:DeleteTags"
],
"Condition": {
"ForAnyValue:StringLike": {
"aws:TagKeys": "AutoTag_*"
},
"StringNotEquals": {
"ec2:CreateAction": [
"RunInstances"
]
}
},
"Resource": "*"
}NOTE: At the time of this writing the deny tag IAM condition (aws:TagKeys) is only available for resources in EC2 and AutoScaling, see the table above for a status of each resource.
If you have questions, feature requests or bugs to report, please do so on the issues section of our github repository.
If you are interested in contributing, please get started by forking our GitHub repository and submit pull-requests.