Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial AWS Org CDK structure and CI/CD #3

Merged
merged 4 commits into from
Sep 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/prerequisites/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# AWS PREREQUISITES FOR THE GITHUB ACTIONS CI/CD PIPELINE

Inspired on:

- https://github.com/aws-actions/configure-aws-credentials/tree/main/examples

## Part 1 (CDK Bootstrap)

Make sure that the target account (on the needed regions) have CDK Bootstrapped already. If not, please follow these steps to have CDK init stack already in place:

- [AWS CDK Bootstrapping Guide](https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping.html)

## Part 2 (GitHub OIDC Federation and Role)

The CI/CD uses aws-action `configure-aws-credentials` with OIDC federation. Prior to using this project, we need to deploy the [github-actions-oidc-federation-and-role](github-actions-oidc-federation-and-role.yml) CloudFormation template in the AWS account(s) that we need to deploy the solution. Specify the GitHub Organization Name, Repository Name, and the specific branch you want to deploy on.

> Note: In this case, "GitHub Organization Name" is "san99tiago" (my GitHub account).

To use the example you will need to set the following GitHub Action Secrets:

| Secret Key | Used With | Description |
| --------------- | ------------------------- | ------------------------ |
| AWS_ACCOUNT_ID | configure-aws-credentials | The AWS account ID |
| AWS_DEPLOY_ROLE | configure-aws-credentials | The name of the IAM role |
92 changes: 92 additions & 0 deletions .github/prerequisites/github-actions-oidc-federation-and-role.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
AWSTemplateFormatVersion: "2010-09-09"
Description: Github Actions configuration - OIDC IAM IdP and associated role CI/CD

Parameters:

GitHubOrganization:
Type: String
Description: This is the root organization or personal account where repos are stored (Case Sensitive)

RepositoryName:
Type: String
Description: The repo(s) these roles will have access to. (Use * for all org or personal repos)
Default: "*"

BranchName:
Type: String
Description: Name of the git branch to to trust. (Use * for all branches)
Default: "*"

RoleName:
Type: String
Description: Name the Role

UseExistingProvider:
Type: String
Description: "Only one GitHub Provider can exists. Choose yes if one is already present in account"
Default: "no"
AllowedValues:
- "yes"
- "no"

Conditions:

CreateProvider: !Equals ["no", !Ref UseExistingProvider]

Resources:

IdpGitHubOidc:
Type: AWS::IAM::OIDCProvider
Condition: CreateProvider
Properties:
Url: https://token.actions.githubusercontent.com
ClientIdList:
- sts.amazonaws.com
- !Sub https://github.com/${GitHubOrganization}/${RepositoryName}
ThumbprintList:
- 6938fd4d98bab03faadb97b34396831e3780aea1
Tags:
- Key: Name
Value: !Sub ${RoleName}-OIDC-Provider

RoleGithubActions:
Type: AWS::IAM::Role
Properties:
RoleName: !Ref RoleName
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Action: sts:AssumeRoleWithWebIdentity
Principal:
Federated: !If
- CreateProvider
- !Ref IdpGitHubOidc
- !Sub arn:${AWS::Partition}:iam::${AWS::AccountId}:oidc-provider/token.actions.githubusercontent.com
Condition:
StringLike:
token.actions.githubusercontent.com:sub: !Sub repo:${GitHubOrganization}/${RepositoryName}:ref:refs/heads/${BranchName}

RoleGithubActionsPolicies:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: !Sub ${RoleName}-Policy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action: "sts:AssumeRole"
Resource: "arn:aws:iam::*:role/cdk-*"
Roles:
- !Ref RoleGithubActions

Outputs:

IdpGitHubOidc:
Condition: CreateProvider
Description: "ARN of Github OIDC Provider"
Value: !GetAtt IdpGitHubOidc.Arn

RoleGithubActionsARN:
Description: "CICD Role for GitHub Actions"
Value: !GetAtt RoleGithubActions.Arn
198 changes: 198 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
name: deploy

on:
push:
branches: [ 'main', 'feature/**']
tags: [ 'v*']
env:
AWS_DEFAULT_REGION: us-east-1
AWS_DEFAULT_OUTPUT: json

jobs:
code-quality:
name: Check coding standards
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: echo "Job triggered by ${{ github.event_name }} event."
- run: echo "Job running on a ${{ runner.os }} server hosted by GitHub."
- run: echo "Branch name is ${{ github.ref }} and repository is ${{ github.repository }}."
- uses: actions/setup-python@v4
with:
python-version: 3.11
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true
- name: Install Poetry dependencies
run: poetry install --no-interaction
- name: Check code formatting
run: poetry run poe black-check

tests:
name: Run tests
needs: code-quality
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- uses: actions/setup-python@v4
with:
python-version: 3.11
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true

- name: Install Poetry dependencies
run: poetry install --no-interaction

- name: Run tests
run: poetry run poe test-unit

- name: Archive code coverage results
uses: actions/upload-artifact@v3
with:
name: code-coverage-report
path: htmlcov

cdk-synth-diff:
name: CDK Synth & Diff
runs-on: ubuntu-latest
needs: code-quality
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.11

- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true

- name: Install Poetry dependencies
run: poetry install --no-interaction

- name: Set up NodeJs
uses: actions/setup-node@v3
with:
node-version: "20"

- name: Install CDK
run: |
npm install -g aws-cdk

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: ${{ env.AWS_DEFAULT_REGION }}
role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/${{ secrets.AWS_DEPLOY_ROLE }}
role-session-name: myGitHubActions

# Sample STS get caller identity for tests
- name: sts get-caller-identity
run: |
aws sts get-caller-identity

- name: CDK Synth
run: |
source .venv/bin/activate
cdk synth

- name: CDK Diff
run: |
source .venv/bin/activate
cdk diff

- name: Archive CDK Synth results (no assets)
uses: actions/upload-artifact@v3
with:
name: cdk-synth-folder
path: |
./cdk.out
!./cdk.out/asset.*
retention-days: 1

iac-checkov:
name: IaC Checkov Validations
runs-on: ubuntu-latest
needs: cdk-synth-diff
steps:
- uses: actions/checkout@v3

- name: Dowload CDK Synth results
uses: actions/download-artifact@v3
with:
name: cdk-synth-folder
path: ./cdk-synth-output-folder

- name: Display files in the output folder
run: ls -lrta
working-directory: ./cdk-synth-output-folder

- name: Run Checkov action
id: checkov
uses: bridgecrewio/checkov-action@v12
with:
directory: cdk-synth-output-folder/
framework: cloudformation
soft_fail: true # optional: do not return an error code if there are failed checks
skip_check: CKV_AWS_2 # optional: skip a specific check_id. can be comma separated list
quiet: true # optional: display only failed checks

cdk-deploy:
name: Deploy CDK
runs-on: ubuntu-latest
needs:
- iac-checkov
- tests
if: github.ref == 'refs/heads/main'
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.11

- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true

- name: Install Poetry dependencies
run: poetry install --no-interaction

- name: Set up NodeJs
uses: actions/setup-node@v3
with:
node-version: "20"

- name: Install CDK
run: npm install -g aws-cdk

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: ${{ env.AWS_DEFAULT_REGION }}
role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/${{ secrets.AWS_DEPLOY_ROLE }}
role-session-name: myGitHubActions

# NOTE: for now no manual approvals are required
- name: Deploy to AWS
run: |
source .venv/bin/activate
cdk deploy --require-approval=never
35 changes: 33 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,42 @@
# :bank: AWS-CDK-ORGANIZATIONS-DEMO :bank:

![Badge Workflow](https://github.com/san99tiago/aws-cdk-organizations-demo/actions/workflows/deploy.yml/badge.svg)

DEMO for the best practices of AWS Organizations with Infrastructure as Code on CDK-Python.

## TODO:

Add a detailed README with diagram, explanation and examples of usage.
Add a detailed README with diagrams, explanations and examples of usage.

## CI/CD and Deployment 🚀

The deployment process is intended to run with GitHub Actions Workflows.

- On `feature/****` branches commits, the CDK project gets **synthesized** and it shows the **state diff** between the current AWS resources and the expected ones.

- When merged to `main` branch, it will get deployed to the AWS Account automatically.

To understand the AWS Credentials usage, please refer to the [`prerequisites/README.md`](.github/prerequisites/README.md).

## Special thanks :gift:

- Thanks to all contributors of the great OpenSource projects that I am using. <br>

## Author :musical_keyboard:

### Santiago Garcia Arango

<table border="1">
<tr>
<td>
<p align="center">Curious DevOps Engineer passionate about advanced cloud-based solutions and deployments in AWS. I am convinced that today's greatest challenges must be solved by people that love what they do.</p>
</td>
<td>
<p align="center"><img src="assets/SantiagoGarciaArangoCDK.png" width=80%></p>
</td>
</tr>
</table>

## LICENSE

Copyright 2023 Santiago Garcia Arango
Copyright 2023 Santiago Garcia Arango.
Binary file added assets/SantiagoGarciaArangoCDK.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 8 additions & 0 deletions cdk.context.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"main_resources_name": "san99tiago-demo-organization",
"tags": {
"Owner": "Santiago Garcia Arango",
"Source": "https://github.com/san99tiago/aws-cdk-organizations-demo",
"Usage": "Configuration of AWS Organizations for san99tiago-demo AWS accounts setup"
}
}
3 changes: 3 additions & 0 deletions cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"app": "python3 cdk/app.py"
}
10 changes: 10 additions & 0 deletions cdk/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
*.swp
package-lock.json
__pycache__
.pytest_cache
.venv
*.egg-info

# CDK asset staging directory
.cdk.staging
cdk.out
Loading
Loading