This repository encompasses a web application backend utilizing Python 3.8 and MySQL, performing a series of tasks: health check (GET - /healthz), user creation (POST - /v1/user) with a subsequent verification email sent to the user, retrieval of user details (GET - /v1/user/self), and user details update (PUT - /v1/user/self). Upon user creation, a message is dispatched to GCP Pub/Sub, triggering a serverless function responsible for emailing the user with a unique verification link. Terraform is utilized to configure all necessary resources for the app setup.
Several GitHub workflows are established for Continuous Integration and Continuous Deployment. These workflows activate upon pull request creation, executing functional tests on the webapp backend to detect failures and examining the packer code formatting. Post successful workflow execution and code merging, packer is employed to generate a golden image of the application, configuring all essential libraries/dependencies. Subsequently, the existing instances in the GCP project are replaced with the updated packer image.
Follow this URL to find the installer steps for your machine https://cloud.google.com/sdk/docs/install
Note: Add the bin directory of gcloud-sdk to PATH variable, else you would have to run the commands with "<path-to-bin of gcloud sdk>/" prefix
$ gcloud auth login
$ gcloud auth application-default login
$ gcloud config set project <project_id>
$ gcloud auth revoke
$ gcloud auth application-default revoke
- Navigate to google cloud dashboard: https://console.cloud.google.com/welcome/new
- From the Navigation Menu > APIs and services > Library
- Enable the following APIs:
- Compute Engine API
- Cloud SQL Admin API
- Service Networking API
- Cloud Source Repositories API
- Identity and Access Management (IAM) API
- Cloud Monitoring API
- Cloud Logging API
- Serverless VPC Access API
- Eventarc API
- Cloud Deployment Manager V2 API
- Cloud DNS API
- Cloud Functions API
- Artifact Registry API
- Cloud Pub/Sub API
- Cloud Build API
- Service Usage API
- Secret Manager API
- Certificate Manager API
- Cloud Key Management Service (KMS) API
- After enabling the APIs it may take about 10-15 mins to be activated
- Navigate to google cloud dashboard: https://console.cloud.google.com/welcome/new
- From the Navigation Menu > IAM and admin > Service accounts
- Create a new / modify the permissions of existing service account with the following permissions
Cloud SQL Editor
Compute Instance Admin (v1)
Compute Network Admin
Compute Security Admin
IAP-secured Tunnel User
OSPolicyAssignment Editor
Pub/Sub Publisher
Secret Manager Secret Accessor
Service Account Token Creator
Service Account User
Storage Object Viewer
- Navigate to google cloud dashboard: https://console.cloud.google.com/welcome/new
- From the Navigation Menu > IAM and admin > Service accounts
- Click on the service account to be used
- Under "KEYS" tab, click on "ADD KEY" dropdown and select "Create new key"
# After downloading the JSON key file, run the following
$ export GOOGLE_APPLICATION_CREDENTIALS="<path to key>/<file_name>.json"
Follow this URL to find the installer steps for your machine https://developer.hashicorp.com/packer/install
https://cloud.google.com/compute/docs/images/os-details
- Create a file named *.pkrvars.hcl
- Add the data in the format shown below (expected variables are mentioned in the variables.tf file):
project_id = "<project_id>"
zone = "<project_zone>"
machine_type = "<machine_type>"
ssh_username = "<name_of_the_service_account>"
use_os_login = false or true
source_image_family = "<os_family>"
webapp_version = <webapp_version>
mysql_root_password = <mysql_password to be created for 'root' user>
Note: If the variable file is named *.auto.pkrvars.hcl it will automatically be picked up by packer. The --var-file="*.pkrvars.hcl" is not required to be passed to validate or build the image
Formatting the packer code
$ packer fmt <file_name or .>
Check and validate the packer config to be created created
$ packer validate -var-file="*.pkrvars.hcl" <file_name or .>
Building the image using packer
$ packer build -var-file="*.pkrvars.hcl" <file_name or .>
# Create environment
$ python -m venv <env_name>
# Activating environment
## For mac and linux os users
$ source <env_name>/bin/activate
## For windows users
$ <env_name>/Scripts/activate
$ pip install -r requirements.txt
Note: Update the DEV_HOST, DEV_PORT, PROD_HOST, PROD_PORT based on the requirement
SQLALCHEMY_DATABASE_URI_DEV = "mysql://<USERNAME>:<PASSWORD>@<HOST>:<PORT>/<DBNAME>"
# The following flag can toggle on/off the tracking of inserts, updates, and deletes for models
SQLALCHEMY_TRACK_MODIFICATIONS = False
DEV_HOST = "0.0.0.0"
DEV_PORT = 8080
PROD_HOST = "0.0.0.0"
PROD_PORT = 8080
PYTHON_ENV = "development"
GOOGLE_PROJECT_ID = "<gcp_project_id>"
GOOGLE_TOPIC_NAME = "<pub/sub_topic_name>"
$ python create_database.py
$ flask db init
$ flask db migrate
$ flask db upgrade
# Run the following command to run the tests
$ python -m pytest
# Run the following command to start the server
$ python ./app.py
- Go to repository settings
- Under "Secrets and variables", select "Actions"
- Create "New repository secrets" for the following:
# To run integration tests workflow
DB_PASSWORD: <password of the database> ('root' works in most cases)
DB_USER: <user of the database> ('root' works in most cases)
ENV_FILE: Add the contents of the webapp .env file with the above user and password for databse URI
# To run packer workflows
GCP_CREDENTIALS_JSON: Add the contents of the JSON key file created for the service account
PACKER_CONFIG: Add the content of the *.pkrvars.hcl or *.auto.pkrvars.hcl file