Deploying FastAPI Microservices on AWS Elastic Kubernetes Service (EKS) using Terraform

This repository contains all necessary configurations and guidelines for deploying a bookstore application built with FastAPI on AWS Elastic Kubernetes Service (EKS) using Terraform. This project demonstrates setting up a robust Kubernetes cluster and deploying microservices that manage books and client data, utilizing an API Gateway for seamless service interaction.

Prerequisites

Before you start, make sure you have installed and configured the following tools:

• AWS CLI - Command-line tool for managing AWS resources.
• Visual Studio Code or PyCharm with the Terraform extension.
• Terraform CLI - Installed and added to your system's PATH.
• Kubectl - Kubernetes command-line tool, also added to your system's PATH.
• An AWS account with an active subscription / Free tier.

Repository Structure

• TerraformScriptsEKS: Contains Terraform scripts for provisioning the EKS cluster.
• FastAPIApps: Contains the FastAPI applications along with Dockerfiles and requirements.txt for each microservice.
• assets: This folder contains a complete list of commands used (commands.txt) and a PDF with step-by-step screenshots detailing the entire process.
• deployments: Kubernetes YAML configurations for deploying the microservices.

Project Setup

Step 1: Terraform Setup for EKS

1. Environment Setup:

   • Install all required tools and ensure they are correctly configured in your system's PATH.

2. Terraform Initialization and Application:

   • Navigate to the TerraformScriptsEKS directory.
   • Initialize Terraform:

     terraform init -upgrade

   • Create an execution plan:

     terraform plan -out main.tfplan

   • Apply the configuration to provision the EKS cluster:

     terraform apply main.tfplan

3. Connect to the EKS Cluster:

   • Update kubeconfig to interact with your cluster:

     aws eks --region us-west-2 update-kubeconfig --name <cluster-name>

Step 2: FastAPI Microservices Development

1. Build and Push Docker Images:

   • Navigate to each FastAPI application directory.
   • Build and push the Docker images:

     docker build . -t <your-dockerhub-username>/<app-name>:<tag> .
     docker push <your-dockerhub-username>/<app-name>:<tag>

2. Deploy Microservices:

   • Apply the Kubernetes deployment and PVC configurations:

     kubectl apply -f books-data-pvc.yml
     kubectl apply -f books-deployment.yml
     kubectl apply -f clients-data-pvc.yml
     kubectl apply -f clients-deployment.yml

Step 3: AWS EFS Integration with Kubernetes

1. Install the AWS EFS CSI Driver:
   • Use Helm to install the AWS EFS CSI driver which enables Kubernetes to manage EFS file systems efficiently:

     helm repo add aws-efs-csi-driver https://kubernetes-sigs.github.io/aws-efs-csi-driver/
     helm repo update
     helm upgrade --install aws-efs-csi-driver --namespace kube-system aws-efs-csi-driver/aws-efs-csi-driver

Step 4: ConfigMap Configuration

1. Setup ConfigMaps:
   • Create a directory for configuration files:

     New-Item -ItemType Directory -Path .\gateway-conf

   • Copy the configuration file into the new directory:

     Copy-Item -Path .\gateway\app\conf.d\main.yml -Destination .\gateway-conf\

   • Create a ConfigMap and deploy the gateway:

     kubectl create cm books-gateway-conf --from-file gateway-conf/
     kubectl create -f gateway-deployment.yml

Step 5: Service Exposure and Load Balancing

1. Internal Exposure Using ClusterIP:

   • Expose each microservice internally:

     kubectl expose deploy books-api --port=8000
     kubectl expose deploy clients-api --port=8000

2. External Exposure Using LoadBalancer:

   • Provide external access through a LoadBalancer:

     kubectl expose deploy gateway-api --port=8000 --type=LoadBalancer

   • Access the services externally using the provided IP:

     http://<external-ip>:8000/docs
     

Step 6: Implementing NGINX Ingress Controller and Ingress Rules

• Installing NGINX Ingress Controller

  • Using Helm to install:

    helm install my-nginx bitnami/nginx-ingress-controller --set rbac.create=true

• Configuring and Applying Ingress Rules

  • Apply limits to prevent abuse:

    kubectl apply -f ingress-limiter.yml

• Monitoring and Troubleshooting

  • Commands to monitor and troubleshoot ingress setup:

    kubectl get ingress
    kubectl logs -n <namespace> -l app=nginx-ingress --tail=100

Additional Resources

• For a complete list of commands used and step-by-step guidance, check the assets folder where you can find the commands.txt and a detailed PDF.