DIGIT Core
PlatformDomainsAcademyDesign SystemFeedback
2.8
2.8
  • ☑️Introducing DIGIT Platform
    • DIGIT - Value Proposition
  • Platform
    • 🔎Overview
      • Principles
      • Architecture
        • Service Architecture
        • Infrastructure Architecture
        • Deployment Architecture
      • Technology
        • API Gateway
        • Open Source Tools
      • Checklists
        • API Checklist
        • Security Checklist
          • Security Guidelines Handbook
          • Security Flow - Exemplar
        • Performance Checklist
        • Deployment Checklist
      • UI Frameworks
        • React UI Framework
    • 🔧Core Services
      • Workflow Service
        • Setting Up Workflows
        • Configuring Workflows For An Entity
        • Workflow Auto Escalation
        • Migration To Workflow 2.0
      • Location Services
      • User Services
      • Access Control Services
      • PDF Generation Service
      • MDMS (Master Data Management Service)
        • Setting up Master Data
          • MDMS Overview
          • MDMS Rewritten
          • Configuring Tenants
          • Configuring Master Data
          • Adding New Master
          • State Level Vs City Level Master
      • Payment Gateway Service
      • User Session Management
      • Indexer Service
        • Indexer Configuration
      • URL Shortening Service
      • XState Core Chatbot
        • Xstate-Chatbot Message Localisation
        • XState-Chatbot Integration Document
      • NLP Engine Service
        • NLP Chatbot
      • SMS Template Approval Process
      • Telemetry Service
      • Document Uploader Service
      • Notification Enhancement Based On Different Channel
      • Report Service
        • Configuring New Reports
          • Impact Of Heavy Reports On Platform
          • Types Of Reports Used In Report Service
      • SMS Notification Service
        • Setting Up SMS Gateway
          • Using The Generic GET & POST SMS Gateway Interface
      • Survey Service
      • Persister Service
        • Persister Configuration
      • Encryption Service
        • Encryption Client Library
        • User Data Security Architecture
        • Guidelines for supporting User Privacy in a module
      • FileStore Service
      • ID Generation Service
      • Localization Service
        • Configuring Localization
          • Setup Base Product Localization
          • Configure SMS and Email
      • Email Notification Service
      • Searcher Service
      • Zuul Service
      • User OTP Service
      • OTP Service
      • Chatbot Service
      • National Dashboard Ingest
        • National Dashboard API Performance Testing Specs and Benchmark
        • National Dashboard: Steps for Index Creation
        • National Dashboard Adaptor Service
          • Deployment of Airflow DAG
          • Trigger Airflow DAG
          • Configure Airflow
          • Insert & Delete Data - Steps
          • Important Links & Credentials
          • Code Structure
          • KT Sessions
          • Pre-requisites For Enabling Adaptor
        • Revenue Maximisation
      • Audit Service
        • Signed Audit Performance Testing Results
      • Service Request
      • Self Contained Service Architecture (HLD)
      • Accelerators
        • Inbox Service
    • ✏️API Specifications
      • User
      • Access Control
      • Employee
      • Location
      • Localisation
      • Encryption
      • Indexer
      • File Store
      • Collection
      • DSS Ingest
      • HRMS
      • National Dashboard Ingest
      • WhatsApp Chatbot
      • Master Data Management
      • ID Generation
      • URL Shortner
      • Workflow Service
      • Workflow v2
      • Document Uploader Service
      • OTP Service
      • Reporting Service
      • PDF Generation Service
      • Payment Gateway Service
    • 🔐Data Protection & Privacy
      • Data Protection & Privacy Definitions
      • Legal Obligations For Privacy - eGov
      • Data Protection & Privacy - Global Best Practices
      • Guidelines
        • Platform Owner Guidelines
        • Implementing Agencies Guidelines
        • Admin Guidelines
        • Program Owner Guidelines
        • Data Security and Data Privacy
      • Data Privacy Policy Templates
        • eGov Data Privacy Policy
        • Implementing Agency Privacy Policy
        • Admin & Program Owner Privacy Policy
        • Supporting Agency Privacy Policy
      • Global Standards For All Roles
    • ▶️Get Started
      • Install DIGIT
      • Access DIGIT
      • Sandbox
      • Training and Certification
        • Training Resources
    • ⚒️Integrations
      • Payment
      • Notification
      • Transaction
      • Verification
      • View
      • Calculation
    • 🛣️Roadmap
    • 🎬Open Events
    • 👩‍💻Source Code
    • 👁️Project Plan
    • 📋Discussion Board
    • 🤝Contribute
  • Guides
    • 📓Installation Guide
      • DIGIT Deployment
      • Quick Setup
        • DIGIT Installation on Azure
        • DIGIT Installation on AWS
      • Production Setup
        • AWS
          • 1. Pre-requisites
          • 2. Understanding EKS
          • 3. Setup AWS Account
          • 4. Provisioning Infra Using Terraform
          • 5. Prepare Deployment Config
          • 6. Deploy DIGIT
          • 7. Bootstrap DIGIT
          • 8. Productionize DIGIT
          • FAQ
        • Azure
          • 1. Azure Pre-requisites
          • 2. Understanding AKS
          • 3. Infra-as-code (Terraform)
        • SDC
          • 1. SDC Pre-requisites
          • 2. Infra-as-code (Kubespray)
          • CI/CD Setup On SDC
        • CI/CD Set Up
          • CI/CD Build Job Pipeline Setup
        • Prepare Helm Release Chart
        • Deployment - Key Concepts
          • Security Practices
          • Readiness & Liveness
          • Resource Requests & Limits
          • Deploying DIGIT Services
          • Deployment Architecture
          • Routing Traffic
          • Backbone Deployment
    • 💽Data Setup Guide
      • User Module
      • Localisation Module
      • Location Module
    • 🚥Design Guide
      • Model Requirements
      • Design Services
      • Design User Interface
      • Checklists
    • ⚒️Developer Guide
      • Pre-requisites Training Resources
      • Backend Developer Guide
        • Section 0: Prep
          • Development Pre-requisites
          • Design Inputs
            • High Level Design
            • Low Level Design
          • Development Environment Setup
        • Section 1: Create Project
          • Generate Project Using API Specs
          • Create Database
          • Configure Application Properties
          • Import Core Models
          • Implement Repository Layer
          • Create Validation & Enrichment Layers
          • Implement Service Layer
          • Build The Web Layer
        • Section 2: Integrate Persister & Kafka
          • Add Kafka Configuration
          • Implement Kafka Producer & Consumer
          • Add Persister Configuration
          • Enable Signed Audit
          • Run Application
        • Section 3: Integrate Microservices
          • Integrate IDGen Service
          • Integrate User Service
          • Add MDMS Configuration
          • Integrate MDMS Service
          • Add Workflow Configuration
          • Integrate Workflow Service
          • Integrate URL Shortener Service
        • Section 4: Integrate Billing & Payment
          • Custom Calculator Service
          • Integrate Calculator Service
          • Payment Back Update
        • Section 5: Other Advanced Integrations
          • Add Indexer Configuration
          • Certificate Generation
        • Section 6: Run Final Application
        • Section 7: Build & Deploy Instructions
        • FAQs
      • Flutter UI Developer Guide
        • Introduction to Flutter
          • Flutter - Key Features
          • Flutter Architecture & Approach
          • Flutter Pre-Requisites
        • Setup Development Environment
          • Flutter Installation & Setup Guide
          • Setup Device Emulators/Simulators
          • Run Application
        • Build User Interfaces
          • Create Form Screen
        • Build Deploy & Publish
          • Build & Deploy Flutter Web Application
          • Generate Android APKs & App Bundles
          • Publishing App Bundle To Play Store
        • State Management With Provider & Bloc
          • Provider State Management
          • BloC State Management
        • Best Practices & Tips
        • Troubleshooting
      • UI Developer Guide
        • DIGIT-UI
        • Android Web View & How To Generate APK
        • DIGIT UI Development Pre-requisites
        • UI Configuration (DevOps)
        • Local Development Setup
        • Run Application
        • Create New Screen In DIGIT-UI
          • Create Screen (FormComposer)
          • Inbox/Search Screen
          • Workflow Component
        • Customisation
          • Integrate External Web Application/UI With DIGIT UI
          • Utility - Pre-Process MDMS Configuration
          • CSS Customisation
        • Citizen Module Setup
          • Sample screenshots
          • Project Structure
          • Install Dependency
          • Import Required Components
          • Write Citizen Module Code
          • Citizen Landing Screen
        • Employee Module Setup
          • Write Employee Module Code
        • Build & Deploy
        • Setup Monitoring Tools
        • FAQs
          • Troubleshoot Using Browser Network Tab
          • Debug Android App Using Chrome Browser
    • 🔄Operations Guide
      • DIGIT - Infra Overview
      • Setup Central Instance Infra
      • Central Monitoring Dashboard Setup
      • Kubernetes
        • RBAC Management
        • DB Dump - Playground
      • Setup Jenkins - Docker way
      • GitOps
        • Git Client installation
        • GitHub organization creation
        • Adding new SSH key to it
        • GitHub repo creation
        • GitHub Team creation
        • Enabling Branch protection:
        • CODEOWNER Reviewers
        • Adding Users to the Git
        • Setting up an OAuth with GitHub
        • Fork (Fork the mdms,config repo with a tenant-specific branch)
      • Working with Kubernetes
        • Installation of Kubectl
      • Containerizing application using Docker
        • Creation of Dockerhub account
      • Infra provisioning using Terraform
        • Installation of Terraform
      • Customization of existing tf templates
      • Cert-Manager
        • Obtaining SSL certificates with the help of cluster-issuer
      • Moving Docker Images
      • Pre and post deployment checklist
      • Multi-tenancy Setup
      • Availability
        • Infrastructure
        • Backbone services
          • Database
          • Kafka
          • Kafka Connect
          • Elastic search
            • ElasticSearch Direct Upgrade
            • Elastic Search Rolling Upgrade
        • Core services
        • DIGIT apps
        • DSS dashboard
      • Observability
        • ES-Curator to clear old logs/indices
        • Monitoring
        • Tracing
        • Jaeger Tracing Setup
        • Logging
        • eGov Monitoring & Alerting Setup
        • eGov Logging Setup
      • Performance
        • What to monitor?
          • Infrastructure
          • Backbone services
          • Core services
        • Identifying bottlenecks
        • Solutions
      • Handling errors
      • Security
      • Reliability and disaster recovery
      • Privacy
      • Skillsets/hiring
      • Incident management processes
      • Kafka Troubleshooting Guide
        • How to clean up Kafka logs
        • How to change or reset consumer offset in Kafka?
      • SRE Rituals
      • FAQs
        • I am unable to login to the citizen or employee portal. The UI shows a spinner.
        • My DSS dashboard is not reflecting accurate numbers? What can I do?
      • Deployment using helm
        • Helm installation:
        • Helm chart creation
        • Helm chart customization
      • How to Dump Elasticsearch Indexes
      • Deploy Nginx-Ingress-Controller
      • Deployment Job Pipeline Setup
      • OAuth2-Proxy Setup
      • Jira Ticket Creation
  • Reference
    • 👉Setup Basics
      • Setup Requirements
        • Tech Enablement Training - Essential Skills and Pre-requisites
        • Tech Enablement Training (eDCR) - Essential Skills and Prerequisites
          • Development Control Rules (Digit-DCR)
          • eDCR Approach Guide
        • DIGIT Rollout Program Governance
        • DevOps Skills Requirements
        • Infra Requirements
        • Team Composition for DIGIT Implementation
        • Infra Best Practices
        • Operational Best Practices
        • Why Kubernetes For DIGIT
      • Supported Clouds
        • Google Cloud
        • Azure
        • AWS
        • VSphere
        • SDC
      • Deployment - Key Concepts
        • Security Practices
        • CI/CD
        • Readiness & Liveness
        • Resource Requests & Limits
      • Understanding ERP Stack
        • ERP Monolithic Architecture
        • ERP Hybrid Architecture
        • ERP Coexistence Architecture
        • APMDP-HYBRID-INFRA ARCHITECTURE
        • eGov SmartCity eGovernance Suite
        • ERP Deployment Process
        • ERP Release Process
        • ERP User Guide
      • Deploying DIGIT Services
        • Deployment Architecture
        • Routing Traffic
        • Backbone Deployment
      • Troubleshooting
        • Distributed Tracing
        • Logging
        • Monitoring & Alerts
    • 📥Reference Reads
      • Analytics
      • DevSecOps
      • Low Code No Code
        • Application Specification
      • Beneficiary Eligibility
      • Government and Open Digital Platforms
      • Microservices and Low Code No Code
      • Registries
      • Platform Orientation - Overview
    • 🔏Data Security
      • Signed Data Audit
      • Encryption Techniques
      • Approaches to handle Encrypted Data
    • ❕Privacy
    • 🕹️DevOps
      • 1. How DNS works
      • 2. Load Balancer
      • 3. SSL/Cert-manager
      • 4.Ingress,WAF
      • 5.VPC
      • 6.Subnets
      • 7.EKS
      • 8.Worker Node Group
      • 9.RDS
      • 10.NAT
      • 11.Internet Gateway
      • 12.Block Storage (EBS Volumes)
      • 13.Object Storage (S3)
      • 14. Telemetry
Powered by GitBook

All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

On this page
  • Overview
  • Pre-requisites
  • 1. Terraform State Setup
  • 2. Network Infrastructure Setup
  • 3. Custom Variables/Configurations
  • Terraform Execution: Infrastructure Resources Provisioning
  • Destroying Infra

Was this helpful?

  1. Guides
  2. Installation Guide
  3. Production Setup
  4. AWS

4. Provisioning Infra Using Terraform

There are multiple options available to deploy the solution to the cloud. Here we provide the steps to use terraform Infra-as-code.

Previous3. Setup AWS AccountNext5. Prepare Deployment Config

Last updated 1 year ago

Was this helpful?

Topics covered:

Overview

Before we provision the cloud resources, we need to understand and be sure about what resources need to be provisioned by terraform to deploy DIGIT. The following picture shows the various key components. (EKS, Worker Nodes, PostGres DB, EBS Volumes, Load Balancer).

EKS Architecture for DIGIT Setup

Considering the above deployment architecture, the following is the resource graph that we are going to provision using terraform in a standard way so that every time and for every env, it'll have the same infra.

  • EKS Control Plane (Kubernetes Master)

  • Work node group (VMs with the estimated number of vCPUs, Memory)

  • EBS Volumes (Persistent Volumes)

  • RDS (PostGres)

  • VPCs (Private network)

  • Users to access, deploy and read-only

Pre-requisites

  1. (Optional) Create your own keybase key before you run the terraform

  2. Fork the DIGIT-DevOps repository into your organization account using the GitHub web portal. Make sure to add the right users to the repository. Clone the forked DIGIT-DevOps repository (not the egovernments one). Navigate to the sample-aws directory which contains the sample AWS infra provisioning script.

git clone --branch release-1.28-kubernetes https://github.com/egovernments/DIGIT-DevOps.git

cd DIGIT-DevOps/infra-as-code/terraform/sample-aws

The sample-aws terraform script is provided as a helper/guide. An experienced DevOps can choose to modify or customize this as per the organization's infra needs.

Create Terraform backend to specify the location of the backend Terraform state file on S3 and the DynamoDB table used for the state file locking. This step is optional. S3 buckets have to be created outside of the Terraform script.

The remote state is simply storing that state file remotely, rather than on your local filesystem. In an enterprise project and/or if Terraform is used by a team, it is recommended to set up and use the remote state.

cd DIGIT-DevOps/Infra-as-code/terraform/sample-aws/remote-state
provider "aws" {
  region = "ap-south-1"
}

#This is a bucket name that you can name as you wish
resource "aws_s3_bucket" "terraform_state" {
  bucket = "try-digit-eks-yourname" 

  versioning {
    enabled = true
  }

  lifecycle {
    prevent_destroy = true
  }
}

#This is a bucket name that you can name as you wish
resource "aws_dynamodb_table" "terraform_state_lock" {
  name           = "try-digit-eks-yourname"
  read_capacity  = 1
  write_capacity = 1
  hash_key       = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }

}

2. Network Infrastructure Setup

The terraform script once executed performs all of the below infrastructure setups.

Amazon EKS requires subnets must be in at least two different availability zones.

  1. Create AWS VPC (Virtual Private Cloud).

  2. Create two public and two private Subnets in different availability zones.

  3. Create an Internet Gateway to provide internet access for services within VPC.

  4. Create NAT Gateway in public subnets. It is used in private subnets to allow services to connect to the internet.

  5. Create Routing Tables and associate subnets with them. Add required routing rules.

  6. Create Security Groups and associate subnets with them. Add required routing rules.

  7. EKS cluster setup

3. Custom Variables/Configurations

Navigate to the directory: DIGIT-DevOps/Infra-as-code/terraform/sample-aws. Configurations are defined in variables.tf and provide the environment-specific cloud requirements.

Following are the values that you need to replace in the following files. The blank ones will be prompted for inputs while execution.

  1. cluster_name - provide your EKS cluster name here.

  2. availability_zones - This is a comma-separated list. If you would like your infra to have multi-AZ setup, please provide multiple zones here. If you provide a single zone, all infra will be provisioned within that zone. For example:

// Single AZ (default)
variable "availability_zones" {
  default = ["ap-south-1b"]
}
// Multiple AZ
variable "availability_zones" {
  default = ["ap-south-1b", "ap-south-1a"]
}

3. bucket_name - if you've created a special S3 bucket to store Terraform state.

4. dbname - Any DB name of your choice. Note that this CANNOT have hyphens or other special characters. Underscore is permitted. Example: digit_test

All other variables are default and can be modified if the admin is knowledgeable about it.

5. In the providers.tf file in the same directory, modify the "profile" variable to point to the AWS profile that was created in Step 3.

  1. Make sure your AWS session tokens are up to date in the /user/home/.aws/credentials file

  2. Before running Terraform, make sure to clean up .terraform.lock.hcl, .terraform, terraform.tfstate files if you are starting from scratch.

Terraform Execution: Infrastructure Resources Provisioning

Once you have finished declaring the resources, you can deploy all resources.

Let's begin to run the terraform scripts to provision infra required to Deploy DIGIT on AWS.

  1. First CD into the following directory and run the following command to create the remote state.

### Create the remote-state first, remember that the state name should be unique
cd DIGIT-DevOps/infra-as-code/terraform/sample-aws/remote-state
terraform init

terraform plan

terraform apply

Once the remote state is created, you are ready to provision DIGIT infra. Please run the following commands:

### Once the remote state is created, you can create the DIGIT Infra
cd DIGIT-DevOps/infra-as-code/terraform/sample-aws
terraform init

terraform plan

terraform apply

Important:

  • DB password will be asked for in the application stage. Please remember the password you have provided. It should be at least 8 characters long. Otherwise, RDS provisioning will fail.

  • The output of the apply command will be displayed on the console. Store this in a file somewhere. Values from this file will be used in the next step of deployment.

aws sts get-caller-identity

# Run the below command and give the respective region-code and the cluster name
aws eks --region <region-code> update-kubeconfig --name <cluster_name>

3. Finally, verify that you are able to connect to the cluster by running the following command

kubectl config use-context <cluster_name>

kubectl get nodes

NAME                                             STATUS AGE   VERSION               OS-Image           
ip-192-168-xx-1.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-2.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-3.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-4.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2 

At this point, your basic infra has been provisioned. Please move to the next step to install DIGIT.

Destroying Infra

To destroy previously-created infrastructure with Terraform, run the command below:

  1. ELB is not deployed via Terraform. ELB has created at deployment time by the setup of Kubernetes Ingress. This has to be deleted manually by deleting the ingress service.

    • kubectl delete deployment nginx-ingress-controller -n <namespace>

    • kubectl delete svc nginx-ingress-controller -n <namespace>

      Note: Namespace can be one of egov or jenkins.

  2. Delete S3 buckets manually from the AWS console and also verify if ELB got deleted.

  3. Run terraform destroy.

Sometimes all artefacts that are associated with a deployment cannot be deleted through Terraform. For example, RDS instances might have to be deleted manually. It is recommended to log in to the AWS management console and look through the infra to delete any remnants.​

Use this URL to , this will create both public and private keys in your machine, upload the public key into the account that you have just created, and give a name to it and ensure that you mention that in your terraform. This allows encrypting of all sensitive information.

Example user keybase user in eGov case is "egovterraform" needs to be created and has to upload the public key here -

you can use this to Decrypt your secret key. To decrypt PGP Message, Upload the PGP Message, PGP Private Key and Passphrase.

1.

In the sample-aws/remote-state/file, specify the s3 bucket to store all the states of the execution to keep track.

The inside the sample-aws folder contains the detailed resource definitions that need to be provisioned, please have a look at it.

2. Use this link to to get the kubeconfig file for the cluster. The region code is the default region provided in the availability zones in variables.tf. Eg. ap-south-1. EKS cluster name also should've been filled in variables.tf.

In case of if ELB is not deleted, you need to delete ELB from .

📓
AWS Account
Terraform
https://keybase.io/
create your own PGP key
keybase
https://keybase.io/egovterraform/pgp_keys.asc
portal
Terraform State Setup
main.tf
main.tf
get the kubeconfig from EKS
AWS console
Brief intro to deploy using terraform Infra-as-code
Pre-requisites to deployment
Steps to setup Terraform State
Steps to setup network infrastructure
Custom variables and configuration details
Provisioning infrastructure resources by executing terraform scripts
Steps to destroy the infrastructure