Using Terraform to deploy apps to Kubernetes.
Terraform manages state in a unique way, allowing you to use various state backends such as Kubernetes or S3 to track your deployments. This allows you to apply or destroy changes to your infrastructure from anywhere.
Additionally, it's a great collaborative tool. You can create reusable modules making it simple to spin up a new instance of any service quickly, and manage different environments with workspaces.
Follow the Terraform installation guide and learn Terraform with one of their tutorials.
Remote state is the recommended way to manage Terraform state. Especially as your workspace grows, and you might want to modify your infrastructure from a different machine, or CI/CD.
Terraform supports S3 as a state backend as well as Kubernetes. Using Kubernetes will store state in a secret, and supports all the features for Terraform's state, but if you want to view the state using something like Terraboard, consider using S3.
Once you start deploying most of your services with Terraform, you might want to automate this process. If you're managing your Terraform modules with Git, CI/CD could be ideal for this.
An example pipeline with Drone:
kind: pipeline
type: kubernetes
name: pull request
steps:
- name: terraform plan
image: hashicorp/terraform:1.3.7
environment:
AWS_ACCESS_KEY_ID:
from_secret: s3_access_key
AWS_SECRET_ACCESS_KEY:
from_secret: s3_secret_key
TERRAFORM_VARS:
from_secret: terraform_vars
commands:
- echo $TERRAFORM_VARS | tee terraform.tfvars.json
- terraform init
- terraform validate
- terraform plan
trigger:
event:
- pull_request
---
kind: pipeline
type: kubernetes
name: apply on merge
steps:
- name: terraform apply
image: hashicorp/terraform:1.3.7
environment:
AWS_ACCESS_KEY_ID:
from_secret: s3_access_key
AWS_SECRET_ACCESS_KEY:
from_secret: s3_secret_key
TERRAFORM_VARS:
from_secret: terraform_vars
commands:
- echo $TERRAFORM_VARS | tee terraform.tfvars.json
- terraform init
- terraform validate
- terraform apply -auto-approve
trigger:
branch:
- main
event:
- push
This pipeline will run terraform plan when a pull request is created in your Git repository, and terraform apply once pushed or merged with main. Additionally, it runs terraform init to initialize the workspace, and terraform validate to validate the configuration.
You can also use Drone's promotion functionality to apply your configuration manually after a pipeline invocation. This makes sense if you want to reduce the overhead of creating a PR every time your configuration has changed.
Oftentimes, you'll have Secrets and ConfigMaps providing values to your deployments and pods, which you might intuitively think that when updated will update those values. Unfortunately, Kubernetes does not re-create Pods when dependencies change, which means a value in the Deployment spec.template block must change.
An easy fix to this problem is to add an anotation, that contains a hash of the data in these dependencies. Whenever values are updated in a Secret or ConfigMap Terraform will update this hash. This will trigger a re-creation of any pods managed by the workload.
resource "kubernetes_deployment" "sample" {
metadata {
name = "sample"
}
spec {
template {
metadata {
annotations = {
"ravianand.me/config-hash" = sha1(jsonencode(merge(
kubernetes_config_map.sample.data,
kubernetes_secret.sample.data,
)))
}
}
}
}
}