Infrastructure Automation Pipeline
Turned manual, error-prone provisioning into reproducible infrastructure-as-code with CI/CD — environments spun up in minutes, drift eliminated, toil cut.
- Role
- Design & implementation
- Timeline
- 2025
- Stack
- Terraform · AWS · GitHub Actions · Docker · Bash
The problem
Environments were provisioned by hand from a runbook. The steps were correct on the day they were written and quietly wrong thereafter — staging and production drifted apart, a new environment took days of careful clicking, and nobody could say with confidence what was actually deployed. The failure mode wasn't dramatic; it was that "works in staging" had stopped being evidence of anything.
Constraints
- Existing infrastructure was already running and could not be torn down to be rebuilt cleanly.
- Credentials could never touch a developer machine or a repository.
- Changes had to be reviewable by people who don't write Terraform daily.
- A wrong apply against production had to be difficult, not merely discouraged.
Architecture
- 01
Modules
Infrastructure is expressed as composable Terraform modules — networking, compute, data, IAM — with environments as thin configuration on top. Staging and production differ in sizing and counts, not in structure, which is what keeps them honest.
- 02
State & isolation
Remote state with locking, split per environment so a staging apply can never touch production state. Adopting the running infrastructure meant importing it into state rather than recreating it.
- 03
Pipeline
GitHub Actions runs fmt, validate and a security scan on every pull request, then posts the plan as a comment so a reviewer reads the actual diff instead of trusting the description. Apply runs only on merge, against a protected environment with required approval.
- 04
Drift detection
A scheduled plan runs against every environment and reports non-empty diffs, so out-of-band console changes surface as a notification rather than as a surprise during the next deploy.
Decisions & tradeoffs
Plan-on-PR with the output posted to the review.
why The plan is the change. Putting it in the review turns infrastructure into something a teammate can actually approve, and catches destructive diffs before they reach an apply.
cost The pipeline needs read access to real cloud state on pull requests, which has to be scoped carefully and kept read-only.
Import existing resources instead of rebuilding.
why The systems were live. Importing let the codebase become authoritative incrementally, without a migration event.
cost A slow, unglamorous import phase, and a period where code and reality had to be reconciled by hand.
OIDC federation for CI credentials, no long-lived keys.
why Short-lived, workflow-scoped credentials remove the standing secret that is the usual root cause of a cloud breach.
cost More setup complexity up front and a harder path for anyone wanting to run the pipeline locally.
Outcomes
[verify]
Provisioning time
Time to stand up a full environment, before versus after.
~40% [verify]
Manual toil reduced
Estimate on the card today — replace with hours-per-week recovered, or drop the number.
[verify]
Drift incidents
Out-of-band changes caught by scheduled plans since detection was added.
What I'd do differently
- The pipeline mattered more than the Terraform. Code alone doesn't stop drift — enforcing that the only path to production is a reviewed apply does.
- Importing running infrastructure is the whole job on a brownfield project; greenfield tutorials skip the part that takes the time.
- Making the plan visible in review changed team behaviour faster than any policy did.
next case study
Resilient Backend Service