No review before a change happened
There was no step where a proposed change was visible to anyone before it ran.
Delivery pipeline
A safe path from a Git commit to a production Azure change.
Deploys ran from whichever engineer had a service principal secret configured locally, valid for longer than anyone could say. The infrastructure worked; the path to changing it had no review, no readable plan, and no record of who approved what.
← Some of our past work · anonymised prior delivery, no client named
Stack
Azure DevOpsTerraformWorkload identity federationMicrosoft Entra IDCheckovOPA
Rough timeline
| Phase | Typical duration |
|---|---|
| Pipeline design and federated credential set up | 1 week |
| Rollout across repositories | 2–4 weeks |
| Long-lived secrets retired, repository by repository | ongoing until the last one is gone |
The challenge
Deploys were a secret in a variable group, not a pipeline.
Whichever engineer had a valid service principal secret configured locally could apply to production, and the record of what happened was whatever they remembered to say in a team channel afterwards.
Long-lived service principal secrets were the default
Client secrets on Microsoft Entra ID app registrations, rotated rarely if at all, sitting in variable groups and on engineers’ own machines.
No environment concept
Staging and production were applied from the same pipeline with the same service connection, told apart only by which variables file someone remembered to pass.
What we found
A service principal that had only ever gained permissions, never lost them.
The credential in use had a history nobody could fully reconstruct, which is itself the finding: a secret with an unknown history is a secret you cannot reason about.
Permissions had accumulated, not been scoped
A single service principal, granted Contributor at the subscription scope early on and reused for everything since, with nobody able to say what the minimum role assignment actually was.
Rotating the secret risked breaking something unrelated
Because the same service principal was shared between people and pipelines, rotating it to respond to a leak could take out something nobody had connected to it.
No plan existed before an apply
The first anyone saw of a change was the change itself — there was nothing to review, only something to notice.
What InfraEdge changed
A pipeline with a plan, a policy gate, and a named approval before anything applies.
The shape of a safe change path does not depend on the provider.
| Change | What it did |
|---|---|
| Workload identity federation | Each pipeline authenticates to Azure using a federated credential on a Microsoft Entra ID app registration, exchanging Azure DevOps’ own OIDC token for a short-lived access token at run time. No client secret exists in a variable group, on anyone’s machine, or in the Terraform provider config. |
| Terraform plans on pull requests | Every pull request runs a plan and posts it for review, so the reviewer sees the actual proposed change, not a description of it. |
| Policy and security checks | Checkov and OPA run against every plan, catching a class of mistake — a storage account open to the internet, an over-broad network security group rule — before a human reviewer needs to look. |
| Environment approvals | Production is an Azure DevOps Environment with required approvers configured as a check; the pipeline pauses at the human gate until a named person approves that specific run. |
| Controlled applies | Apply runs the exact plan that was reviewed, not a fresh one, so what gets approved is what gets applied. |
Outcome
Every change has a commit, a plan, an approver and a deployment record.
That is a different thing from every change having gone well — it means that when one does not, there is a specific plan and a specific approval to look at, rather than a reconstruction exercise.
Long-lived service principal secrets are being retired repository by repository as each one moves onto federated credentials. For a team that had never had a dedicated platform function, InfraEdge helped define what that role should own once the pipeline was in place, and vetted candidates for it.
Book a 20-minute triage call
Twenty minutes, no charge. We work out what would actually help — which is sometimes us and sometimes not. Nothing is priced on the call; if there is work worth doing, a written scope and a price reach you within 24 hours.