Two Terraform problems generate more 2 a.m. pages than any other: a state lock you cannot acquire, and drift between what state believes and what the cloud actually looks like. Both are recoverable, and both are dangerous if you reach for the aggressive fix first. force-unlock and a reflexive apply are exactly how a locked pipeline becomes a corrupted state file or a destroyed database. This guide covers the mechanism and the disciplined recovery for each.


Part 1 — State locking

Why the lock exists

Terraform state is a single source of truth that must not be written by two processes at once. Backends that support locking (S3 with a DynamoDB lock table or S3 native locking, Azure Blob leases, GCS, Terraform Cloud, Consul) take a lock at the start of any operation that reads-then-writes state — apply, plan with refresh, state subcommands — and release it at the end. If a second run starts while the first holds the lock, you get:

Error: Error acquiring the state lock

Lock Info:
  ID:        9f3c1a2b-...        # the Lock ID you need for force-unlock
  Path:      s3://.../terraform.tfstate
  Operation: OperationTypeApply
  Who:       jenkins@ci-agent-7
  Created:   2026-09-07 01:12:44 UTC
  Info:      

Diagnose before you unlock

The Lock Info tells you almost everything. Ask, in order:

  1. Is a real run in progress? Check CI (is there a running pipeline for this stack?) and ask the person in Who. If yes — wait. This is not a stale lock; unlocking it will corrupt state.
  2. How old is Created? A lock from seconds ago is almost certainly live. A lock hours old whose owning job you can prove is dead (crashed agent, cancelled pipeline, killed laptop) is a stale lock.
  3. Did a previous run crash? Network drops, SIGKILL, spot-instance reclaim of a CI runner, or Ctrl-C at the wrong moment all leave a lock behind because the release never ran.

Releasing a stale lock safely

Only once you have confirmed nothing is running:

terraform force-unlock <LOCK_ID>
# use the exact ID from the error; do not force-unlock blindly

If you cannot use the CLI (backend broken), you can delete the lock item directly — e.g. the DynamoDB lock table row keyed on the state path — but treat that as a last resort and know exactly which item you are removing. The cardinal rule: a lock protects state integrity; removing it while an apply is mid-write means two writers and a good chance of a corrupted or half-written state file that then needs restoring from backend versioning.

Preventing lock incidents

  • Enable state file versioning on the backend bucket so you can roll back a bad write.
  • Serialise applies per state in CI (concurrency group / mutex) so two pipelines never target the same state.
  • Split large monolithic states so unrelated changes don't queue behind each other on one lock.
  • Set sensible pipeline timeouts so a hung run fails and releases rather than holding the lock indefinitely.

Part 2 — Drift

What drift is

Drift is any divergence between the real resource and what Terraform state records: a security group rule added in the console during an incident, a tag changed by a governance tool, an autoscaling group resized by hand, or a provider default that shifted between versions. Terraform is declarative — it will try to make reality match config on the next apply — so undetected drift plus a routine apply is how unrelated manual fixes silently get reverted.

Detecting drift without changing anything

terraform plan -refresh-only     # show ONLY drift: real state vs recorded state
terraform plan                   # drift + the config changes you're about to make

Read a -refresh-only plan first. It answers "what changed underneath me?" without mixing in your own pending edits. Each drifted attribute is shown as recorded-value → real-value. Now decide, per resource, which side is right.

The reconciliation decision

SituationCorrect action
Manual change was intentional and should stayUpdate the Terraform config to match reality, so the next plan is clean
Manual change was a mistake / unauthorisedRun a normal apply to restore the declared state
Resource was created outside Terraformterraform import it into state, then write matching config
Plan shows -/+ (replace) on a stateful resourceSTOP — review before applying; a replace can destroy a database/volume

The replace trap

The single most destructive drift outcome is a plan that quietly proposes to destroy and recreate a stateful resource — an RDS instance, an EBS volume, a stateful set's storage — because a "ForceNew" attribute drifted or your config changed one. Terraform marks these -/+ resource must be replaced. Never approve an apply without reading every replace line. When a change genuinely needs a replace but the data must survive, use create_before_destroy, snapshots, or a manual migration — not a blind apply.

Targeted, reviewable applies

  • Save the plan and apply exactly that: terraform plan -out tf.plan then terraform apply tf.plan — no surprises between review and execution.
  • Scope risky changes: terraform apply -target=aws_security_group.web to fix one resource without touching the blast radius.
  • For import-heavy reconciliation, prefer config-driven import blocks so the import is code-reviewed, not a one-off CLI command.

Common wrong approaches

  • force-unlock as a reflex. If the lock is live, you have just set up a state-corrupting double write.
  • Applying to "clean up" drift without reading the plan. That reverts other people's legitimate emergency fixes and can trigger replacements.
  • Editing the state file by hand. Almost never necessary; use terraform import, state mv, state rm, which are validated operations.
  • One giant state for everything. Guarantees lock contention and makes every drift review enormous. Split by blast radius.

Related resources

If a Terraform apply is blocked or a drifted plan is proposing something frightening in a live environment, real-time proxy job support gives you a second senior pair of eyes on the plan before you approve it. And if you are being asked to reason about state, locking and drift in a technical screen, that is standard ground for a DevOps proxy interview support session.

Last reviewed: September 2026.