Remote #

Remote state moves the terraform.tfstate file from your local machine to centralized storage accessible by all team members and CI/CD pipelines. This isn’t just “state somewhere else” — remote backends usually come with crucial extras: encryption at rest, automatic versioning, and state locking. This article covers how to configure the most commonly used backends and when to choose which one.

The Backend Concept #

The backend is the Terraform component responsible for storing state and running certain operations. By default, the backend is local. Configuring a remote backend changes where state is stored.

LOCAL BACKEND (default):
  State → terraform.tfstate (on your machine)
  Locking → none
  Versioning → terraform.tfstate.backup (only 1 version)

REMOTE BACKEND (S3, GCS, Terraform Cloud, etc.):
  State → bucket/container in the cloud
  Locking → DynamoDB / GCS locks / built-in
  Versioning → S3 versioning / built-in history
  Encryption → at rest + in transit
  Audit → access logs available

flowchart TD
    A["terraform apply"] --> B{"Backend?"}

    B -->|"Default\n(local)"| C["terraform.tfstate"]
    B -->|"Remote"| D["S3 / GCS /\nTerraform Cloud"]

    C --> E["⚠️ No locking"]
    C --> F["Only 1 backup file"]
    C --> G["Not encrypted"]

    D --> H["✅ State locking"]
    D --> I["✅ Automatic versioning"]
    D --> J["✅ Encryption at rest"]
    D --> K["✅ Audit log"]

    style A fill:#e3f2fd,stroke:#1565c0
    style C fill:#fff3e0,stroke:#e65100
    style D fill:#e8f5e9,stroke:#2e7d32

The S3 Backend (AWS) #

S3 is the most common choice for teams working on AWS. The combination of S3 for storage and DynamoDB for locking is the industry-standard pattern.

# backend.tf
terraform {
  backend "s3" {
    bucket         = "my-terraform-state-prod"
    key            = "production/terraform.tfstate"
    region         = "ap-southeast-1"

    # Encryption at rest
    encrypt        = true

    # State locking using DynamoDB
    dynamodb_table = "terraform-state-lock"

    # Optional: use an IAM role for access
    # role_arn = "arn:aws:iam::123456789:role/TerraformStateRole"
  }
}

# Create the S3 bucket and DynamoDB table for the state backend
# (usually done separately before the main configuration)

resource "aws_s3_bucket" "terraform_state" {
  bucket = "my-terraform-state-prod"

  # Prevents deleting the bucket containing state
  lifecycle {
    prevent_destroy = true
  }
}

resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"  # Required — enables old state recovery
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_dynamodb_table" "terraform_lock" {
  name         = "terraform-state-lock"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

flowchart TD
    A["Terraform\nConfiguration"] --> B["S3 Bucket\n(state storage)"]
    A --> C["DynamoDB Table\n(locking)"]
    B --> D["Versioning\nenabled"]
    B --> E["Encryption\nAES256"]
    B --> F["prevent_destroy\nlifecycle"]

    style A fill:#e3f2fd,stroke:#1565c0
    style B fill:#e8f5e9,stroke:#2e7d32
    style C fill:#fff3e0,stroke:#e65100

The GCS Backend (Google Cloud) #

For teams working on Google Cloud, the GCS backend already supports locking natively without needing extra resources.

terraform {
  backend "gcs" {
    bucket  = "my-terraform-state"
    prefix  = "production"
    # The GCS backend has built-in locking — no extra resources needed
  }
}

Terraform Cloud / HCP Terraform #

Terraform Cloud (now called HCP Terraform) is the managed option that includes state storage, locking, versioning, run history, and a UI for plan review — all in one package.

terraform {
  cloud {
    organization = "my-org"

    workspaces {
      name = "production"
    }
  }
}

# Log in to Terraform Cloud
terraform login
# Will open a browser for authentication

# After logging in, init will set up the workspace in Terraform Cloud
terraform init

Good Key Structure for S3 #

When using the S3 backend for multiple environments or projects, a consistent key structure makes navigation easier.

RECOMMENDED KEY STRUCTURE:

s3://my-terraform-state/
  ├── production/
  │   ├── networking/terraform.tfstate
  │   ├── compute/terraform.tfstate
  │   └── database/terraform.tfstate
  ├── staging/
  │   ├── networking/terraform.tfstate
  │   └── compute/terraform.tfstate
  └── dev/
      └── terraform.tfstate

Configuration per environment:
  key = "production/networking/terraform.tfstate"
  key = "production/compute/terraform.tfstate"
  key = "staging/networking/terraform.tfstate"

Remote State Output: Reading State from Another Workspace #

One of the powerful remote state features is the ability to read outputs from another Terraform workspace. This allows infrastructure to be modularized into separate workspaces that can share data with each other.

# The "networking" workspace creates a VPC and outputs the VPC ID
# outputs.tf in the networking workspace:
output "vpc_id" {
  value = aws_vpc.main.id
}

output "private_subnet_ids" {
  value = aws_subnet.private[*].id
}

# The "compute" workspace reads outputs from the "networking" workspace
data "terraform_remote_state" "networking" {
  backend = "s3"

  config = {
    bucket = "my-terraform-state"
    key    = "production/networking/terraform.tfstate"
    region = "ap-southeast-1"
  }
}

# Use the outputs from the networking workspace
resource "aws_instance" "web" {
  ami       = var.ami_id
  subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0]
}

terraform_remote_state creates a direct coupling between two workspaces — an output change in the networking workspace can affect the compute workspace. Consider using input variables and passing values explicitly if this tight coupling isn’t desired.

sequenceDiagram
    participant C as Compute Workspace
    participant B as S3 Backend
    participant N as Networking Workspace
    
    Note over N: terraform apply complete
    N->>B: Write state + outputs
    Note over B: State stored<br/>vpc_id = vpc-abc123<br/>subnet_ids = [subnet-x, subnet-y]
    
    Note over C: terraform plan starts
    C->>B: data "terraform_remote_state" "networking"
    B-->>C: Return outputs
    C->>C: Use vpc_id, subnet_ids
    Note over C: Plan complete

flowchart LR
    A["Networking\nWorkspace"] -->|outputs:\nvpc_id, subnet_ids| B["terraform_remote_state\ndata source"]
    B -->|read outputs| C["Compute\nWorkspace"]
    C -->|"subnet_id =\ndata.terraform_remote_state...\n.outputs.private_subnet_ids[0]"| D["aws_instance.web"]

    style A fill:#e8f5e9,stroke:#2e7d32
    style B fill:#fff3e0,stroke:#e65100
    style C fill:#e3f2fd,stroke:#1565c0

Multi-Backend Failover #

Terraform doesn’t support backend failover natively — if the S3 backend is down, terraform apply will fail. But there are patterns you can apply to improve resilience.

# PROBLEM: S3 backend unavailable → pipeline stuck
$ terraform plan
# Error: failed to retrieve state from S3: RequestError...
# → The entire CI/CD pipeline stops

# SOLUTION 1: Cross-region replication on the S3 bucket

# Enable cross-region replication on the S3 state bucket
resource "aws_s3_bucket_replication_configuration" "terraform_state" {
  role   = aws_iam_role.replication.arn
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    id     = "replicate-state"
    status = "Enabled"

    destination {
      bucket        = aws_s3_bucket.terraform_state_replica.arn
      storage_class = "STANDARD"
    }
  }
}

# Replica bucket in a different region
resource "aws_s3_bucket" "terraform_state_replica" {
  provider = aws.replica_region
  bucket   = "my-terraform-state-prod-replica"
}

# THE BACKEND CAN'T FAIL OVER AUTOMATICALLY
# But if the primary S3 is down, you can temporarily switch the backend:

# EMERGENCY STEPS: Switch to the backup bucket
# 1. Update backend.tf to the replica bucket
# 2. Run terraform init to reconfigure
terraform init -migrate-state -force-copy
# 3. After the primary is back, switch back and re-migrate

# SOLUTION 2: Backend configuration with workspaces
# Store the backend configuration in Terraform Cloud/Enterprise
# hosted by HashiCorp → higher availability

State Encryption Deep Dive #

# AWS S3 SSE-S3 (default encryption, basic)
# Every object is encrypted with an S3-managed key
# No configuration needed — active by default on new buckets

# AWS S3 SSE-KMS (recommended for production)
# Encryption using a KMS key you can manage yourself
# Enables an audit trail of who accessed the key

# SSE-KMS encryption for the state bucket
resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    apply_server_side_encryption_by_default {
      kms_key_id        = aws_kms_key.terraform_state.arn
      sse_algorithm     = "aws:kms"
    }
    bucket_key_enabled = true  # Reduces KMS API costs
  }
}

resource "aws_kms_key" "terraform_state" {
  description             = "KMS key for Terraform state encryption"
  deletion_window_in_days = 30
  enable_key_rotation     = true  # Auto-rotate the key every year
}

resource "aws_kms_alias" "terraform_state" {
  name          = "alias/terraform-state"
  target_key_id = aws_kms_key.terraform_state.key_id
}

ENCRYPTION OPTIONS COMPARISON:

  SSE-S3 (AES256):
    Security: Standard
    Key management: S3-managed (can't be configured)
    Audit: Can't track who decrypted
    Cost: Free
    Best for: Dev, staging, personal projects

  SSE-KMS:
    Security: High
    Key management: Customer-managed key (CMK)
    Audit: CloudTrail logs all key usage
    Cost: $1/key/month + API call charges
    Best for: Production, compliance requirements

  Customer-provided key (SSE-C):
    Security: Very high
    Key management: Customer supplies the key on every request
    Audit: Full control
    Cost: Free (but high management overhead)
    Best for: Extreme security requirements (rarely used)

Remote State Data Lifecycle #

Understanding the remote state lifecycle helps you plan a state-handling strategy from creation to deletion.

flowchart TD
    A["terraform init\n(set up the backend)"] --> B["Empty state"]
    B --> C["terraform apply #1\n(create the first resource)"]
    C --> D["State v1\n(serial: 1)"]
    D --> E["terraform apply #2\n(add resources)"]
    E --> F["State v2\n(serial: 2)"]
    F --> G["S3 Versioning:\nv1 stays stored"]
    G --> H["terraform destroy\n(delete all resources)"]
    H --> I["State vN\n(serial: N, resources: [])"]
    I --> J["State stays in S3\n(empty but present)"]
    J --> K["Manual cleanup\nrequired"]

    style A fill:#e3f2fd,stroke:#1565c0
    style D fill:#e8f5e9,stroke:#2e7d32
    style G fill:#fff3e0,stroke:#e65100
    style K fill:#ffebee,stroke:#c62828

# State is NOT automatically deleted from S3 after terraform destroy
# The state contains {"resources":[]} but the file remains
# Plus all old versions remain in S3 versioning

# Cleanup of unused state:
# 1. Delete the current state version
aws s3 rm s3://my-terraform-state/production/terraform.tfstate

# 2. Delete all old versions (including delete markers)
aws s3api list-object-versions \
  --bucket my-terraform-state \
  --prefix production/terraform.tfstate \
  --query 'Versions[*].[VersionId,Key]' \
  --output text | while read vid key; do
    aws s3api delete-object --bucket my-terraform-state --key "$key" --version-id "$vid"
  done

# 3. Delete the DynamoDB lock entry (if it still exists)
aws dynamodb delete-item \
  --table-name terraform-state-lock \
  --key '{"LockID":{"S":"my-terraform-state/production/terraform.tfstate"}}'

# STATE BUCKET MONITORING:
# State files are usually small (KB-MB), but versioning can
# accumulate storage if you apply often

# Check the number of versions per state file
aws s3api list-object-versions \
  --bucket my-terraform-state \
  --query 'length(Versions)'

# Set a lifecycle policy for auto-cleaning old versions
# (if you don't want to keep all history)

# Lifecycle rule to delete old versions
resource "aws_s3_bucket_lifecycle_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    id     = "cleanup-old-versions"
    status = "Enabled"

    noncurrent_version_expiration {
      noncurrent_days = 90  # Delete old versions after 90 days
    }

    noncurrent_version_transition {
      noncurrent_days = 30
      storage_class   = "GLACIER"  # Move to Glacier after 30 days
    }
  }
}

Summary #

• Remote backends move state to centralized storage with extra features: encryption, automatic versioning, and locking.
• S3 + DynamoDB is the industry-standard combination for AWS — S3 for storage, DynamoDB for locking.
• The GCS backend has built-in locking — no extra resources needed like DynamoDB.
• Terraform Cloud is the complete managed option — state storage, locking, run history, and a UI in one package.
• A consistent key structure in S3 (<env>/<component>/terraform.tfstate) makes navigation easier for multi-environment projects.
• terraform_remote_state allows other workspaces to read outputs — powerful but creates coupling, use it thoughtfully.
• Multi-backend failover isn’t supported natively — use S3 cross-region replication and reconfigure the backend in emergencies.
• SSE-KMS is recommended for production — it provides an audit trail through CloudTrail and automatic key rotation.
• State stays in S3 after terraform destroy — perform manual cleanup to delete the empty file and old versions.
• S3 lifecycle policies can automatically move old versions to Glacier and delete them after 90 days.

← Previous: Local   Next: Locking →