State Sharing #

As infrastructure grows and gets split into several separate Terraform configurations, an unavoidable need arises: configuration A needs to know the outputs of configuration B. The networking team creates a VPC, the compute team needs that VPC ID to create EC2 instances. The platform team creates EKS, the application team needs the cluster endpoint to deploy workloads. How this information moves between configurations is an architectural decision with long-term implications for coupling, flexibility, and ease of troubleshooting.

Two Approaches to State Sharing #

There are two main ways to share information between separate Terraform configurations, with very different trade-offs.

APPROACH 1: terraform_remote_state
  Method : Configuration B reads configuration A's state directly
  Coupling: Tight — B must know A's backend details
  Flexible: Low — an output change in A can break B
  Debugging: Hard — dependencies hidden inside the configuration

APPROACH 2: Input Variables
  Method : Values are passed explicitly at terraform apply
  Coupling: Loose — B only receives values, doesn't know their origin
  Flexible: High — the value source can change without modifying B
  Debugging: Easy — dependencies explicit and clearly visible

terraform_remote_state: When Coupling Is Acceptable #

terraform_remote_state makes the most sense when the relationship between configurations is very stable and managed by the same team.

# The "networking" configuration produces outputs:
# outputs.tf in the networking workspace

output "vpc_id" {
  value       = aws_vpc.main.id
  description = "VPC ID — consumed by the compute and database workspaces"
}

output "private_subnet_ids" {
  value       = aws_subnet.private[*].id
}

output "database_subnet_ids" {
  value       = aws_subnet.database[*].id
}
# The "compute" configuration reads the networking state
data "terraform_remote_state" "networking" {
  backend = "s3"

  config = {
    bucket = "my-terraform-state"
    key    = "production/networking/terraform.tfstate"
    region = "ap-southeast-1"
  }
}

resource "aws_instance" "app" {
  ami       = data.aws_ami.ubuntu.id
  subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0]
}
WHEN terraform_remote_state MAKES SENSE:
  ✓ Configurations managed by the same team
  ✓ The outputs being read are stable (names/types don't change often)
  ✓ The dependency between configurations is by design (not accidental)
  ✓ The source configuration is more "foundational" than the consumer
    (networking → compute, not the other way around)

WHEN TO AVOID terraform_remote_state:
  ✗ Configurations managed by different teams without coordination
  ✗ Outputs in the source configuration change frequently
  ✗ There's a circular dependency (A reads B's state, B reads A's state)
  ✗ The consumer configuration needs to run independently
    (for example: testing, isolated development)

Input Variables: Healthier Decoupling #

Instead of reading state directly, the configuration receives the needed values as input variables. The value source can be another configuration’s output, a CI/CD pipeline, or a tfvars file.

# The "compute" configuration receives values as variables — doesn't know their origin
variable "vpc_id" {
  description = "ID of the VPC where resources will be placed"
  type        = string
}

variable "private_subnet_ids" {
  description = "List of private subnet IDs"
  type        = list(string)
}

resource "aws_instance" "app" {
  ami       = data.aws_ami.ubuntu.id
  subnet_id = var.private_subnet_ids[0]
}
# Method 1: Grab outputs from another configuration manually
cd environments/production/networking
NETWORKING_OUTPUTS=$(terraform output -json)

VPC_ID=$(echo $NETWORKING_OUTPUTS | jq -r '.vpc_id.value')
SUBNET_IDS=$(echo $NETWORKING_OUTPUTS | jq -r '.private_subnet_ids.value | @json')

# Then pass them to the compute configuration
cd ../compute
terraform apply \
  -var="vpc_id=$VPC_ID" \
  -var="private_subnet_ids=$SUBNET_IDS"
# Method 2: A CI/CD pipeline passing values between configurations
# .github/workflows/deploy.yml

jobs:
  deploy-networking:
    outputs:
      vpc_id: ${{ steps.outputs.outputs.vpc_id }}
      subnet_ids: ${{ steps.outputs.outputs.subnet_ids }}
    steps:
      - name: Apply networking
        run: terraform apply -auto-approve
        working-directory: environments/production/networking

      - name: Capture outputs
        id: outputs
        run: |
          echo "vpc_id=$(terraform output -raw vpc_id)" >> $GITHUB_OUTPUT
          echo "subnet_ids=$(terraform output -json private_subnet_ids)" >> $GITHUB_OUTPUT          
        working-directory: environments/production/networking

  deploy-compute:
    needs: deploy-networking
    steps:
      - name: Apply compute
        run: |
          terraform apply -auto-approve \
            -var="vpc_id=${{ needs.deploy-networking.outputs.vpc_id }}" \
            -var='private_subnet_ids=${{ needs.deploy-networking.outputs.subnet_ids }}'          
        working-directory: environments/production/compute

Hybrid: Outputs Files as an Explicit Interface #

This pattern makes the dependency more explicit than terraform_remote_state but more automatic than passing values manually via the CLI.

# After the networking apply, generate an outputs.auto.tfvars file
# that will be automatically read by the compute configuration

# Script or CI step after the networking apply:
# terraform output -json | jq '{
#   vpc_id: .vpc_id.value,
#   private_subnet_ids: .private_subnet_ids.value
# }' > ../compute/networking-outputs.auto.tfvars.json

# environments/compute will read this file automatically
# (*.auto.tfvars.json is read without needing -var-file)

Anti-Pattern: Circular Dependency #

# ANTI-PATTERN: A reads B's state, B reads A's state
# A circular dependency that can't be applied without breaking the cycle

# networking/main.tf:
data "terraform_remote_state" "compute" {
  # Networking reads outputs from compute
  config = { key = "production/compute/terraform.tfstate" ... }
}

# compute/main.tf:
data "terraform_remote_state" "networking" {
  # Compute reads outputs from networking
  config = { key = "production/networking/terraform.tfstate" ... }
}

# Problem:
# - networking can't be applied because it needs compute's state
# - compute can't be applied because it needs networking's state
# - Nothing can run first (chicken-and-egg)

# SOLUTION: Identify the "foundational" resources
# and separate them into a configuration that doesn't depend on any other

Determining the Right Split Granularity #

TOO MONOLITHIC:
  One configuration for the entire infrastructure
  → Slow plans, large blast radius, collaboration bottleneck

TOO FRAGMENTED:
  50 small configurations that all depend on each other
  → Complicated orchestration, lots of state sharing, hard to understand

THE RIGHT GRANULARITY — split by:

  LIFECYCLE: Resources changing at different frequencies
    Networking (rarely changes) → its own configuration
    Compute (changes often) → its own configuration
    Applications (deployed often) → its own configuration

  OWNERSHIP: Resources owned by different teams
    Networking team → the networking configuration
    Platform team → the platform configuration (EKS, RDS)
    Application team → the application configuration

  BLAST RADIUS: Resources that must not affect each other
    Production database → its own configuration with very restricted access
    Production networking → its own configuration


State Sharing via terraform_remote_state #

The most direct way to share data between workspaces.

# Workspace A: networking — outputs the VPC and subnets
output "vpc_id" {
  value = aws_vpc.main.id
}

output "private_subnet_ids" {
  value = aws_subnet.private[*].id
}

# Workspace B: compute — reads from workspace A
data "terraform_remote_state" "networking" {
  backend = "s3"
  config = {
    bucket = "terraform-state"
    key    = "networking/terraform.tfstate"
    region = "ap-southeast-1"
  }
}

resource "aws_instance" "web" {
  subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0]
}
sequenceDiagram
    participant N as Networking<br/>(workspace A)
    participant S as S3 State
    participant C as Compute<br/>(workspace B)

    N->>S: terraform apply → outputs(vpc_id, subnet_ids)
    C->>S: data.terraform_remote_state
    S-->>C: Return outputs
    C->>C: Use subnet_ids for EC2

State Sharing Security Considerations #

Sharing state between workspaces must be done carefully because state can contain sensitive data.

# PROBLEM: terraform_remote_state can access ALL outputs
# Including sensitive ones!

# SOLUTION 1: Create dedicated outputs for sharing (without sensitive data)
output "shared_info" {
  description = "Info for other workspaces (non-sensitive)"
  value = {
    vpc_id     = aws_vpc.main.id
    subnet_ids = aws_subnet.private[*].id
    # DON'T include: passwords, API keys, etc.
  }
}

# SOLUTION 2: Use a data source from the provider directly
data "aws_vpc" "shared" {
  filter {
    name   = "tag:Name"
    values = ["shared-vpc"]
  }
}
# Safer — only fetches the needed info
flowchart TD
    A["Remote State\n(CONTAINS SECRETS!)"] -->|"terraform_remote_state"| B["Workspace B\n(accesses ALL outputs)"]

    C["Provider\nData Source"] -->|"data.aws_vpc"| D["Workspace B\n(only specific info)"]

    style A fill:#ffebee,stroke:#c62828
    style B fill:#fff3e0,stroke:#e65100
    style C fill:#e3f2fd,stroke:#1565c0
    style D fill:#e8f5e9,stroke:#2e7d32

Data Source vs Remote State #

# Approach 1: terraform_remote_state
data "terraform_remote_state" "networking" {
  backend = "s3"
  config = {
    bucket = "tf-state"
    key    = "networking/terraform.tfstate"
    region = "ap-southeast-1"
  }
}

# Then:
resource "aws_instance" "web" {
  subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0]
}

# Approach 2: Provider data source (PREFERRED)
data "aws_vpc" "shared" {
  tags = { Name = "shared-vpc" }
}

data "aws_subnets" "private" {
  filter {
    name   = "vpc-id"
    values = [data.aws_vpc.shared.id]
  }
  filter {
    name   = "tag:Tier"
    values = ["private"]
  }
}

# Then:
resource "aws_instance" "web" {
  subnet_id = data.aws_subnets.private.ids[0]
}

Summary #

  • Two approaches: terraform_remote_state (tight coupling, easy setup) vs input variables (decoupled, more flexible). Choose based on how stable the relationship between configurations is.
  • terraform_remote_state for closely related configurations managed by the same team — networking → compute is a classic right fit.
  • Input variables for decoupling between teams — a configuration doesn’t need to know who provides its values, only what the values are.
  • A CI/CD pipeline can be the orchestrator that takes outputs from one configuration and passes them as variables to the next.
  • Avoid circular dependencies — if A needs B and B needs A, identify the foundational resources and separate them into a layer that depends on nothing.
  • Split granularity by lifecycle (how often things change), ownership (who manages what), and blast radius (what must not affect each other).

← Previous: Cross Provider Reference   Next: CI Pipeline →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact