State Sharing #
As infrastructure grows and gets split into several separate Terraform configurations, an unavoidable need arises: configuration A needs to know the outputs of configuration B. The networking team creates a VPC, the compute team needs that VPC ID to create EC2 instances. The platform team creates EKS, the application team needs the cluster endpoint to deploy workloads. How this information moves between configurations is an architectural decision with long-term implications for coupling, flexibility, and ease of troubleshooting.
Two Approaches to State Sharing #
There are two main ways to share information between separate Terraform configurations, with very different trade-offs.
APPROACH 1: terraform_remote_state
Method : Configuration B reads configuration A's state directly
Coupling: Tight — B must know A's backend details
Flexible: Low — an output change in A can break B
Debugging: Hard — dependencies hidden inside the configuration
APPROACH 2: Input Variables
Method : Values are passed explicitly at terraform apply
Coupling: Loose — B only receives values, doesn't know their origin
Flexible: High — the value source can change without modifying B
Debugging: Easy — dependencies explicit and clearly visible
terraform_remote_state: When Coupling Is Acceptable #
terraform_remote_state makes the most sense when the relationship between configurations is very stable and managed by the same team.
# The "networking" configuration produces outputs:
# outputs.tf in the networking workspace
output "vpc_id" {
value = aws_vpc.main.id
description = "VPC ID — consumed by the compute and database workspaces"
}
output "private_subnet_ids" {
value = aws_subnet.private[*].id
}
output "database_subnet_ids" {
value = aws_subnet.database[*].id
}
# The "compute" configuration reads the networking state
data "terraform_remote_state" "networking" {
backend = "s3"
config = {
bucket = "my-terraform-state"
key = "production/networking/terraform.tfstate"
region = "ap-southeast-1"
}
}
resource "aws_instance" "app" {
ami = data.aws_ami.ubuntu.id
subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0]
}
WHEN terraform_remote_state MAKES SENSE:
✓ Configurations managed by the same team
✓ The outputs being read are stable (names/types don't change often)
✓ The dependency between configurations is by design (not accidental)
✓ The source configuration is more "foundational" than the consumer
(networking → compute, not the other way around)
WHEN TO AVOID terraform_remote_state:
✗ Configurations managed by different teams without coordination
✗ Outputs in the source configuration change frequently
✗ There's a circular dependency (A reads B's state, B reads A's state)
✗ The consumer configuration needs to run independently
(for example: testing, isolated development)
Input Variables: Healthier Decoupling #
Instead of reading state directly, the configuration receives the needed values as input variables. The value source can be another configuration’s output, a CI/CD pipeline, or a tfvars file.
# The "compute" configuration receives values as variables — doesn't know their origin
variable "vpc_id" {
description = "ID of the VPC where resources will be placed"
type = string
}
variable "private_subnet_ids" {
description = "List of private subnet IDs"
type = list(string)
}
resource "aws_instance" "app" {
ami = data.aws_ami.ubuntu.id
subnet_id = var.private_subnet_ids[0]
}
# Method 1: Grab outputs from another configuration manually
cd environments/production/networking
NETWORKING_OUTPUTS=$(terraform output -json)
VPC_ID=$(echo $NETWORKING_OUTPUTS | jq -r '.vpc_id.value')
SUBNET_IDS=$(echo $NETWORKING_OUTPUTS | jq -r '.private_subnet_ids.value | @json')
# Then pass them to the compute configuration
cd ../compute
terraform apply \
-var="vpc_id=$VPC_ID" \
-var="private_subnet_ids=$SUBNET_IDS"
# Method 2: A CI/CD pipeline passing values between configurations
# .github/workflows/deploy.yml
jobs:
deploy-networking:
outputs:
vpc_id: ${{ steps.outputs.outputs.vpc_id }}
subnet_ids: ${{ steps.outputs.outputs.subnet_ids }}
steps:
- name: Apply networking
run: terraform apply -auto-approve
working-directory: environments/production/networking
- name: Capture outputs
id: outputs
run: |
echo "vpc_id=$(terraform output -raw vpc_id)" >> $GITHUB_OUTPUT
echo "subnet_ids=$(terraform output -json private_subnet_ids)" >> $GITHUB_OUTPUT
working-directory: environments/production/networking
deploy-compute:
needs: deploy-networking
steps:
- name: Apply compute
run: |
terraform apply -auto-approve \
-var="vpc_id=${{ needs.deploy-networking.outputs.vpc_id }}" \
-var='private_subnet_ids=${{ needs.deploy-networking.outputs.subnet_ids }}'
working-directory: environments/production/compute
Hybrid: Outputs Files as an Explicit Interface #
This pattern makes the dependency more explicit than terraform_remote_state but more automatic than passing values manually via the CLI.
# After the networking apply, generate an outputs.auto.tfvars file
# that will be automatically read by the compute configuration
# Script or CI step after the networking apply:
# terraform output -json | jq '{
# vpc_id: .vpc_id.value,
# private_subnet_ids: .private_subnet_ids.value
# }' > ../compute/networking-outputs.auto.tfvars.json
# environments/compute will read this file automatically
# (*.auto.tfvars.json is read without needing -var-file)
Anti-Pattern: Circular Dependency #
# ANTI-PATTERN: A reads B's state, B reads A's state
# A circular dependency that can't be applied without breaking the cycle
# networking/main.tf:
data "terraform_remote_state" "compute" {
# Networking reads outputs from compute
config = { key = "production/compute/terraform.tfstate" ... }
}
# compute/main.tf:
data "terraform_remote_state" "networking" {
# Compute reads outputs from networking
config = { key = "production/networking/terraform.tfstate" ... }
}
# Problem:
# - networking can't be applied because it needs compute's state
# - compute can't be applied because it needs networking's state
# - Nothing can run first (chicken-and-egg)
# SOLUTION: Identify the "foundational" resources
# and separate them into a configuration that doesn't depend on any other
Determining the Right Split Granularity #
TOO MONOLITHIC:
One configuration for the entire infrastructure
→ Slow plans, large blast radius, collaboration bottleneck
TOO FRAGMENTED:
50 small configurations that all depend on each other
→ Complicated orchestration, lots of state sharing, hard to understand
THE RIGHT GRANULARITY — split by:
LIFECYCLE: Resources changing at different frequencies
Networking (rarely changes) → its own configuration
Compute (changes often) → its own configuration
Applications (deployed often) → its own configuration
OWNERSHIP: Resources owned by different teams
Networking team → the networking configuration
Platform team → the platform configuration (EKS, RDS)
Application team → the application configuration
BLAST RADIUS: Resources that must not affect each other
Production database → its own configuration with very restricted access
Production networking → its own configuration
State Sharing via terraform_remote_state #
The most direct way to share data between workspaces.
# Workspace A: networking — outputs the VPC and subnets
output "vpc_id" {
value = aws_vpc.main.id
}
output "private_subnet_ids" {
value = aws_subnet.private[*].id
}
# Workspace B: compute — reads from workspace A
data "terraform_remote_state" "networking" {
backend = "s3"
config = {
bucket = "terraform-state"
key = "networking/terraform.tfstate"
region = "ap-southeast-1"
}
}
resource "aws_instance" "web" {
subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0]
}
sequenceDiagram
participant N as Networking<br/>(workspace A)
participant S as S3 State
participant C as Compute<br/>(workspace B)
N->>S: terraform apply → outputs(vpc_id, subnet_ids)
C->>S: data.terraform_remote_state
S-->>C: Return outputs
C->>C: Use subnet_ids for EC2State Sharing Security Considerations #
Sharing state between workspaces must be done carefully because state can contain sensitive data.
# PROBLEM: terraform_remote_state can access ALL outputs
# Including sensitive ones!
# SOLUTION 1: Create dedicated outputs for sharing (without sensitive data)
output "shared_info" {
description = "Info for other workspaces (non-sensitive)"
value = {
vpc_id = aws_vpc.main.id
subnet_ids = aws_subnet.private[*].id
# DON'T include: passwords, API keys, etc.
}
}
# SOLUTION 2: Use a data source from the provider directly
data "aws_vpc" "shared" {
filter {
name = "tag:Name"
values = ["shared-vpc"]
}
}
# Safer — only fetches the needed info
flowchart TD
A["Remote State\n(CONTAINS SECRETS!)"] -->|"terraform_remote_state"| B["Workspace B\n(accesses ALL outputs)"]
C["Provider\nData Source"] -->|"data.aws_vpc"| D["Workspace B\n(only specific info)"]
style A fill:#ffebee,stroke:#c62828
style B fill:#fff3e0,stroke:#e65100
style C fill:#e3f2fd,stroke:#1565c0
style D fill:#e8f5e9,stroke:#2e7d32Data Source vs Remote State #
# Approach 1: terraform_remote_state
data "terraform_remote_state" "networking" {
backend = "s3"
config = {
bucket = "tf-state"
key = "networking/terraform.tfstate"
region = "ap-southeast-1"
}
}
# Then:
resource "aws_instance" "web" {
subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0]
}
# Approach 2: Provider data source (PREFERRED)
data "aws_vpc" "shared" {
tags = { Name = "shared-vpc" }
}
data "aws_subnets" "private" {
filter {
name = "vpc-id"
values = [data.aws_vpc.shared.id]
}
filter {
name = "tag:Tier"
values = ["private"]
}
}
# Then:
resource "aws_instance" "web" {
subnet_id = data.aws_subnets.private.ids[0]
}
Summary #
- Two approaches:
terraform_remote_state(tight coupling, easy setup) vs input variables (decoupled, more flexible). Choose based on how stable the relationship between configurations is.terraform_remote_statefor closely related configurations managed by the same team — networking → compute is a classic right fit.- Input variables for decoupling between teams — a configuration doesn’t need to know who provides its values, only what the values are.
- A CI/CD pipeline can be the orchestrator that takes outputs from one configuration and passes them as variables to the next.
- Avoid circular dependencies — if A needs B and B needs A, identify the foundational resources and separate them into a layer that depends on nothing.
- Split granularity by lifecycle (how often things change), ownership (who manages what), and blast radius (what must not affect each other).