Anti-Pattern: Terraform as CM #

Terraform can create EC2 instances, and EC2 instances need configuration after creation — install nginx, copy config files, set up a monitoring agent. Because Terraform is already in hand and can run remote-exec provisioners, the temptation to do server configuration directly from Terraform is very strong. This is an anti-pattern that looks like a simple solution at first, but creates ever-growing problems over time. Terraform is designed for infrastructure provisioning, not configuration management.

flowchart LR
    subgraph Terraform["Terraform (Provisioning)"]
        A["Create EC2"] --> B["Create VPC"]
        B --> C["Create S3"]
    end
    subgraph CM["Ansible/Puppet (Configuration)"]
        D["Install nginx"] --> E["Config files"]
        E --> F["Setup agents"]
    end
    A --> D

    style A fill:#3b82f6,stroke:#1e40af,color:#fff
    style B fill:#3b82f6,stroke:#1e40af,color:#fff
    style C fill:#3b82f6,stroke:#1e40af,color:#fff
    style D fill:#10b981,stroke:#059669,color:#fff
    style E fill:#10b981,stroke:#059669,color:#fff
    style F fill:#10b981,stroke:#059669,color:#fff

What “Terraform as CM” Means #

Configuration Management (CM) is the practice of managing configuration inside servers: installing packages, managing files, setting service states, creating users, and so on. The tools designed for this are Ansible, Chef, Puppet, and SaltStack. Terraform is a provisioning tool — it creates and manages cloud infrastructure, not the content inside that infrastructure.

THE LINE THAT SHOULD BE CLEAR:

  TERRAFORM (provisioning):
    ✓ Create EC2 instances
    ✓ Create VPCs, security groups, load balancers
    ✓ Create RDS databases
    ✓ Create S3 buckets
    ✓ Manage IAM roles

  CONFIGURATION MANAGEMENT TOOL (configuration inside servers):
    ✓ Install nginx inside an EC2 instance
    ✓ Copy configuration files to servers
    ✓ Start and enable services
    ✓ Create users and set permissions
    ✓ Deploy applications

  THIS LINE MUST NOT BE BLURRED.

Why the remote-exec Provisioner Is Problematic #

# ANTI-PATTERN: Using remote-exec for server configuration

resource "aws_instance" "web" {
  ami           = "ami-0abcdef1234567890"
  instance_type = "t3.micro"
  key_name      = aws_key_pair.deployer.key_name

  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update -y",
      "sudo apt-get install -y nginx",
      "sudo systemctl enable nginx",
      "sudo systemctl start nginx",
      "sudo bash -c 'echo Hello > /var/www/html/index.html'",
    ]

    connection {
      type        = "ssh"
      user        = "ubuntu"
      private_key = file("~/.ssh/id_rsa")  # ✗ Hardcoded path, not portable
      host        = self.public_ip
    }
  }
}

The problems with this approach:

remote-exec PROBLEMS:

  1. NOT IDEMPOTENT
     Run terraform apply twice → likely errors because
     apt-get or nginx already exists. Or worse: silent success
     that actually didn't execute what was expected.

  2. CAN'T BE VERIFIED
     terraform plan can't show what the provisioner will do.
     You don't know its effect before running it.

  3. NEEDS SSH/WinRM ACCESS
     The EC2 instance must be reachable from the runner running Terraform.
     This means the security group must open port 22 to the CI/CD server —
     an unnecessary security concern.

  4. CONFIGURATION ISN'T STORED IN STATE
     Once the provisioner has run, Terraform doesn't know or track
     what was configured. There's no idempotency check.

  5. PROVISIONERS ONLY RUN AT CREATE TIME
     If the server configuration needs updating, Terraform can't
     "re-run" the provisioner without destroying and recreating the instance.

  6. TERRAFORM PLAN BECOMES UNTRUSTWORTHY
     "No changes" doesn't mean the server configuration matches expectations.

The Problem with the file Provisioner #

# ANTI-PATTERN: Using a file provisioner to deploy configuration

resource "aws_instance" "app" {
  ami           = "ami-0abcdef1234567890"
  instance_type = "t3.micro"

  provisioner "file" {
    source      = "config/nginx.conf"    # ✗ Fragile relative path
    destination = "/etc/nginx/nginx.conf"

    connection {
      type = "ssh"
      # ...
    }
  }

  provisioner "remote-exec" {
    inline = ["sudo nginx -t && sudo systemctl reload nginx"]
    # ...
  }
}

# Problems:
# - If nginx.conf changes, Terraform doesn't know because it's not in state
# - terraform plan always says "No changes" even when the config file changed
# - No way to update the server configuration without recreating the instance

The Right Way: Separate Provisioning and Configuration #

# CORRECT: Terraform only provisions infrastructure
# Server configuration is delegated to the right tool

resource "aws_instance" "web" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"

  # Minimal bootstrap via user_data (cloud-init)
  # Only for very basic initial setup
  user_data = base64encode(<<-EOF
    #!/bin/bash
    # Install the SSM agent so the instance can be managed via AWS Systems Manager
    snap install amazon-ssm-agent --classic
    systemctl enable snap.amazon-ssm-agent.amazon-ssm-agent
    systemctl start snap.amazon-ssm-agent.amazon-ssm-agent

    # Install Ansible for subsequent config management
    apt-get update
    apt-get install -y ansible
  EOF
  )

  # No provisioners — no SSH from Terraform
  # No remote-exec — no file provisioner
}

# Outputs that Ansible or the next pipeline will use
output "instance_ids" {
  value = aws_instance.web[*].id
}

# The correct pipeline: Terraform for infra, Ansible for configuration

# .github/workflows/deploy.yml
jobs:
  provision:
    name: Terraform — Provision Infrastructure
    steps:
      - name: Terraform Apply
        run: terraform apply -auto-approve
      - name: Save Instance IDs
        run: terraform output -json instance_ids > instances.json

  configure:
    name: Ansible — Configure Servers
    needs: provision  # Run after provisioning finishes
    steps:
      - name: Generate Ansible Inventory
        run: |
          cat instances.json | python3 scripts/generate-inventory.py > inventory.ini          

      - name: Run Ansible Playbook
        run: |
          ansible-playbook \
            -i inventory.ini \
            playbooks/web-server.yml

When user_data Is Acceptable #

user_data (cloud-init) is an acceptable exception for minimal configuration at first instance boot.

# Appropriate user_data — minimal bootstrap only
resource "aws_instance" "app" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.medium"

  user_data = base64encode(<<-EOF
    #!/bin/bash
    # 1. Install the agent for configuration management
    apt-get install -y ansible
    # or: install the Puppet agent, Chef client, SSM agent

    # 2. Register with the CM tool for subsequent configuration
    ansible-pull -U https://github.com/myorg/infra-playbooks.git

    # DON'T: install complete applications, copy many files, complex setup
    # Minimal bootstrap is only to "bootstrap" into the CM tool
  EOF
  )

  lifecycle {
    # user_data usually doesn't need updates once using a CM tool
    ignore_changes = [user_data]
  }
}

Alternative: Immutable Infrastructure #

The best approach to avoid configuration management entirely is immutable infrastructure — build a new AMI containing all the configuration, then replace the old instance with the new one.

# Immutable infrastructure pattern with Packer + Terraform

# 1. Packer creates an AMI with all the configuration inside
#    (see: packer build web-server.pkr.hcl)

# 2. Terraform only needs to know the latest AMI ID
data "aws_ami" "web_server" {
  most_recent = true
  owners      = ["self"]  # AMIs created by your own team

  filter {
    name   = "name"
    values = ["web-server-*"]
  }

  filter {
    name   = "tag:Version"
    values = [var.app_version]
  }
}

resource "aws_instance" "web" {
  ami           = data.aws_ami.web_server.id  # The already-configured AMI
  instance_type = "t3.micro"
  # No provisioners, no complex user_data
  # All configuration is already inside the AMI
}

flowchart TD
    A["Terraform\nprovisions infra"] --> B["OS Image\n(AMI/Packer)"]
    A --> C["Ansible\nconfigures server"]
    A --> D["User Data\nbootstrap script"]
    B --> E["Immutable\n✅"]
    C --> F["Mutable\n✅"]
    D --> G["Simple only\n⚠️"]

    style A fill:#8b5cf6,stroke:#6d28d9,color:#fff
    style B fill:#10b981,stroke:#059669,color:#fff
    style C fill:#10b981,stroke:#059669,color:#fff
    style D fill:#f59e0b,stroke:#d97706,color:#fff
    style E fill:#10b981,stroke:#059669,color:#fff
    style F fill:#10b981,stroke:#059669,color:#fff
    style G fill:#f59e0b,stroke:#d97706,color:#fff

GitOps Workflow with Terraform #

flowchart TD
    A["Developer\ncreates a PR"] --> B["CI: terraform plan"]
    B --> C["Plan reviewed\nby the team"]
    C --> D["Merge the PR"]
    D --> E["CD: terraform apply"]
    E --> F["Infrastructure\nupdated"]

    style A fill:#e3f2fd,stroke:#1565c0
    style F fill:#e8f5e9,stroke:#2e7d32
# GitOps pipeline
on:
  push:
    branches: [main]
  pull_request:

jobs:
  plan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: terraform init
      - run: terraform plan -out=tfplan
      - uses: actions/upload-artifact@v4
        with:
          name: tfplan
          path: tfplan

  apply:
    needs: plan
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
      - run: terraform apply tfplan

Terraform Cloud Integration #

# Terraform Cloud as a remote backend + collaboration tool
terraform {
  cloud {
    organization = "my-org"
    
    workspaces {
      name = "my-app-production"
    }
  }
}

flowchart TD
    A["Developer\npushes code"] --> B["TFC: detects\nchanges"]
    B --> C["TFC: auto\nplan"]
    C --> D["Team review\nin the TFC UI"]
    D --> E["Approve"]
    E --> F["TFC: auto\napply"]

    style A fill:#e3f2fd,stroke:#1565c0
    style F fill:#e8f5e9,stroke:#2e7d32

Summary #

  • Terraform is a provisioning tool, not a configuration management tool — creating cloud resources is its domain, configuring server contents isn’t.
  • remote-exec provisioners create idempotency, security (needs open SSH), and visibility (not in the plan output) problems.
  • file provisioners don’t track changes in state — changed configuration isn’t detected by terraform plan.
  • Separate the pipeline: Terraform for infrastructure, Ansible/Chef/Puppet for server configuration — both can run sequentially in the CI/CD pipeline.
  • user_data is acceptable for minimal bootstrap (installing the CM tool agent) — not for complete server configuration.
  • Immutable infrastructure is the best approach — use Packer to build an already-configured AMI, Terraform only deploys that AMI.

← Previous: Performance Optimization   Next: Anti-Pattern: Over-Complex Module →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact