Terragrunt - Dealing with AWS Infrastructure State
Published: August 20, 2023 Author:
We require an AWS S3 bucket to serve as the state backend for all our resources, including the bucket itself. This bucket should have both versioning and server-side encryption enabled. For the encryption, we'll need an AWS KMS key. Additionally, we'll set up a DynamoDB table for central locking.
We have two options:
- Create a custom module tailored to our requirements.
- Utilize existing modules available online and stitch them together using Terragrunt dependencies.
Given the distinct requirements (S3 bucket, KMS key, DynamoDB table), it would be logical to separate each of these into individual Terraform modules. Terragrunt's dependency or dependencies block ensures resources are created in the right order, given their dependencies. For example, you would want the KMS key created before the S3 bucket, because the bucket relies on the key for encryption.
Updated Directory Structure
We will be updating our directory structure. Additionally, I'll make some revisions to the configuration files mentioned in my previous blog post.
|-- modules
|-- single-account
|-- environments
|-- _common
| |-- kms.hcl
| `-- s3_bucket.hcl
|-- global
| |-- state_bucket
| | `-- terragrunt.hcl
| |-- state_bucket_kms_key
| | `-- terragrunt.hcl
| |-- state_locking_table
| | `-- terragrunt.hcl
| `-- environment.yaml
|-- common.yaml
`-- terragrunt.hcl
Changes to Existing Configuration
Let's begin by addressing the revision made to the existing configuration files i.e. common.yaml and terragrunt.yaml under single-account/environments directory. As mentioned in the previous post, the common.yaml file included parameters for the S3 bucket. These will be redefined under single-account/environments/global/environment.yaml. I will also add default module parameters to common.yaml:
single-account/environments/common.yaml
terraform:
remote_modules:
s3:
source: git::git@github.com:terraform-aws-modules/terraform-aws-s3-bucket.git///
version: v3.14.1
kms:
source: git::git@github.com:terraform-aws-modules/terraform-aws-kms.git///
version: v1.5.0
dynamodb:
source: git::git@github.com:terraform-aws-modules/terraform-aws-dynamodb-table.git///
version: v3.3.0
aws:
region: us-east-1
tags:
Managed-By: Terragrunt
In the terragrunt.hcl file located in the single-account/environments directory, I previously included each environment.yaml configuration file I planned to create. This was to ensure that tags from those environments were set as default AWS tags. However, I'm changing this approach. Instead, I'll source the default tags from common.yaml and handle tags individually within each environment. Additionally, I'll be adding a remote state configuration to this file, though it will be commented out for the time being.
single-account/environments/terragrunt.hcl
locals {
common_vars = yamldecode(file("common.yaml"))
state_bucket_name = yamldecode(file("global/environment.yaml")).module_config.state_bucket.bucket
dynamodb_table = yamldecode(file("global/environment.yaml")).module_config.state_locking_table.name
}
# Configure Terragrunt to store state files in an S3 bucket
# remote_state {
# backend = "s3"
# config = {
# encrypt = true
# bucket = local.state_bucket_name
# key = "${path_relative_to_include()}/state.json"
# region = local.common_vars.aws.region
# dynamodb_table = local.dynamodb_table
# }
# generate = {
# path = "backend.tf"
# if_exists = "overwrite_terragrunt"
# }
# }
# Setup provider
generate "provider" {
path = "provider.tf"
if_exists = "overwrite"
contents = <<EOF
variable "aws_provider_default_tags" {
type = map
}
provider "aws" {
region = "${local.common_vars.aws.region}"
default_tags {
tags = var.aws_provider_default_tags
}
}
EOF
}
inputs = {
aws_provider_default_tags = local.common_vars.tags
}
The Global Environment
The global directory under single-account/environments houses modules that are common among all resources. Within it, the single-account/environments/global/environment.yaml specifies input parameters for these global modules.
single-account/environments/global/environment.yaml
tags: &tags
Environment: Global
module_config:
state_bucket_kms_key:
description: "AWS KMS key for Terraform state bucket."
enable_key_rotation: false
aliases:
- KMS-Key-S3-State
tags:
<<: *tags
state_bucket:
bucket: nixknight-terragrunt-demo-state
versioning:
status: true
tags:
<<: *tags
state_locking_table:
name: terragrunt-state-lock-table
hash_key: LockID
table_class: STANDARD
billing_mode: PAY_PER_REQUEST
attributes:
- { name: LockID, type: S }
tags:
<<: *tags
The file single-account/environments/global/state_bucket_kms_key/terragrunt.hcl file references the kms.hcl from the single-account/environments/_common/ directory, which establishes both common and environment-specific locals. The terraform {} block within the file determines the module to be used.
single-account/environments/_common/kms.hcl
locals {
common_vars = yamldecode(file("${find_in_parent_folders()}/../common.yaml")).terraform.remote_modules.kms
env_vars = yamldecode(file("${path_relative_to_include()}/../environment.yaml"))
module_source_url = local.common_vars.source
module_version = local.common_vars.version
}
terraform {
source = "${local.module_source_url}?ref=${local.module_version}"
}
single-account/environments/global/state_bucket_kms_key/terragrunt.hcl
include {
path = find_in_parent_folders()
}
include "kms" {
path = "../../_common/kms.hcl"
expose = true
}
locals {
module_vars = include.kms.locals.env_vars.module_config.state_bucket_kms_key
}
inputs = merge(local.module_vars)
The initial include block in the file searches for a terragrunt.hcl in its parent directories. This action retrieves the common provider and state configuration from the root terragrunt.hcl file located in single-account/environments/.
The subsequent include "kms" {} block additionally sets expose = true. It allows the child terragrunt.hcl file to reference any of the configurations (like inputs, locals, etc.) from the parent terragrunt.hcl file. Without this attribute, or if it's set to false, the child configuration can't directly reference or see those parent configurations.
This approach is consistently applied to other modules.
single-account/environments/_common/dynamodb.hcl
locals {
common_vars = yamldecode(file("${find_in_parent_folders()}/../common.yaml")).terraform.remote_modules.dynamodb
env_vars = yamldecode(file("${path_relative_to_include()}/../environment.yaml"))
module_source_url = local.common_vars.source
module_version = local.common_vars.version
}
terraform {
source = "${local.module_source_url}?ref=${local.module_version}"
}
single-account/environments/global/state_locking_table/terragrunt.hcl
include {
path = find_in_parent_folders()
}
include "dynamodb" {
path = "../../_common/dynamodb.hcl"
expose = true
}
locals {
module_vars = include.dynamodb.locals.env_vars.module_config.state_locking_table
}
inputs = merge(local.module_vars)
single-account/environments/_common/s3_bucket.hcl
locals {
common_vars = yamldecode(file("${find_in_parent_folders()}/../common.yaml")).terraform.remote_modules.s3
env_vars = yamldecode(file("${path_relative_to_include()}/../environment.yaml"))
module_source_url = local.common_vars.source
module_version = local.common_vars.version
}
terraform {
source = "${local.module_source_url}?ref=${local.module_version}"
}
single-account/environments/global/state_bucket/terragrunt.hcl
include {
path = find_in_parent_folders()
}
include "s3_bucket" {
path = "../../_common/s3_bucket.hcl"
expose = true
}
dependency "state_bucket_kms_key" {
config_path = "../state_bucket_kms_key"
}
locals {
module_vars = include.s3_bucket.locals.env_vars.module_config.state_bucket
}
inputs = merge(
local.module_vars,
{
server_side_encryption_configuration = {
rule = {
apply_server_side_encryption_by_default = {
kms_master_key_id = "${dependency.state_bucket_kms_key.outputs.key_arn}"
sse_algorithm = "aws:kms"
}
}
}
}
)
In Terragrunt, we can set up a dependency to create a connection between modules. For instance, our S3 state bucket module relies on the KMS key module for a master key ID, which is the ARN of the KMS key. Once the KMS module creates this key, it exports its ARN as an output. We can then seamlessly reference this ARN in another module using dependency.state_bucket_kms_key.outputs.key_arn. This is the aspect of Terragrunt that I genuinely appreciate. The ability to reference one module's state from another through the dependency block adds a layer of elegance and efficiency to infrastructure-as-code practices.
Applying Initial Set of Configurations
With our configuration files now in place, we're ready to execute Terragrunt. From the single-account/environments directory, which contains our central configuration, we'll use the terragrunt run-all <COMMAND>. This command concurrently operates on multiple modules, streamlining the management of dependencies and operations across various Terraform configurations within our directory structure. We will replace <COMMAND> with any Terragrunt or Terraform command, like init, plan, apply, or destroy.
Terragrunt provides a lot of output. I'll be clipping some of that.
We'll start by initializing as we do using Terraform:
terragrunt run-all init
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.48.4
INFO[0000] The stack at /home/saadali/Projects/Terragrunt-AWS-Demo will be processed in the following order for command init:
Group 1
- Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key
- Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_locking_table
Group 2
- Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket
Initializing the backend...
...
Terraform has been successfully initialized!
...
ERRO[0039] Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket has finished with an error: /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key/terragrunt.hcl is a dependency of /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket/terragrunt.hcl but detected no outputs. Either the target module has not been applied yet, or the module has no outputs. If this is expected, set the skip_outputs flag to true on the dependency block. prefix=[/home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket]
ERRO[0039] 1 error occurred:
* /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key/terragrunt.hcl is a dependency of /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket/terragrunt.hcl but detected no outputs. Either the target module has not been applied yet, or the module has no outputs. If this is expected, set the skip_outputs flag to true on the dependency block.
ERRO[0039] Unable to determine underlying exit code, so Terragrunt will exit with error code 1
Terragrunt attempted to initialize all modules, but encountered an issue with the state_bucket module because the outputs from state_bucket_kms_key module were not yet available. There are 2 ways to resolve this issue:
- Apply the KMS module using
terragrunt apply -auto-approveafter changing the current working directory fromsingle-account/environmentstosingle-account/environments/global/state_bucket_kms_key. - Introduce a mock output.
Terragrunt's mock outputs are placeholder values for module outputs that are used during specific commands, such as init or plan. This feature allows you to simulate outputs from modules that haven't been applied yet or to test configurations without relying on real infrastructure changes. However, once the modules are applied, Terragrunt seamlessly transitions from these mock outputs, automatically replacing them with the genuine outputs from the applied resources. This ensures that configurations integrate real infrastructure data post initial setups or tests using mock values.
We will now update the dependency block in single-account/environments/global/state_bucket/terragrunt.hcl and introduce mock outputs as follows:
...
dependency "state_bucket_kms_key" {
config_path = "../state_bucket_kms_key"
mock_outputs = {
key_arn = "mock_arn"
}
mock_outputs_allowed_terraform_commands = ["init", "plan"]
}
...
Lets initialize again:
terragrunt run-all init
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.48.4
INFO[0000] The stack at /home/saadali/Projects/Terragrunt-AWS-Demo will be processed in the following order for command init:
Group 1
- Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key
- Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_locking_table
Group 2
- Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket
Initializing the backend...
...
Terraform has been successfully initialized!
...
Now that the modules have been initialized lets apply them directly.
terragrunt run-all apply
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.48.4
INFO[0000] The stack at /home/saadali/Projects/Terragrunt-AWS-Demo will be processed in the following order for command apply:
Group 1
- Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key
- Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_locking_table
Group 2
- Module /home/saadali/Projects/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket
Are you sure you want to run 'terragrunt apply' in each folder of the stack described above? (y/n) y
Initializing the backend...
...
Terraform has been successfully initialized!
...
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
data.aws_caller_identity.current: Reading...
data.aws_partition.current: Reading...
data.aws_partition.current: Read complete after 0s [id=aws]
data.aws_caller_identity.current: Read complete after 1s [id=<ACCOUNT_ID>]
data.aws_iam_policy_document.this[0]: Reading...
data.aws_iam_policy_document.this[0]: Read complete after 0s [id=1088477093]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_kms_alias.this["KMS-Key-S3-State"] will be created
+ resource "aws_kms_alias" "this" {
+ arn = (known after apply)
+ id = (known after apply)
+ name = "alias/KMS-Key-S3-State"
+ name_prefix = (known after apply)
+ target_key_arn = (known after apply)
+ target_key_id = (known after apply)
}
# aws_kms_key.this[0] will be created
+ resource "aws_kms_key" "this" {
+ arn = (known after apply)
+ bypass_policy_lockout_safety_check = false
+ customer_master_key_spec = "SYMMETRIC_DEFAULT"
+ description = "AWS KMS key for Terraform State bucket"
+ enable_key_rotation = false
+ id = (known after apply)
+ is_enabled = true
+ key_id = (known after apply)
+ key_usage = "ENCRYPT_DECRYPT"
+ multi_region = false
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "kms:*"
+ Effect = "Allow"
+ Principal = {
+ AWS = "arn:aws:iam::<ACCOUNT_ID>:root"
}
+ Resource = "*"
+ Sid = "Default"
},
]
+ Version = "2012-10-17"
}
)
+ tags = {
+ "Environment" = "Global"
}
+ tags_all = {
+ "Environment" = "Global"
+ "Managed-By" = "Terragrunt"
}
}
Plan: 2 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ aliases = {
+ KMS-Key-S3-State = {
+ arn = (known after apply)
+ id = (known after apply)
+ name = "alias/KMS-Key-S3-State"
+ name_prefix = (known after apply)
+ target_key_arn = (known after apply)
+ target_key_id = (known after apply)
}
}
+ grants = {}
+ key_arn = (known after apply)
+ key_id = (known after apply)
+ key_policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "kms:*"
+ Effect = "Allow"
+ Principal = {
+ AWS = "arn:aws:iam::<ACCOUNT_ID>:root"
}
+ Resource = "*"
+ Sid = "Default"
},
]
+ Version = "2012-10-17"
}
)
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_dynamodb_table.this[0] will be created
+ resource "aws_dynamodb_table" "this" {
+ arn = (known after apply)
+ billing_mode = "PAY_PER_REQUEST"
+ hash_key = "LockID"
+ id = (known after apply)
+ name = "terragrunt-state-lock-table"
+ read_capacity = (known after apply)
+ stream_arn = (known after apply)
+ stream_enabled = false
+ stream_label = (known after apply)
+ stream_view_type = (known after apply)
+ tags = {
+ "Environment" = "Global"
+ "Name" = "terragrunt-state-lock-table"
}
+ tags_all = {
+ "Environment" = "Global"
+ "Managed-By" = "Terragrunt"
+ "Name" = "terragrunt-state-lock-table"
}
+ write_capacity = (known after apply)
+ attribute {
+ name = "LockID"
+ type = "S"
}
+ point_in_time_recovery {
+ enabled = false
}
+ server_side_encryption {
+ enabled = false
+ kms_key_arn = (known after apply)
}
+ timeouts {
+ create = "10m"
+ delete = "10m"
+ update = "60m"
}
+ ttl {
+ enabled = false
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ dynamodb_table_arn = (known after apply)
+ dynamodb_table_id = (known after apply)
aws_kms_key.this[0]: Creating...
aws_dynamodb_table.this[0]: Creating...
aws_kms_key.this[0]: Still creating... [10s elapsed]
aws_dynamodb_table.this[0]: Still creating... [10s elapsed]
aws_dynamodb_table.this[0]: Creation complete after 12s [id=terragrunt-state-lock-table]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
dynamodb_table_arn = "arn:aws:dynamodb:us-east-1:<ACCOUNT_ID>:table/terragrunt-state-lock-table"
dynamodb_table_id = "terragrunt-state-lock-table"
aws_kms_key.this[0]: Still creating... [20s elapsed]
aws_kms_key.this[0]: Creation complete after 20s [id=<KMS_KEY_ID>]
aws_kms_alias.this["KMS-Key-S3-State"]: Creating...
aws_kms_alias.this["KMS-Key-S3-State"]: Creation complete after 2s [id=alias/KMS-Key-S3-State]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Outputs:
...
Initializing the backend...
...
Terraform has been successfully initialized!
...
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
data.aws_canonical_user_id.this: Reading...
data.aws_region.current: Reading...
data.aws_caller_identity.current: Reading...
data.aws_partition.current: Reading...
data.aws_partition.current: Read complete after 0s [id=aws]
data.aws_region.current: Read complete after 0s [id=us-east-1]
data.aws_canonical_user_id.this: Read complete after 2s [id=305c6d3d4e2313b198fce101d564d29bfd884d06034a9aeb934edb2ae495dde0]
data.aws_caller_identity.current: Read complete after 4s [id=<ACCOUNT_ID>]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_s3_bucket.this[0] will be created
+ resource "aws_s3_bucket" "this" {
+ acceleration_status = (known after apply)
+ acl = (known after apply)
+ arn = (known after apply)
+ bucket = "nixknight-terragrunt-demo-state"
+ bucket_domain_name = (known after apply)
+ bucket_prefix = (known after apply)
+ bucket_regional_domain_name = (known after apply)
+ force_destroy = false
+ hosted_zone_id = (known after apply)
+ id = (known after apply)
+ object_lock_enabled = false
+ policy = (known after apply)
+ region = (known after apply)
+ request_payer = (known after apply)
+ tags = {
+ "Environment" = "Global"
}
+ tags_all = {
+ "Environment" = "Global"
+ "Managed-By" = "Terragrunt"
}
+ website_domain = (known after apply)
+ website_endpoint = (known after apply)
}
# aws_s3_bucket_public_access_block.this[0] will be created
+ resource "aws_s3_bucket_public_access_block" "this" {
+ block_public_acls = true
+ block_public_policy = true
+ bucket = (known after apply)
+ id = (known after apply)
+ ignore_public_acls = true
+ restrict_public_buckets = true
}
# aws_s3_bucket_server_side_encryption_configuration.this[0] will be created
+ resource "aws_s3_bucket_server_side_encryption_configuration" "this" {
+ bucket = (known after apply)
+ id = (known after apply)
+ rule {
+ apply_server_side_encryption_by_default {
+ kms_master_key_id = "arn:aws:kms:us-east-1:<ACCOUNT_ID>:key/<KMS_KEY_ID>"
+ sse_algorithm = "aws:kms"
}
}
}
# aws_s3_bucket_versioning.this[0] will be created
+ resource "aws_s3_bucket_versioning" "this" {
+ bucket = (known after apply)
+ id = (known after apply)
+ versioning_configuration {
+ mfa_delete = (known after apply)
+ status = "Enabled"
}
}
Plan: 4 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ s3_bucket_arn = (known after apply)
+ s3_bucket_bucket_domain_name = (known after apply)
+ s3_bucket_bucket_regional_domain_name = (known after apply)
+ s3_bucket_hosted_zone_id = (known after apply)
+ s3_bucket_id = (known after apply)
+ s3_bucket_lifecycle_configuration_rules = ""
+ s3_bucket_policy = ""
+ s3_bucket_region = (known after apply)
+ s3_bucket_website_domain = ""
+ s3_bucket_website_endpoint = ""
aws_s3_bucket.this[0]: Creating...
aws_s3_bucket.this[0]: Still creating... [10s elapsed]
aws_s3_bucket.this[0]: Creation complete after 13s [id=nixknight-terragrunt-demo-state]
aws_s3_bucket_public_access_block.this[0]: Creating...
aws_s3_bucket_versioning.this[0]: Creating...
aws_s3_bucket_server_side_encryption_configuration.this[0]: Creating...
aws_s3_bucket_public_access_block.this[0]: Creation complete after 1s [id=nixknight-terragrunt-demo-state]
aws_s3_bucket_server_side_encryption_configuration.this[0]: Creation complete after 2s [id=nixknight-terragrunt-demo-state]
aws_s3_bucket_versioning.this[0]: Creation complete after 4s [id=nixknight-terragrunt-demo-state]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Outputs:
...
At this point, we will uncomment the state backend configuration in single-account/environments/terragrunt.hcl adn re-run the init command so that we can move the current state to the S3 bucket. We need to add -input=true to our command so that we can answer yes when Terraform asks us to move state to the new backend.
terragrunt run-all init -input=true
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.48.4
INFO[0000] The stack at /home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments will be processed in the following order for command init:
Group 1
- Module /home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key
- Module /home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_locking_table
Group 2
- Module /home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket
WARN[0003] The remote state S3 bucket nixknight-terragrunt-demo-state needs to be updated: prefix=[/home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_locking_table]
WARN[0003] - Bucket Root Access prefix=[/home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_locking_table]
WARN[0003] - Bucket Enforced TLS prefix=[/home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_locking_table]
Remote state S3 bucket nixknight-terragrunt-demo-state is out of date. Would you like Terragrunt to update it? (y/n) y
WARN[0004] The remote state S3 bucket nixknight-terragrunt-demo-state needs to be updated: prefix=[/home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key]
WARN[0004] - Bucket Root Access prefix=[/home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key]
WARN[0004] - Bucket Enforced TLS prefix=[/home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key]
Remote state S3 bucket nixknight-terragrunt-demo-state is out of date. Would you like Terragrunt to update it? (y/n) y
Initializing the backend...
Initializing the backend...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "s3" backend. No existing state was found in the newly
configured "s3" backend. Do you want to copy this state to the new "s3"
backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value: Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "s3" backend. No existing state was found in the newly
configured "s3" backend. Do you want to copy this state to the new "s3"
backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value: yes
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
...
Terraform has been successfully initialized!
...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
...
We've now established all the required resources and migrated the state files for all modules to an S3 bucket. Moving forward, this S3 bucket will serve as the centralized storage for our infrastructure state.
Final Changes to the State Bucket
In the above output, Terragrunt prompts us to update the S3 bucket with a new configuration. Specifically, it seeks to enforce TLS communication and root account access by adding these to the bucket policy document. I promptly responded with y for expedience. Nonetheless, it's crucial that we adjust our configuration to guarantee that these Terragrunt-driven changes are part of Terraform state.
I have created a policy template specifically for this purpose.
templates/policies/state_bucket_policy.yaml.tftpl
---
Version: '2012-10-17'
Statement:
- Sid: EnforcedTLS
Effect: Deny
Principal: "*"
Action: s3:*
Resource:
- arn:aws:s3:::${bucket_name}
- arn:aws:s3:::${bucket_name}/*
Condition:
Bool:
aws:SecureTransport: 'false'
- Sid: RootAccess
Effect: Allow
Principal:
AWS: arn:aws:iam::${account_id}:root
Action: s3:*
Resource:
- arn:aws:s3:::${bucket_name}
- arn:aws:s3:::${bucket_name}/*
Next we update the inputs section of single-account/environments/global/state_bucket/terragrunt.hcl.
...
inputs = merge(
local.module_vars,
{
server_side_encryption_configuration = {
rule = {
apply_server_side_encryption_by_default = {
kms_master_key_id = "${dependency.state_bucket_kms_key.outputs.key_arn}"
sse_algorithm = "aws:kms"
}
}
}
attach_policy = true
policy = jsonencode(
yamldecode(
templatefile(
"${get_repo_root()}/templates/policies/state_bucket_policy.yaml.tftpl",
{
bucket_name = local.module_vars.bucket
account_id = "${get_aws_account_id()}"
}
)
)
)
}
)
...
And finally we apply.
terragrunt run-all apply
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.48.4
INFO[0000] The stack at /home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments will be processed in the following order for command apply:
Group 1
- Module /home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket_kms_key
- Module /home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_locking_table
Group 2
- Module /home/saadali/Projects/Terraform/Terragrunt/Terragrunt-AWS-Demo/single-account/environments/global/state_bucket
...
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_s3_bucket_policy.this[0] will be created
+ resource "aws_s3_bucket_policy" "this" {
+ bucket = "nixknight-terragrunt-demo-state"
+ id = (known after apply)
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "s3:*"
+ Condition = {
+ Bool = {
+ "aws:SecureTransport" = [
+ "false",
]
}
}
+ Effect = "Deny"
+ Principal = "*"
+ Resource = [
+ "arn:aws:s3:::nixknight-terragrunt-demo-state",
+ "arn:aws:s3:::nixknight-terragrunt-demo-state/*",
]
+ Sid = "EnforcedTLS"
},
+ {
+ Action = "s3:*"
+ Effect = "Allow"
+ Principal = {
+ AWS = "arn:aws:iam::<ACCOUNT_ID>:root"
}
+ Resource = [
+ "arn:aws:s3:::nixknight-terragrunt-demo-state",
+ "arn:aws:s3:::nixknight-terragrunt-demo-state/*",
]
+ Sid = "RootAccess"
},
]
+ Version = "2012-10-17"
}
)
}
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
~ s3_bucket_id = "nixknight-terragrunt-demo-state" -> (known after apply)
~ s3_bucket_policy = "" -> jsonencode(
{
+ Statement = [
+ {
+ Action = "s3:*"
+ Condition = {
+ Bool = {
+ "aws:SecureTransport" = [
+ "false",
]
}
}
+ Effect = "Deny"
+ Principal = "*"
+ Resource = [
+ "arn:aws:s3:::nixknight-terragrunt-demo-state",
+ "arn:aws:s3:::nixknight-terragrunt-demo-state/*",
]
+ Sid = "EnforcedTLS"
},
+ {
+ Action = "s3:*"
+ Effect = "Allow"
+ Principal = {
+ AWS = "arn:aws:iam::<ACCOUNT_ID>:root"
}
+ Resource = [
+ "arn:aws:s3:::nixknight-terragrunt-demo-state",
+ "arn:aws:s3:::nixknight-terragrunt-demo-state/*",
]
+ Sid = "RootAccess"
},
]
+ Version = "2012-10-17"
}
)
aws_s3_bucket_policy.this[0]: Creating...
aws_s3_bucket_policy.this[0]: Creation complete after 2s [id=nixknight-terragrunt-demo-state]
Releasing state lock. This may take a few moments...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
...
With these last changes, our directory structure is updated and it now looks like this:
|-- modules
|-- single-account
| |-- environments
| |-- _common
| | |-- kms.hcl
| | `-- s3_bucket.hcl
| |-- global
| | |-- state_bucket
| | | `-- terragrunt.hcl
| | |-- state_bucket_kms_key
| | | `-- terragrunt.hcl
| | |-- state_locking_table
| | | `-- terragrunt.hcl
| | `-- environment.yaml
| |-- common.yaml
| `-- terragrunt.hcl
|-- templates
|-- policies
`-- state_bucket_policy.yaml.tftpl
Thank you for reaching the conclusion of this blog post. Your feedback is invaluable to me. Please share your thoughts and insights in the comments section below. I look forward to reading them!
Tagged as: Linux Terraform Terragrunt IaC AWS S3 DRY