From dc334e6686bbded8674accd2f9f3cac757303787 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Mon, 8 Apr 2024 15:15:01 -0700 Subject: [PATCH 01/16] Re-order Terraform README steps Co-authored-by: Ryan Ahearn --- terraform/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 40ab78a19..530920433 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -26,6 +26,7 @@ The bootstrap module is used to create an s3 bucket for later terraform runs to 1. Follow instructions under `Use bootstrap credentials` 1. Ensure that `import.sh` includes a line and correct IDs for any resources created 1. Run `./teardown_creds.sh` to remove the space deployer account used to create the s3 bucket +1. Copy `bucket` from `bucket_credentials` output to the backend block of `staging/providers.tf` and `production/providers.tf` ### To make changes to the bootstrap module @@ -40,6 +41,7 @@ The bootstrap module is used to create an s3 bucket for later terraform runs to ### Retrieving existing bucket credentials +1. Run `./import.sh` to pull existing terraform state into the local state 1. Run `./run.sh show` 1. Follow instructions under `Use bootstrap credentials` @@ -52,8 +54,6 @@ The bootstrap module is used to create an s3 bucket for later terraform runs to aws_secret_access_key = ``` -1. Copy `bucket` from `bucket_credentials` output to the backend block of `staging/providers.tf` and `production/providers.tf` - ## SpaceDeployers A [SpaceDeployer](https://cloud.gov/docs/services/cloud-gov-service-account/) account is required to run terraform or From 1f3b69ad530769ad18b906e9b0400ad833db8113 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Mon, 8 Apr 2024 15:28:34 -0700 Subject: [PATCH 02/16] Note "initial setup" section is outdated --- terraform/README.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 530920433..4523ff032 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -6,13 +6,15 @@ Prerequisite: install the `jq` JSON processor: `brew install jq` ## Initial setup +These instructions were used for deploying the project for the first time, years ago. We should not have to perfrom these steps again. They are provided here for reference. + 1. Manually run the bootstrap module following instructions under `Terraform State Credentials` 1. Setup CI/CD Pipeline to run Terraform - 1. Copy bootstrap credentials to your CI/CD secrets using the instructions in the base README - 1. Create a cloud.gov SpaceDeployer by following the instructions under `SpaceDeployers` - 1. Copy SpaceDeployer credentials to your CI/CD secrets using the instructions in the base README + 1. Copy bootstrap credentials to your CI/CD secrets using the instructions in the base README + 1. Create a cloud.gov SpaceDeployer by following the instructions under `SpaceDeployers` + 1. Copy SpaceDeployer credentials to your CI/CD secrets using the instructions in the base README 1. Manually Running Terraform - 1. Follow instructions under `Set up a new environment` to create your infrastructure + 1. Follow instructions under `Set up a new environment` to create your infrastructure ## Terraform State Credentials From f2ef5efb1df140b408f98adc6209eb108d33f84f Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Mon, 8 Apr 2024 15:35:50 -0700 Subject: [PATCH 03/16] References to headings into anchor links --- terraform/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 4523ff032..52b4fc168 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -6,17 +6,17 @@ Prerequisite: install the `jq` JSON processor: `brew install jq` ## Initial setup -These instructions were used for deploying the project for the first time, years ago. We should not have to perfrom these steps again. They are provided here for reference. +These instructions were used for deploying the project for the first time, years ago. We should not have to perform these steps again. They are provided here for reference. -1. Manually run the bootstrap module following instructions under `Terraform State Credentials` +1. Manually run the bootstrap module following instructions under [Terraform State Credentials](#terraform-state-credentials) 1. Setup CI/CD Pipeline to run Terraform 1. Copy bootstrap credentials to your CI/CD secrets using the instructions in the base README - 1. Create a cloud.gov SpaceDeployer by following the instructions under `SpaceDeployers` + 1. Create a cloud.gov SpaceDeployer by following the instructions under [SpaceDeployers](#spacedeployers) 1. Copy SpaceDeployer credentials to your CI/CD secrets using the instructions in the base README 1. Manually Running Terraform - 1. Follow instructions under `Set up a new environment` to create your infrastructure + 1. Follow instructions under [Set up a new environment manually](#set-up-a-new-environment-manually) to create your infrastructure -## Terraform State Credentials +## Terraform state credentials The bootstrap module is used to create an s3 bucket for later terraform runs to store their state in. @@ -25,7 +25,7 @@ The bootstrap module is used to create an s3 bucket for later terraform runs to 1. Run `terraform init` 1. Run `./run.sh plan` to verify that the changes are what you expect 1. Run `./run.sh apply` to set up the bucket and retrieve credentials -1. Follow instructions under `Use bootstrap credentials` +1. Follow instructions under [Use bootstrap credentials](#use-bootstrap-credentials) 1. Ensure that `import.sh` includes a line and correct IDs for any resources created 1. Run `./teardown_creds.sh` to remove the space deployer account used to create the s3 bucket 1. Copy `bucket` from `bucket_credentials` output to the backend block of `staging/providers.tf` and `production/providers.tf` From e054d9b03f6aa39b1f89abd32360bde588a5c7c0 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Mon, 8 Apr 2024 17:11:20 -0700 Subject: [PATCH 04/16] Remove insecure output of creds, adjust README --- terraform/README.md | 38 ++++++++++++++++++++----------------- terraform/bootstrap/main.tf | 4 ---- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 52b4fc168..239865f9c 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -4,6 +4,25 @@ This directory holds the terraform modules for maintaining your complete persist Prerequisite: install the `jq` JSON processor: `brew install jq` +## Retrieving existing bucket credentials + +Assuming [initial setup](#initial-setup) is complete, new developers start here! + +1. Enter the bootstrap module with `cd bootstrap` +1. Run `./import.sh` to pull existing terraform state into the local state +1. Follow instructions under [Use bootstrap credentials](#use-bootstrap-credentials) + +### Use bootstrap credentials + +1. Run `./run.sh show -json`. +1. In the output, locate `access_key_id` and `secret_access_key` within `bucket_credentials`. These values are secret, so, don't share them with anyone or copy them to anywhere online. +1. Add the following to `~/.aws/credentials`: + ``` + [notify-terraform-backend] + aws_access_key_id = + aws_secret_access_key = + ``` + ## Initial setup These instructions were used for deploying the project for the first time, years ago. We should not have to perform these steps again. They are provided here for reference. @@ -22,9 +41,9 @@ The bootstrap module is used to create an s3 bucket for later terraform runs to ### Bootstrapping the state storage s3 buckets for the first time -1. Run `terraform init` +1. Within the `bootstrap` directory, run `terraform init` 1. Run `./run.sh plan` to verify that the changes are what you expect -1. Run `./run.sh apply` to set up the bucket and retrieve credentials +1. Run `./run.sh apply` to set up the bucket 1. Follow instructions under [Use bootstrap credentials](#use-bootstrap-credentials) 1. Ensure that `import.sh` includes a line and correct IDs for any resources created 1. Run `./teardown_creds.sh` to remove the space deployer account used to create the s3 bucket @@ -41,21 +60,6 @@ The bootstrap module is used to create an s3 bucket for later terraform runs to 1. Make your changes 1. Continue from step 2 of the boostrapping instructions -### Retrieving existing bucket credentials - -1. Run `./import.sh` to pull existing terraform state into the local state -1. Run `./run.sh show` -1. Follow instructions under `Use bootstrap credentials` - -#### Use bootstrap credentials - -1. Add the following to `~/.aws/credentials` - ``` - [notify-terraform-backend] - aws_access_key_id = - aws_secret_access_key = - ``` - ## SpaceDeployers A [SpaceDeployer](https://cloud.gov/docs/services/cloud-gov-service-account/) account is required to run terraform or diff --git a/terraform/bootstrap/main.tf b/terraform/bootstrap/main.tf index 625cb8093..2394f4ab8 100644 --- a/terraform/bootstrap/main.tf +++ b/terraform/bootstrap/main.tf @@ -14,7 +14,3 @@ resource "cloudfoundry_service_key" "bucket_creds" { name = "${local.s3_service_name}-access" service_instance = module.s3.bucket_id } - -output "bucket_credentials" { - value = cloudfoundry_service_key.bucket_creds.credentials -} From 150cbeb54f1c58eca5a751d9a3b600c4c45b2079 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Mon, 8 Apr 2024 18:32:20 -0700 Subject: [PATCH 05/16] Explain directory structure --- terraform/README.md | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 239865f9c..1890e409d 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -1,12 +1,10 @@ # Terraform -This directory holds the terraform modules for maintaining your complete persistent infrastructure. - -Prerequisite: install the `jq` JSON processor: `brew install jq` +This directory holds the Terraform modules for maintaining Notify.gov's infrastructure. You can [read about the structure](#structure) or [get set up to develop](#retrieving-existing-bucket-credentials). ## Retrieving existing bucket credentials -Assuming [initial setup](#initial-setup) is complete, new developers start here! +:green_book: Assuming [initial setup](#initial-setup) is complete, new developers start here! 1. Enter the bootstrap module with `cd bootstrap` 1. Run `./import.sh` to pull existing terraform state into the local state @@ -35,11 +33,11 @@ These instructions were used for deploying the project for the first time, years 1. Manually Running Terraform 1. Follow instructions under [Set up a new environment manually](#set-up-a-new-environment-manually) to create your infrastructure -## Terraform state credentials +### Terraform state credentials The bootstrap module is used to create an s3 bucket for later terraform runs to store their state in. -### Bootstrapping the state storage s3 buckets for the first time +#### Bootstrapping the state storage s3 buckets for the first time 1. Within the `bootstrap` directory, run `terraform init` 1. Run `./run.sh plan` to verify that the changes are what you expect @@ -49,7 +47,7 @@ The bootstrap module is used to create an s3 bucket for later terraform runs to 1. Run `./teardown_creds.sh` to remove the space deployer account used to create the s3 bucket 1. Copy `bucket` from `bucket_credentials` output to the backend block of `staging/providers.tf` and `production/providers.tf` -### To make changes to the bootstrap module +#### To make changes to the bootstrap module *This should not be necessary in most cases* @@ -106,7 +104,13 @@ The below steps rely on you first configuring access to the Terraform state in s ## Structure -Each environment has its own module, which relies on a shared module for everything except the providers code and environment specific variables and settings. +The `terraform` directory contains sub-directories (`staging`, `production`, etc.) named for deployment environments. Each of these is a *module*, which is just Terraform's word for a directory with some .tf files in it. Each module governs the infrastructure of the environment for which it is named. This directory structure forms "[bulkheads](https://blog.gruntwork.io/how-to-manage-terraform-state-28f5697e68fa)" which isolate Terraform commands to a single environment, limiting accidental damage. + +The `development` module is rather different from the other environment modules. While the other environments can be used to create (or destroy) cloud resources, the development module mostly just sets up access to pre-existing resources needed for local software development. + +The `bootstrap` directory is not an environment module. Instead, it sets up infrastructure needed to deploy Terraform in any of the environments. If you are new to the project, [this is where you should start](#retrieving-existing-bucket-credentials). Similarly, `shared` is not an environment; this module lends code to all the environments. + +Files within these directories look like this: ``` - bootstrap/ From e228a0265373e130a9dca0700514ef6fb38936e9 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Tue, 9 Apr 2024 11:26:50 -0700 Subject: [PATCH 06/16] Explanation of the purpose of bucket credentials --- terraform/README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/terraform/README.md b/terraform/README.md index 1890e409d..096c360aa 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -4,7 +4,9 @@ This directory holds the Terraform modules for maintaining Notify.gov's infrastr ## Retrieving existing bucket credentials -:green_book: Assuming [initial setup](#initial-setup) is complete, new developers start here! +:green_book: new developers start here! + +Assuming [initial setup](#initial-setup) is complete — which it should be if Notify.gov is online — Terraform state is stored in a remote backend. If you are going to be developing Terraform, you'll need to hook up to this backend: 1. Enter the bootstrap module with `cd bootstrap` 1. Run `./import.sh` to pull existing terraform state into the local state @@ -21,6 +23,8 @@ This directory holds the Terraform modules for maintaining Notify.gov's infrastr aws_secret_access_key = ``` +These credentials will allow Terraform to access the AWS/Cloud.gov bucket in which developers share Terraform state files. + ## Initial setup These instructions were used for deploying the project for the first time, years ago. We should not have to perform these steps again. They are provided here for reference. From 7dbcc1793db90268bdc749c4dd8cb48b10fb2f1d Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Wed, 10 Apr 2024 19:18:29 -0700 Subject: [PATCH 07/16] Add Troubleshooting section, note about local state in dev env --- terraform/README.md | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 096c360aa..0f86112e2 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -4,9 +4,9 @@ This directory holds the Terraform modules for maintaining Notify.gov's infrastr ## Retrieving existing bucket credentials -:green_book: new developers start here! +:green_book: New developers start here! -Assuming [initial setup](#initial-setup) is complete — which it should be if Notify.gov is online — Terraform state is stored in a remote backend. If you are going to be developing Terraform, you'll need to hook up to this backend: +Assuming [initial setup](#initial-setup) is complete — which it should be if Notify.gov is online — Terraform state is stored in a shared remote backend. If you are going to be writing Terraform for any of our deployment environments you'll need to hook up to this backend. (You don't need to do this if you are just writing code for the `development` module, becase it stores state locally on your laptop.) 1. Enter the bootstrap module with `cd bootstrap` 1. Run `./import.sh` to pull existing terraform state into the local state @@ -15,12 +15,12 @@ Assuming [initial setup](#initial-setup) is complete — which it should be ### Use bootstrap credentials 1. Run `./run.sh show -json`. -1. In the output, locate `access_key_id` and `secret_access_key` within `bucket_credentials`. These values are secret, so, don't share them with anyone or copy them to anywhere online. +1. In the output, locate `access_key_id` and `secret_access_key` within the `bucket_creds` resource. These values are secret, so, don't share them with anyone or copy them to anywhere online. 1. Add the following to `~/.aws/credentials`: ``` [notify-terraform-backend] - aws_access_key_id = - aws_secret_access_key = + aws_access_key_id = + aws_secret_access_key = ``` These credentials will allow Terraform to access the AWS/Cloud.gov bucket in which developers share Terraform state files. @@ -144,3 +144,16 @@ In the bootstrap module: - `run.sh` Helper script to set up a space deployer and run terraform. The terraform action (`show`/`plan`/`apply`/`destroy`) is passed as an argument - `teardown_creds.sh` Helper script to remove the space deployer setup as part of `run.sh` - `import.sh` Helper script to create a new local state file in case terraform changes are needed + +## Troubleshooting + +### Expired token + +``` +The token expired, was revoked, or the token ID is incorrect. Please log back in to re-authenticate. +``` +You need to re-authenticate with the Cloud Foundry CLI +``` +cf login -a api.fr.cloud.gov --sso +``` +You may also need to log in again to the Cloud.gov website. From e426402247eac50bd12200cc6dde1f7073ebc2b4 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Wed, 10 Apr 2024 19:30:05 -0700 Subject: [PATCH 08/16] Instruction on switching AWS CLI profiles --- terraform/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/terraform/README.md b/terraform/README.md index 0f86112e2..eae17cb15 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -22,6 +22,7 @@ Assuming [initial setup](#initial-setup) is complete — which it should be aws_access_key_id = aws_secret_access_key = ``` +1. Check which AWS profile you are using with `aws configure list`. If needed, use `export AWS_PROFILE=notify-terraform-backend` to change to profile and credentials you just added. These credentials will allow Terraform to access the AWS/Cloud.gov bucket in which developers share Terraform state files. From a5e7383871ac7d3566900174e9ee3f223ef09eb1 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 11 Apr 2024 15:10:19 -0700 Subject: [PATCH 09/16] Rewrite manual new env instructions to be general workflow --- terraform/README.md | 63 ++++++++++++++++++++++++++++++--------------- 1 file changed, 42 insertions(+), 21 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index eae17cb15..73270129b 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -9,7 +9,7 @@ This directory holds the Terraform modules for maintaining Notify.gov's infrastr Assuming [initial setup](#initial-setup) is complete — which it should be if Notify.gov is online — Terraform state is stored in a shared remote backend. If you are going to be writing Terraform for any of our deployment environments you'll need to hook up to this backend. (You don't need to do this if you are just writing code for the `development` module, becase it stores state locally on your laptop.) 1. Enter the bootstrap module with `cd bootstrap` -1. Run `./import.sh` to pull existing terraform state into the local state +1. Run `./import.sh` to import the bucket containing remote terraform state into your local state 1. Follow instructions under [Use bootstrap credentials](#use-bootstrap-credentials) ### Use bootstrap credentials @@ -24,7 +24,7 @@ Assuming [initial setup](#initial-setup) is complete — which it should be ``` 1. Check which AWS profile you are using with `aws configure list`. If needed, use `export AWS_PROFILE=notify-terraform-backend` to change to profile and credentials you just added. -These credentials will allow Terraform to access the AWS/Cloud.gov bucket in which developers share Terraform state files. +These credentials will allow Terraform to access the AWS/Cloud.gov bucket in which developers share Terraform state files. Now you are ready to develop Terraform using the [Workflow for deployed environments](#workflow-for-deployed-environments). ## Initial setup @@ -36,11 +36,11 @@ These instructions were used for deploying the project for the first time, years 1. Create a cloud.gov SpaceDeployer by following the instructions under [SpaceDeployers](#spacedeployers) 1. Copy SpaceDeployer credentials to your CI/CD secrets using the instructions in the base README 1. Manually Running Terraform - 1. Follow instructions under [Set up a new environment manually](#set-up-a-new-environment-manually) to create your infrastructure + 1. Follow instructions under [Workflow for deployed environments](#workflow-for-deployed-environments) to create your infrastructure ### Terraform state credentials -The bootstrap module is used to create an s3 bucket for later terraform runs to store their state in. +The bootstrap module is used to create an s3 bucket for later terraform runs to store their state in. (If the bucket is already created, you should [Use bootstrap credentials](#use-bootstrap-credentials)) #### Bootstrapping the state storage s3 buckets for the first time @@ -70,43 +70,64 @@ deploy the application from the CI/CD pipeline. Create a new account by running: `./create_service_account.sh -s -u ` -## Set up a new environment manually +## Workflow for deployed environments -The below steps rely on you first configuring access to the Terraform state in s3 as described in [Terraform State Credentials](#terraform-state-credentials). +These are the steps for developing Terraform code for our deployed environment modules (`sandbox`, `demo`, `staging` and `production`) locally on your laptop. Or for setting a new deployment environment, or otherwise for running Terraform manually in any module that uses remote state. You don't need to do all this to run code in the `development` module, because it is not a deployed environment and it does not use remote state. -1. `cd` to the environment you are working in +:skull: Note that there is one risky step below (`apply`) which is safe only in the `sandbox` environment and **should not** be run in any other deployed environment. -1. Set up a SpaceDeployer +These steps assume shared [Terraform state credentials](#terraform-state-credentials) exist in s3, and that you are [Using those credentials](#use-bootstrap-credentials). + +1. `cd` to the environment you plan to work in. When developing new features/resources, try out your code in `sandbox`. Only once the code is proven should you copy-and-paste it to each higher environment. + +1. Run `cf spaces` and, from the output, copy the space name for the environment you are working in, such as `notify-sandbox`. + +1. Next you will set up a SpaceDeployer. Prepare to fill in these values: + * `` will be the string you copied from the prior step + * `` can be anything, although we recommend something that communicates the purpose of the deployer. For example: "circleci-deployer" for the credentials CircleCI uses to deploy the application, or "sandbox-" for credentials to run terraform manually. + Put those two values into this command: ```bash - # create a space deployer service instance that can log in with just a username and password - # the value of < SPACE_NAME > should be `staging` or `prod` depending on where you are working - # the value for < ACCOUNT_NAME > can be anything, although we recommend - # something that communicates the purpose of the deployer - # for example: circleci-deployer for the credentials CircleCI uses to - # deploy the application or -terraform for credentials to run terraform manually ./create_service_account.sh -s -u > secrets.auto.tfvars ``` - The script will output the `username` (as `cf_user`) and `password` (as `cf_password`) for your ``. Read more in the [cloud.gov service account documentation](https://cloud.gov/docs/services/cloud-gov-service-account/). + The script will output the `username` (as `cf_user`) and `password` (as `cf_password`) for your ``. Read more in the [cloud.gov service account documentation](https://cloud.gov/docs/services/cloud-gov-service-account/). Then, the command uses the redirection operator (`>`) to write that output to the `secrets.auto.tfvars` file. Terraform will find the username and password there, and use them as input variables. - The easiest way to use this script is to redirect the output directly to the `secrets.auto.tfvars` file it needs to be used in - -1. Run terraform from your new environment directory with +1. While till in an environment directory, initialize Terraform: ```bash terraform init + ``` + + If this command fails, you may need to run `terraform init -upgrade` to make sure new module versions are picked up. Or, `terraform init -migrate-state` to bump the remote backend. + +1. Then, run Terraform in a non-destructive way: + ```bash terraform plan ``` - If the `terraform init` command fails, you may need to run `terraform init -upgrade` to make sure new module versions are picked up. + This will show you any pending changes that Terraform is ready to make. Now is the time to write any code you are planning to write, re-running `terraform plan` to confirm that the code works as you develop. -1. Apply changes with `terraform apply`. +1. **Only if it is safe to do so**, apply your changes. -1. Remove the space deployer service instance if it doesn't need to be used again, such as when manually running terraform once. + > [!CAUTION] + > Applying changes in the wrong directory can mess up a deployed environment that people are relying on + + Double-check what directory you are in, like with the `pwd` command. You should probably only apply while in the `sandbox` directory / environment. + + Once you are sure it is safe, run: + ```bash + terraform apply + ``` + + This command *will deploy your changes* to the cloud. This is a healthy part of testing your code in the sandbox, or if you are creating a new environment. **Do not** apply in enviornments that people are relying upon. + +1. Remove the space deployer service instance when you are done manually running Terraform. ```bash # and have the same values as used above. ./destroy_service_account.sh -s -u ``` + Optionally, you can also `rm secrets.auto.tfvars` + ## Structure The `terraform` directory contains sub-directories (`staging`, `production`, etc.) named for deployment environments. Each of these is a *module*, which is just Terraform's word for a directory with some .tf files in it. Each module governs the infrastructure of the environment for which it is named. This directory structure forms "[bulkheads](https://blog.gruntwork.io/how-to-manage-terraform-state-28f5697e68fa)" which isolate Terraform commands to a single environment, limiting accidental damage. From 35bb04f082e1fccbcc8c3a92cc274585356cd804 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 11 Apr 2024 15:29:36 -0700 Subject: [PATCH 10/16] Note about risk of saving shared code --- terraform/README.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 73270129b..6ce0cee6e 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -72,7 +72,7 @@ deploy the application from the CI/CD pipeline. Create a new account by running: ## Workflow for deployed environments -These are the steps for developing Terraform code for our deployed environment modules (`sandbox`, `demo`, `staging` and `production`) locally on your laptop. Or for setting a new deployment environment, or otherwise for running Terraform manually in any module that uses remote state. You don't need to do all this to run code in the `development` module, because it is not a deployed environment and it does not use remote state. +These are the steps for developing Terraform code for our deployed environment modules (`sandbox`, `demo`, `staging` and `production`) locally on your laptop. Or for setting up a new deployment environment, or otherwise for running Terraform manually in any module that uses remote state. You don't need to do all this to run code in the `development` module, because it is not a deployed environment and it does not use remote state. :skull: Note that there is one risky step below (`apply`) which is safe only in the `sandbox` environment and **should not** be run in any other deployed environment. @@ -92,7 +92,7 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent The script will output the `username` (as `cf_user`) and `password` (as `cf_password`) for your ``. Read more in the [cloud.gov service account documentation](https://cloud.gov/docs/services/cloud-gov-service-account/). Then, the command uses the redirection operator (`>`) to write that output to the `secrets.auto.tfvars` file. Terraform will find the username and password there, and use them as input variables. -1. While till in an environment directory, initialize Terraform: +1. While still in an environment directory, initialize Terraform: ```bash terraform init ``` @@ -104,7 +104,7 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent terraform plan ``` - This will show you any pending changes that Terraform is ready to make. Now is the time to write any code you are planning to write, re-running `terraform plan` to confirm that the code works as you develop. + This will show you any pending changes that Terraform is ready to make. Now is the time to write any HCL code you are planning to write, re-running `terraform plan` to confirm that the code works as you develop. 1. **Only if it is safe to do so**, apply your changes. @@ -118,7 +118,7 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent terraform apply ``` - This command *will deploy your changes* to the cloud. This is a healthy part of testing your code in the sandbox, or if you are creating a new environment. **Do not** apply in enviornments that people are relying upon. + This command *will deploy your changes* to the cloud. This is a healthy part of testing your code in the sandbox, or if you are creating a new environment (a new directory). **Do not** apply in environments that people are relying upon. 1. Remove the space deployer service instance when you are done manually running Terraform. ```bash @@ -134,7 +134,12 @@ The `terraform` directory contains sub-directories (`staging`, `production`, etc The `development` module is rather different from the other environment modules. While the other environments can be used to create (or destroy) cloud resources, the development module mostly just sets up access to pre-existing resources needed for local software development. -The `bootstrap` directory is not an environment module. Instead, it sets up infrastructure needed to deploy Terraform in any of the environments. If you are new to the project, [this is where you should start](#retrieving-existing-bucket-credentials). Similarly, `shared` is not an environment; this module lends code to all the environments. +The `bootstrap` directory is not an environment module. Instead, it sets up infrastructure needed to deploy Terraform in any of the environments. If you are new to the project, [this is where you should start](#retrieving-existing-bucket-credentials). + +Similarly, `shared` is not an environment. It is a module that lends code to all the environments. Please note that changes to `shared` codebase will be applied to all envrionments the next time CI/CD (or a user) runs Terraform in that environment. + +> [!WARNING] +> Editing `shared` code is risky because it will be applied to production Files within these directories look like this: From 3c141ef82359c732dc8fd0f2951172740d45348a Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 11 Apr 2024 15:52:21 -0700 Subject: [PATCH 11/16] Does the Caution markdown not work? --- terraform/README.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 6ce0cee6e..0220c215b 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -22,7 +22,7 @@ Assuming [initial setup](#initial-setup) is complete — which it should be aws_access_key_id = aws_secret_access_key = ``` -1. Check which AWS profile you are using with `aws configure list`. If needed, use `export AWS_PROFILE=notify-terraform-backend` to change to profile and credentials you just added. +1. Check which AWS profile you are using with `aws configure list`. If needed, use `export AWS_PROFILE=notify-terraform-backend` to change to the profile and credentials you just added. These credentials will allow Terraform to access the AWS/Cloud.gov bucket in which developers share Terraform state files. Now you are ready to develop Terraform using the [Workflow for deployed environments](#workflow-for-deployed-environments). @@ -85,12 +85,15 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent 1. Next you will set up a SpaceDeployer. Prepare to fill in these values: * `` will be the string you copied from the prior step * `` can be anything, although we recommend something that communicates the purpose of the deployer. For example: "circleci-deployer" for the credentials CircleCI uses to deploy the application, or "sandbox-" for credentials to run terraform manually. + Put those two values into this command: ```bash ./create_service_account.sh -s -u > secrets.auto.tfvars ``` - The script will output the `username` (as `cf_user`) and `password` (as `cf_password`) for your ``. Read more in the [cloud.gov service account documentation](https://cloud.gov/docs/services/cloud-gov-service-account/). Then, the command uses the redirection operator (`>`) to write that output to the `secrets.auto.tfvars` file. Terraform will find the username and password there, and use them as input variables. + The script will output the `username` (as `cf_user`) and `password` (as `cf_password`) for your ``. The [cloud.gov service account documentation](https://cloud.gov/docs/services/cloud-gov-service-account/) has more information. + + The command uses the redirection operator (`>`) to write that output to the `secrets.auto.tfvars` file. Terraform will find the username and password there, and use them as input variables. 1. While still in an environment directory, initialize Terraform: ```bash @@ -104,11 +107,13 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent terraform plan ``` - This will show you any pending changes that Terraform is ready to make. Now is the time to write any HCL code you are planning to write, re-running `terraform plan` to confirm that the code works as you develop. + This will show you any pending changes that Terraform is ready to make. + + :pencil: Now is the time to write any HCL code you are planning to write, re-running `terraform plan` to confirm that the code works as you develop. Keep in mind that any changes to the codebase that you commit will be run by the CI/CD pipeline. 1. **Only if it is safe to do so**, apply your changes. - > [!CAUTION] + > [!WARNING] > Applying changes in the wrong directory can mess up a deployed environment that people are relying on Double-check what directory you are in, like with the `pwd` command. You should probably only apply while in the `sandbox` directory / environment. From 7e9a504a641f363739031ce95254994484469926 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 11 Apr 2024 15:56:22 -0700 Subject: [PATCH 12/16] I guess you just cant indent cautions / warnings --- terraform/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 0220c215b..8b57a9d41 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -74,7 +74,8 @@ deploy the application from the CI/CD pipeline. Create a new account by running: These are the steps for developing Terraform code for our deployed environment modules (`sandbox`, `demo`, `staging` and `production`) locally on your laptop. Or for setting up a new deployment environment, or otherwise for running Terraform manually in any module that uses remote state. You don't need to do all this to run code in the `development` module, because it is not a deployed environment and it does not use remote state. -:skull: Note that there is one risky step below (`apply`) which is safe only in the `sandbox` environment and **should not** be run in any other deployed environment. +> [!CAUTION] +> There is one risky step below (`apply`) which is safe only in the `sandbox` environment and **should not** be run in any other deployed environment. These steps assume shared [Terraform state credentials](#terraform-state-credentials) exist in s3, and that you are [Using those credentials](#use-bootstrap-credentials). @@ -113,8 +114,7 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent 1. **Only if it is safe to do so**, apply your changes. - > [!WARNING] - > Applying changes in the wrong directory can mess up a deployed environment that people are relying on + :skull: Applying changes in the wrong directory can mess up a deployed environment that people are relying on Double-check what directory you are in, like with the `pwd` command. You should probably only apply while in the `sandbox` directory / environment. From d2d39b210b224d953b24f92f53f0e732cdd72d1f Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Fri, 12 Apr 2024 12:07:37 -0700 Subject: [PATCH 13/16] Note about the purpose of `deployers` in main.tf --- terraform/README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/terraform/README.md b/terraform/README.md index 8b57a9d41..4f8a971a0 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -1,6 +1,10 @@ # Terraform -This directory holds the Terraform modules for maintaining Notify.gov's infrastructure. You can [read about the structure](#structure) or [get set up to develop](#retrieving-existing-bucket-credentials). +This directory holds the Terraform modules for maintaining Notify.gov's API infrastructure. You can might want to: +* [read about the directory structure](#structure), or +* [get set up to develop HCL code](#retrieving-existing-bucket-credentials). + +The Admin app repo [has its own terraform directory](https://github.com/GSA/notifications-admin/tree/main/terraform) but a lot of the below instructions apply to both apps. ## Retrieving existing bucket credentials @@ -70,6 +74,8 @@ deploy the application from the CI/CD pipeline. Create a new account by running: `./create_service_account.sh -s -u ` +SpaceDeployers are also needed to run Terraform locally — they fill user and password input variables (via `deployers` within `main.tf`) that some of our Terraform modules require when they start running. Using a SpaceDeployer account locally is covered in [the next section](#workflow-for-deployed-environments). + ## Workflow for deployed environments These are the steps for developing Terraform code for our deployed environment modules (`sandbox`, `demo`, `staging` and `production`) locally on your laptop. Or for setting up a new deployment environment, or otherwise for running Terraform manually in any module that uses remote state. You don't need to do all this to run code in the `development` module, because it is not a deployed environment and it does not use remote state. From cdf87c02d1b48e21987597e019a757b510bdeae7 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Fri, 12 Apr 2024 14:43:10 -0700 Subject: [PATCH 14/16] Update terraform/README.md Co-authored-by: Carlo Costino --- terraform/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/terraform/README.md b/terraform/README.md index 4f8a971a0..b7488e99d 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -1,6 +1,6 @@ # Terraform -This directory holds the Terraform modules for maintaining Notify.gov's API infrastructure. You can might want to: +This directory holds the Terraform modules for maintaining Notify.gov's API infrastructure. You might want to: * [read about the directory structure](#structure), or * [get set up to develop HCL code](#retrieving-existing-bucket-credentials). From fdfb7e56e88cc61fc5d87458a6aa48b09186b97d Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Fri, 12 Apr 2024 14:43:23 -0700 Subject: [PATCH 15/16] Update terraform/README.md Co-authored-by: Carlo Costino --- terraform/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/terraform/README.md b/terraform/README.md index b7488e99d..27b4042f3 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -10,7 +10,7 @@ The Admin app repo [has its own terraform directory](https://github.com/GSA/noti :green_book: New developers start here! -Assuming [initial setup](#initial-setup) is complete — which it should be if Notify.gov is online — Terraform state is stored in a shared remote backend. If you are going to be writing Terraform for any of our deployment environments you'll need to hook up to this backend. (You don't need to do this if you are just writing code for the `development` module, becase it stores state locally on your laptop.) +Assuming [initial setup](#initial-setup) is complete — which it should be if Notify.gov is online — Terraform state is stored in a shared remote backend. If you are going to be writing Terraform for any of our deployment environments you'll need to hook up to this backend. (You don't need to do this if you are just writing code for the `development` module, because it stores state locally on your laptop.) 1. Enter the bootstrap module with `cd bootstrap` 1. Run `./import.sh` to import the bucket containing remote terraform state into your local state From 2cee820e743382e12c8cd274954fd67ef1151be0 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Fri, 12 Apr 2024 14:44:48 -0700 Subject: [PATCH 16/16] Update terraform/README.md Co-authored-by: Carlo Costino --- terraform/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/terraform/README.md b/terraform/README.md index 27b4042f3..1d75967f0 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -19,7 +19,7 @@ Assuming [initial setup](#initial-setup) is complete — which it should be ### Use bootstrap credentials 1. Run `./run.sh show -json`. -1. In the output, locate `access_key_id` and `secret_access_key` within the `bucket_creds` resource. These values are secret, so, don't share them with anyone or copy them to anywhere online. +1. In the output, locate `access_key_id` and `secret_access_key` within the `bucket_creds` resource. These values are secret, so don't share them with anyone or copy them to anywhere online. 1. Add the following to `~/.aws/credentials`: ``` [notify-terraform-backend]