Store app secrets on ENV vars using AWS Secrets Manager and Terraform.

Jose López
February 12th, 2020 · 2 min read

Environment variables are used to avoid storage of app secrets in code or in local configuration files. Environment variables override configuration values for all previously specified configuration sources.

AWS Secrets Manager is an AWS service that helps you protect secrets needed to access your applications, services, and IT resources. Users and applications retrieve secrets with a call to Secrets Manager APIs, eliminating the need to hardcode sensitive information in plain text.

By combining AWS Secret manager and environment variables, we can find an easy and secure way to manage application secrets. In this article, we are going to see how to do this and manage all the code using Terraform.

Create secrets and parameters on AWS Secrets Manager using Terraform

AWS Secrets Manager pricing is $0.40 per secret and $0.05 per 10,000 API calls. If you store one value per secret, and your application has multiple secrets, this can have a big impact on your billing. However, in one secret you can store multiple key/value pairs using JSON, and this is what we are going to do to store our application secrets.

So, the first thing we need to do is creating the secrets on AWS Secrets Manager. To do this, we are going to use Terraform, although we will manage the values outside of Terraform (using AWS Console).

1resource "aws_secretsmanager_secret" "app_secrets" {
2 name = "app-secrets"
3 kms_key_id = "${var.kms_arn}" # If not specified, it will use the default `aws/secretsmanager` KMS key
4}

This is all we need, then, we need to use AWS Console to add key/value pairs containing our application secrets.

It’s worth noting that if you only want to set a secret with a single value, you can pass it directly to the ECS task and it will be automatically set as an environment variable, but this doesn’t work for multiple key/value pairs.

If we want to manage secret values using Terraform, we need to create also a secret_version and map the values from Terraform variables. This might be interesting for variables that depend on Terraform, such as other AWS services IDs.

1locals {
2 app_parameters = {
3 VAR_1 = "${var.var1}"
4 VAR_2 = "${var.var2}"
5 }
6
7}
8
9# Secrets
10resource "aws_secretsmanager_secret" "app_parameters" {
11 name = "app-parameters"
12}
13
14# Versions
15resource "aws_secretsmanager_secret_version" "worker_parameters" {
16 secret_id = "${aws_secretsmanager_secret.app_parameters.id}"
17 secret_string = "${jsonencode(local.app_parameters)}"
18}

After secrets are created, we need to grant permissions to get these secrets to our application IAM Role.

1statement {
2 actions = [
3 "secretsmanager:GetSecretValue",
4 ]
5
6 resources = [
7 "${aws_secretsmanager_secret.app_secrets.arn}"
8 ]
9 }

We are also going to pass an environment variable to our application (deployed on ECS) containing the secret ID of the secret we created above (we will use this variable later). So, on the container definitions file, we add the following statement.

1"environment": [
2 {
3 "name": "AWS_SECRET_ID",
4 "value": "${aws_secretsmanager_secret.app_secrets.name}"
5 }
6 ]

Retrieve secrets from AWS Secrets Manager and set them as environment variables on your application

Now that we have application secrets on AWS Secrets Manager, we need to retrieve them on the application and set them as environment variables.

To do this, we are going to use awscli and jq.

So, the first thing we need is to install those tools on our application Docker image.

Once they’re installed, we are going to modify the Docker entrypoint of the application and add the following lines.

1#!/bin/sh
2
3if [ -n "$AWS_SECRET_ID" ]
4then
5 aws secretsmanager get-secret-value --secret-id ${AWS_SECRET_ID} --query SecretString --output text | jq -r 'to_entries|map("\(.key)=\(.value|tostring)")|.[]' > /tmp/secrets.env
6 eval $(cat /tmp/secrets.env | sed 's/^/export /')
7 rm -f /tmp/secrets.env
8fi

The above code uses awscli to get the secret values, then it uses jq to parse them and create a file with the format <KEY>=<VALUE>. Then, using eval and sed commands we use that file to set up environment variables.

And that’s it! We have environment variables configured for our application secrets, and we can use them to set up application variables accordingly.

More articles from Obytes

Manage logs of a Python app on AWS Fargate using Datadog Logs and CloudWatch.

How-to configure a Python (Django) on AWS Fargate app to send logs to both CloudWatch and Datadog using FireLens.

December 30th, 2019 · 3 min read

Monitoring external services status using RSS and Slack.

Monitoring external services (AWS, Cloudflare, DockerHub) and report status to Slack.

December 19th, 2019 · 3 min read

ABOUT US

Our mission and ambition is to challenge the status quo, by doing things differently we nurture our love for craft and technology allowing us to create the unexpected.