Securely Storing Environment Variables with AWS
Author
Zach Baker
Date Published
Introduction
Securely storing secrets is an important challenge when deploying production applications. It's a bad idea to store them in version control, so how do you configure secrets on a remote machine without having to manually ssh into the machine and set them manually.
Prerequisites
There are a few things you'll need if you plan on following along with this guide:
- An AWS account with adminstrator access
- An EC2 instance with the AWS CLI installed
- An application which requires environment variables
- A deployment pipeline using AWS CodeDeploy
Building
Secrets Manager
The first step in building this process is to create our secret in the AWS console.
Start by navigating to the Secrets Manager dashboard in the AWS console. You should see a button labeled "store a new secret" at the top right corner. Click this button to get started.
The first option you'll notice is the secret type. Since we aren't making a database credential, we'll choose "other type of secret" from the list.
Once you've clicked this, you should see a section labeled "key/value pairs". For each environment variable you need to store in secrets manager, click "add row" and place the name of the environment variable in the first column and its value in the second column.
You may customize your encryption key or leave it as the default "aws/secretsmanager".
Select next to move to the next page.
Give your secret a memorable name and then click next.
We don't need credential rotation so we can click next again.
Finally, review your options and click "store" to save your secret.
You should be redirected back to the secrets list. You may need to reload to see your new secret. Click on the name of your secret to open it. In the secret details section, copy your secret ARN and save it for later.
You can edit your secret at any point by navigating to this page, clicking "retrieve secret" and then "edit".
IAM
In order for our EC2 instance to access our secret, we need to create a role in IAM. Go to the IAM dashbaord to get started.
Once you're in the IAM dashboard, select roles in the left sidebar.
Creating a new Role
If you already have a role attached to your EC2 instance, select it and skip to the adding an inline policy section.
To create a new role, select "create role" in the top right corner.
Since we're binding this to EC2, we select "AWS service" for the trusted entity type, EC2 for both the "service" and the "use case", then click next.
For now, we don't need to attach a policy because we'll create an inline policy later. Click next.
In the role details section, provide a name for the role. This can be anything, but remember it for later. Click create role. You should be redirected back to the roles dashboard.
Adding an Inline Policy
Click the role you created (or already have).
We need to create an inline policy to allow access to only the secret we created.
Click add permissions then create inline policy in the permissions policy section.
In the policy editor, click where it says choose a service and select secrets manager from the list. Open the read dropdown and check the box next to GetSecretValue.
A new section labeled resources should be available now. Inside this section, make sure you click the bubble next to specific and click add ARNs. Paste the ARN you copied from your secret earlier in this tutorial into the box labeled resource ARN. Click the add ARNs button. Now you should see the ARN listed under the resources section. Click next.
Now you can name your policy. You can name this anything you want. Finally click create policy.
EC2
In order for our EC2 instance to access our secret, we need to make sure it has the role we attached our policy to.
Setting EC2 role
Start by going to the EC2 dashboard. Select instances in the left sidebar. Click on the instance ID of the instance you'd like to use.
Once on your instance, select actions > security > modify IAM role in the top right corner. From the dropdown, select the role you attached your policy to and click update IAM role to save.
Setting system environment variables
Note: This tutorial assumes you're using bash. If you aren't using bash. If you aren't using bash these steps will not work.
It's bad practice to store ARNs in your git repository, but your server does need the ARN of your secret in order to access it. The easiest way to add this environment variable is to add it to the top of your ~/.bashrc file. You can edit this file by typing nano ~/.bashrc from an ssh or ssm session to your EC2 instance.
At the top, add the following line, changing the environment variable name and secret arn to match your own:
export APPLICATION_SECRET_ARN=arn:aws:secretsmanager:us-east-1:xxxxxxxxxxxx:secret:xxxxxxxxxxxx
If you don't put this line towards the top of the file, it may not get loaded as calling a bash script is considered non-interactive in some use cases. Most ~/.bashrc files have a if statement that exits for non-interactive shells.
Deployment Pipeline
Everything is ready for us to edit our pipeline. It only takes a few commands to retrieve, save, and convert our secret into an environment variable file.
This tutorial assumes you're using AWS CodeDeploy, however this process can be adapted for other build pipelines.
BeforeInstall
In your BeforeInstall script, we need to make sure we install a dependency to convert json files to env files. Add these lines wherever they make sense in your BeforeInstall script:
sudo apt-get install npm -y
sudo npm i -g convert-json-env
The first command installs NPM (Node Package Manager) and the second installs the convert-json-env package.
AfterInstall
After installing our package, we want to fetch our secrets and convert them to environment variables.
You should add the following lines to your AfterInstall script:
aws --region us-east-1 secretsmanager get-secret-value --secret-id "$APPLICATION_SECRET_ARN" --query SecretString --output text > secrets.json
convert-json-env secrets.json --out .env
Make sure to change your region and the environment variable name in the aws command. If you are using a file other than .env change .env to the name of your environment variable file.
Authentication is provided by the EC2 instance role and your EC2 instance can retrieve the secrets from Secrets Manager. There is no manual configuration required for configuring the AWS CLI.
Conclusion
For only $0.40 a month, you've stored all of your environment variables securely in AWS Secrets Manager. To update your environment variables, simple edit the values from the Secrets Manager dashboard and retrigger a deployment.
Credits
Photo by Markus Spiske on Unsplash.