Bootstrapping an Amazon EC2 instance using user-data to run a Python web app

Manually setting up and configuring the packages required to run a Python web app using Nginx and uWSGI on a server can be time consuming — and it's tough to accomplish without any errors. EC2 instances have the ability to run user data scripts when the instance start up. You can automate creating all the infrastructure and scripts using CDK to configure your instance when it first boots to reduce errors while setting up Nginx and uWSGI. We will be using a bash script to install and configure Nginx and uWSGI, set up a systemd service for uWSGI, and copy our application using CDK. We will cover how to:

• Create an AWS CDK stack with an Amazon EC2 instance, a security group with inbound access, and an IAM instance profile.
• Install software packages on the EC2 instance's first launch by creating a user data asset.
• Configure the software packages after installation using a script downloaded by the user data.
• Deploy the application using user data.

Table of Contents

Introduction

Before we deploy the web application, we need to create an Amazon EC2 instance and supporting resources to run it. In this tutorial, we'll create a new AWS CDK app to create our infrastructure, install the dependencies for our web app, allow access to it, and finally deploy it.

First, let's check if our CDK version is up to date—this guide is based on v2 of the CDK. If you are still using v1, please read through the migration docs. To check the version, run the following:

If you see output showing 1.x.x, or you just want to ensure you are on the latest version, run the following:

We will now create the skeleton CDK application using TypeScript as our language of choice:

Create the code for the resource stack

CDK uses the folder name for the files it generates. For this tutorial, we will be using ec2-cdk. If you named your directory differently, please replace this with the folder name you used. To start adding infrastructure, go to the file lib/ec2-cdk-stack.ts. This is where we will write the code for the resource stack you are going to create.

A resource stack is a set of cloud infrastructure resources (in your particular case, they will be all AWS resources) that will be provisioned into a specific account. The account and Region where these resources are provisioned, can be configured in the stack—we will cover this later on.

In this resource stack, you are going to create the following resources:

• IAM role: This role will be assigned to the EC2 instance to allow it to call other AWS services.
• EC2 instance: The virtual machine you will use to host your web application.
• Security group: The virtual firewall to allow inbound requests to your web application.
• EC2 SSH key pair: A set of credentials you can use to SSH to the instance to execute commands to set everything up.

Create the EC2 instance

To start creating the EC2 instance, and other resources, you first need to import the CDK constructs and EC2 key pair libraries. Run the following command in the ec2-cdk folder to install the libraries:

To use these libraries, we need to import them into our CDK project. Open lib/ec2-cdk-stack.ts to add the dependencies at the top of the file below the existing import statement (you can remove the commented out line for aws-sqs):

During the course of this tutorial, there will be code checkpoints where we show what the full file should look like at that point. We do recommend following step by step by typing out or copying and pasting the sample code blocks to ensure you understand what each code block does.

These modules provide access to all the components you need to deploy the web application. The first step is to find the existing default VPC in your account by adding the following code below the placeholder comment:

Using the cdk-ec2-key-pair package, we will create an SSH key pair and store it in AWS Secrets Manager so we can download it later to be able to SSH to the instance.

We also need to be able to access our instance via two ports: 22 and 80. SSH uses port 22, and we will serve the web app through port 80 (http). To allow traffic to these ports, we need to set up firewall rules by creating a security group. Note that SSH access from all IP addresses is acceptable for a short time in a test environment, but not recommended for production environments - we recommend setting up EC2 Instance Connect for frequent access. We will also create an IAM role for the EC2 instance to allow it to call other AWS services, and attach a pre-built policy to read configurations out of AWS Secrets Manager (where the SSH public key will be stored):

We're now ready to create the EC2 instance using a pre-built Amazon Machine Image (AMI - pronounced "Ay-Em-Eye")—for this tutorial, we will be using the Amazon Linux 2 AMI for X86_64 CPU architecture. We will also pass the IAM role you created, the default VPC, and the instance type to run on, in your case, a t2.micro that has 1 vCPU and 1GB of memory. If you are running this tutorial in one of the newer AWS Regions, the t2.micro type may not be available. Just use the t3.micro one instead. To view all the different instance types, see the EC2 instance types page.

We have now defined our AWS CDK stack to create an EC2 instance, a security group with inbound access rules, and an IAM role, attached to the EC2 instance as an IAM instance profile. Before deploying the stack, we still need to install the packages on the host OS to run your application, and also copy our sample application code to the instance.

✅ ✅ ✅ Checkpoint 1 ✅ ✅ ✅

Your lib/ec2-cdk-stack.ts file should now look like this:

Adding user data to your EC2 instance

We will be using an existing sample Python web app that also includes the required scripts to install the dependencies on the instance to run it. Download a copy to your ec2-cdk folder using the following git command:

The sample web application hosted in the sample-pyton-web-app folder is a Python application that we will be deploying. It requires Nginx and uWSGI to run. To install these components, we need to follow a number of steps. First, we need to install all the OS packages, then configure Nginx and uWSGI, ensure they're are running, and copy the sample application to the instance. A bash script file that configures all of these setup steps is provided in sample-pyton-web-app/configure_amz_linux_sample_app.sh. Have a look at the steps in it if you want to know more about how the instance is configured.

To deploy the web application, we need to add code to our CDK application that will copy the configuration files and scripts to a location that the instance has access to, along with the sample app code—we will use Amazon S3 for this. To do so, add the following code in lib/ec2-cdk-stack.ts below the previous code:

We have one more step before deploying everything: adding output to the CDK stack to print out the command to download the SSH key, SSH command to connect to the instance, and IP address to access the instance on. In the infrastructure above, you created an SSH key, and stored it in AWS Secrets Manager. To print out these outputs, add the following to the bottom of the CDK app:

These three outputs will show you the following:

• How to download the SSH key to access the instance
• The public IP of the instance to use in your browser to see the Python web app running
• An SSH command to access the instance

We're now ready to deploy the stack.

✅ ✅ ✅ Checkpoint 2 ✅ ✅ ✅

We have now completed all code changes to our CDK app, and the lib/ec2-cdk-stack.ts file should look like this:

Bootstrap CDK

Before we can deploy our CDK app, we need to configure CDK on the account you are deploying to. Edit the bin/ec2-cdk.ts and uncomment line 14:

This will use the account ID and Region configured in the AWS CLI—if you have not yet set this up, please follow this tutorial section. We also need to bootstrap CDK in our account. This will create the required infrastructure for CDK to manage infrastructure in your account, and it only needs to be done once per account. If you have already done the bootstrapping, or aren't sure, you can just run the command again. It will only bootstrap if needed. To bootstrap CDK, run cdk bootstrap (your account ID will be different from the placeholder ones below):

Once the bootstrapping has completed, we're ready to deploy all the infrastructure. Run the following:

You will be presented with the following output and confirmation screen. Because there are security implications for our stack, you will see a summary of these and need to confirm them before deployment proceeds.

Enter y to continue with the deployment and create the resources. The CLI will show the deployment progress, and in the end, the output we defined in our CDK app.

Your infrastructure is now deployed, the instance is spinning up, and you can use the outputs at the bottom to download the SSH key, and then access the EC2 instance if you need to. You can also access the application in your browser by pasting in the IP printed above. You may need to wait a few minutes before the instance has completed spinning up and running the user data script. The first of these commands will be the same for every one:

Let's go through each of the command steps. First, the \ at the end of the lines tells your shell that the command is split onto a new line, which is why we add it to the first five lines to make the command more readable. The first of the commands, mkdir -p ~/.ssh, creates the directory .ssh in your OS user's home directory. The -p indicates that it should create the directory with parents directories if they don't exist, and to also not error if it already exists — this may be the first time you use SSH on your local machine, in which case the directory may not exist. What's meant by "with parents" is that if you would like to create a new nested directory structure (such as mkdir -p ~/something/very/nested/here), it would create all the parent directories of something, with very in it, nested inside very, and here inside nested without you needing to run multiple mkdir commands. The structure will look like this:

The second command uses the AWS CLI to call secretsmanager where we stored the SSH key to fetch it raw data, and we specify the query as SecretString to only bring back that field as text. We then use the > operator to tell our shell to take the output and write it to the file ~/.ssh/cd-key.pem, overwriting any existing content (if the file does not exist, it will be created; if it did exist, we will overwrite the contents). Lastly, the chmod command limits who can access the file—SSH requires keys to be locked down, so we limit read and write access to our OS user only.

Once we have run this command, we can SSH to the instance by using the public IP address, specifying which SSH key to use with the -i ~/.ssh/ec2-cdk-key.pem, and to only offer that key to the server by specifying -o IdentitiesOnly=yes—there is quite a lot of detail in that parameter which we won't cover here. The short version is that if you have many SSH keys, the SSH server in your instance would reject your login attempt if too many incorrect keys were sent as it would try each of them in your ~/.ssh directory with certain names.

Lastly, to access the Python web app we just deployed, use the Ec2CdkStack.IPAddress value in your browser as the address to open to see the app running. It will take a few minutes for the installation script to complete. You can follow the progress by using the Ec2CdkStack.SshCommand command to connect to the instance, and running sudo less /var/log/cloud-init-output.log, and then pressing shift + f to tail the file. Once it has completed, you should see the following:

Cleaning up your AWS environment

You have now completed this tutorial, but we still need to clean up the resources created during this tutorial. If your account is still in the Free Tier, there will not be any monthly charges. Once out of the Free Tier, it will cost ~0.0126 per hour.

To remove all the infrastructure we created, use the cdk destroy command. This will only remove infrastructure created during this tutorial in our CDK application. You will see a confirmation:

When the output shows Ec2CdkStack: destroyed, your resources have been removed. There is one more step for the cleanup: removing the S3 bucket used by CDK to upload the scripts and sample application. These resources aren't deleted by CDK as a safety precaution. Open the S3 console in your browser, and look for a bucket with a name like cdk-<9-random-letters-and-numbers>-assets-123456789012-us-east-1 (yours will have a different random number and your account number instead of 123456789012). If you see more than one (usually if you have used the CDK asset feature before), you can sort by Creation Date to see the latest created one. Open the bucket to confirm that you see three files: a .zip, .json, and .sh, each with a GUID as the filename. Select all the files, then choose actions -> delete, and follow the prompts to delete the objects. Lastly, go back to the S3 console, and delete the cdk-<9-random-letters-and-numbers>-assets-123456789012-us-east-1 bucket.

Conclusion

Congratulations! You have finished the Build a Web Application on Amazon EC2 tutorial using CDK to provision all infrastructure, and configured your EC2 instance to install and configure OS packages to run the sample Python web app. If you enjoyed this tutorial, found an issues, or have feedback us, please send it our way!