Ephemeral dev environments

Scenario is to create ephemeral dev environmnets in an AWS account with git branches.
Note: We assume all dev environments connect to a single database instance.

The idea is to create frontend (cloudfront+s3) and backend (ALB+TargetGroup+Ec2) and add respective DNS records.

Frontend:

AWS Amplify Console is a ready to use solution for the use case. When developer creates/updates/deletes a branch, amplify does the same on corresponding environment.

How to setup?

UI is easiest way to set this up as this is one time activity for a repository+account. However, cli is great too.

aws amplify create-app --name "example" --repository https://github.com/example/example-repo.git --enable-basic-auth --basic-auth-credentials "uname:password" --enable-branch-auto-deletion --access-token "github_personal_access_token" --enable-auto-branch-creation

More on cli here:

How does it work?

Amplify authenticates with github for first time, with an admin on repo. During the initial setup, it creates webhook and adds deploy keys. These are used for further operations.

Henceforth, everytime it receives a webhook trigger from github, it creates the corresponding environment with s3 and cloudfront, with subdomain as we intended. The build happens in isolated environments where it launches a new run time container, builds the codebase, uploads the codebase to edge locations and runs integration tests in verify stage.

Backend

While Amplify console works well for frontend/static deployments, backend is often custom requirements. In our scenario, since we want ephemeral dev environments.. it means we've to create load balancer and ec2 instance for each environment.

*If load balancer doesn't have any routing rules and is merely acting as proxy, we can create ec2 instance alone.

Unlike frontend, we don't need continuous deployment here. Instead, we run on-demand manually, triggered via jenkins. The script in jenkins would quickly zip the codebase, run terraform to build the infrastructure desired, zip file is shipped into ec2 instance and adds DNS entry.

Alternative to doing scp of zip file is to create an AMI like below:

However, AMI creation takes significant time and hence it's good if we're avoiding it for dev environment. For production however, it's recommended to use sophisticated tools like CodeDeploy rather.

There are other jobs on jenkins to deal with deleting infrastructure when the corresponding branch on github is deleted; And one job to run scheduled stop-start of the dev instances.

Thus, we've ephemeral dev environments with end-to-end setup for a backend+frontend application.