Tutorial to build a NodeJs Serverless application using the AWS Serverless Application Model.
Tutorial Objectives:
1. Learn to build Serverless application using AWS SAM
2. Learn to fully automate builds and deployments by building a CI/CD pipeline using the AWS CDK.
3. Learn to run a Serverless application locally using the SAM CLI.
Prerequisites:
1. Download and extract the necessary code zip file from here.
Step 1: CREATE A CLOUD9 WORKSPACE
Navigate to the Cloud9 console: https://console.aws.amazon.com/cloud9 Choose Create environment.
Once you navigate to the Cloud9 console, click on the create environment button:
Chose a name for your environment.
Leave the default configuration, we don’t need a heavy server for this workshop.
Once you confirm creation, after a couple of minutes, your environment should look like the following image:
(Optional) If you prefer a dark theme, choose the theme in the top menu: View > Themes > Cloud9 > Cloud 9 Night.
Congratulations you have successfully created your Cloud9 Environment.
Step 2: UPGRADE SAM CLI
We have put together a bootstrap script that will make the upgrade easier for you. Download it by running the following command from your Cloud9 terminal. Download the bootstrap.sh file provided in the prerequisite. Upload the bootstrap.sh file to your cloud9 environment and run the following commands.
chmod +x bootstrap.sh
./bootstrap.sh
THIS MAY TAKE A FEW MINUTES TO COMPLETE.
Example Output:
To verify the new version, Run the following command:
sam --version
You should see SAM CLI, version 0.43.0 or greater.
Step 3: Create a new SAM Application
To initialize a new SAM Application Project, run the following command.
sam init
It will prompt for project configuration parameters:
Type 1 to select AWS Quick Start Templates.
Type 1 to select Zip as the package type.
Type 1 to select the NodeJs14.x runtime.
Leave sam-app for the project name
Type 1 to select the Hello World Example.
Project should now be initialized. You should see a new folder sam-app created with a basic Hello World scaffolding.
Step 4: RUN PROJECT LOCALLY
Install Dependencies
Before we run the application locally, it’s a common practice to install third-party libraries or dependencies that your application might be using.
cd sam-app/hello-world
And install the dependencies:
npm install
Run Using SAM-CLI In the terminal, run the following command from the root directory of the sam-app folder:
cd ~/environment/sam-app
sam local start-api --port 8080
To test our endpoint, we have 2 options
Option A) : Using CURL:
Now, we’re going to test our endpoint using curl.
First, without killing the running process, open a new terminal
Test your endpoint by running a CURL command that triggers an HTTP GET request.
curl http://localhost:8080/hello
The output should look like this
Option B) : Using a browser window
From the Cloud9 Menu Bar, Select Tools> Preview > Preview Running Application.
As you can see, it responded with the message “Hello World”.
Now, While the app is still running, open the file sam-app/hello-world/app.js and change the code to make the application say “hello my friend”
Save the file and refresh your Cloud9 browser tab.
Let’s run a Unit Test on our application, Navigate to the sam-app/hello-world folder and run the test command
cd ~/environment/sam-app/hello-world
npm run test
Don’t worry! This test is supposed to fail
Let’s fix our unit test now.
Locate the file sam-app/hello-world/tests/unit/test-handler.js and change the expected value from “hello world” to “hello my friend”
Your file should look like this:
Run the test again
npm run test
This test should pass.
Step 5: Manually Deploy to AWS
Building the app:
To build our SAM project we’re going to use sam build command. Navigate to sam-app folder and run the following command.
cd ~/environment/sam-app
sam build
The Build should be completed without any errors. Also, make sure you’ve made hidden files visible.
Deploy the application Now, we’re going to deploy our application manually on AWS with the help of sam deploy command
sam deploy --guided
This command is likely to ask for some input parameters. Say “Y” to everything and keep everything else as default
Confirm your deployment by entering “Y” again
Once the deployment is complete it should look like this,
Step 6: Create a pipeline
Creating a git repository First, we’re going to create a remote git repository using AWS CodeCommit. Run the following command to create the git repository.
aws codecommit create-repository --repository-name sam-app
You’ll see the following output. Copy the value of cloneUrlHttp, you will need it later.
Configure the credentials for CodeCommit with following command. Don’t forget to replace “Replace with your name” and replace_with_your_email@example.com with your own name and email
git config --global user.name "Replace with your name"
git config --global user.email "replace_with_your_email@example.com"
Locate your sam-app/.gitignore file (Make sure you have enabled to show hidden file as mentioned at the start of Step 5). And paste the following snippet there.
.aws-sam/
packaged.yaml
Your gitignore file should look like this
Navigate to your root directory of sam-app and run following commands to commit your changes.
cd ~/environment/sam-app
git init
git add .
git commit -m "Initial commit"
Push the code
Push your code to your remote repository with the following command
git remote add origin codecommit://sam-app
git push -u origin master
Verify your commit in CodeCommit
From your AWS Management console, Navigate to AWS CodeCommit. And verify the changes are actually reflected there.
Setting up a CDK Project
First, we’re going to initialize our project with the following command within our sam-app directory where the pipeline code with reside.
cd ~/environment/sam-app
mkdir pipeline
cd pipeline
Initialize a new CDK project within the pipeline folder by running the following command:
cdk init --language typescript
Now install the CDK modules that we will be using to build a pipeline:
npm install --save @aws-cdk/aws-codedeploy @aws-cdk/aws-codebuild
npm install --save @aws-cdk/aws-codecommit @aws-cdk/aws-codepipeline-actions
npm install --save @aws-cdk/aws-s3
After a few seconds, the pipeline folder will be created in our directory.
Run this command to resize your Cloud9 volume
wget https://cicd.serverlessworkshops.io/assets/resize.sh
chmod +x resize.sh
./resize.sh 20
Modify stack name
In your sam-app/pipeline/bin/pipeline.ts file change the project name to sam-app-cicd and make sure you save the file.
Building the CDK project
Even though we haven’t wrote any code yet, let’s get familiar with how to build and deploy a CDK project, as you will be doing it multiple times in this workshop and you should get comfortable with the process. Start by building the project with the following command:
cd ~/environment/sam-app/pipeline
npm run build
Now, we’re going to deploy our project.
cdk deploy
Our output should look like this,
Optionally, you can open the CloudFormation console and check if your CloudFormation stack has been created.
Artifacts Bucket
Every Code Pipeline needs an artifacts bucket, also known as Artifact Store. CodePipeline will use this bucket to pass artifacts to the downstream jobs and its also where SAM will upload the artifacts during the build process.
Make sure you are editing the pipeline-stack file with .ts extension This file is located at sam-app/pipeline/lib/pipeline-stack.ts
Paste the following code in the pipeline-stack.ts file
// lib/pipeline-stack.ts
import * as cdk from '@aws-cdk/core';
import s3 = require('@aws-cdk/aws-s3');
import codecommit = require('@aws-cdk/aws-codecommit');
import codepipeline = require('@aws-cdk/aws-codepipeline');
import codepipeline_actions = require('@aws-cdk/aws-codepipeline-actions');
import codebuild = require('@aws-cdk/aws-codebuild');
export class PipelineStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
// The code that defines your stack goes here
const artifactsBucket = new s3.Bucket(this, "ArtifactsBucket");
}
}
Now, again build and deploy the application.
cd ~/environment/sam-app/pipeline
npm run build
cdk deploy
The output should look like this
Note: If you get an error during the build process, make sure all @aws-cdk dependencies in the package.json file have the same version number, if not, fix it (by giving same version number to every @aws-cdk dependency), delete the node_modules folder and run npm install. More info: Visit this GitHub thread
Now, Open sam-app/pipeline/lib/pipeline-stack.ts file. And paste the following snippet after your bucket definition
// Import existing CodeCommit sam-app repository
const codeRepo = codecommit.Repository.fromRepositoryName(
this,
'AppRepository', // Logical name within CloudFormation
'sam-app' // Repository name
);
// Pipeline creation starts
const pipeline = new codepipeline.Pipeline(this, 'Pipeline', {
artifactBucket: artifactsBucket
});
// Declare source code as an artifact
const sourceOutput = new codepipeline.Artifact();
// Add source stage to pipeline
pipeline.addStage({
stageName: 'Source',
actions: [
new codepipeline_actions.CodeCommitSourceAction({
actionName: 'CodeCommit_Source',
repository: codeRepo,
output: sourceOutput,
}),
],
});
Please refer the pipeline-stack-1.txt file from the extracted zip package to confirm no errors are made.
Adding the build stage We’re going to add a build stage for our project. For this we’ll be using AWS CodeBuild service.
// Declare build output as artifacts
const buildOutput = new codepipeline.Artifact();
// Declare a new CodeBuild project
const buildProject = new codebuild.PipelineProject(this, 'Build', {
environment: { buildImage: codebuild.LinuxBuildImage.AMAZON_LINUX_2_2 },
environmentVariables: {
'PACKAGE_BUCKET': {
value: artifactsBucket.bucketName
}
}
});
// Add the build stage to our pipeline
pipeline.addStage({
stageName: 'Build',
actions: [
new codepipeline_actions.CodeBuildAction({
actionName: 'Build',
project: buildProject,
input: sourceOutput,
outputs: [buildOutput],
}),
],
});
Please refer the pipeline-stack-2.txt file from the extracted zip package to confirm no errors are made. Note: If you get any errors at this stage, Go to sam-app/pipeline/package.json and make sure all sure all @aws-cdk dependencies in the package.json file have the same version number, if not, fix it (by giving same version number to every @aws-cdk/core dependency), After that run the command
npm install –-save @aws-cdk/core
More info: Visit this GitHub thread e.g: Once during the execution. We faced this error.
Now navigate to sam-app/pipeline/package.json and see if the version numbers of all the @aws-cdk dependencies are same or not.
As you can see in the above picture, the version of @aws-cdk/core is 1.135.0 whereas other dependencies are running at 1.137.0 In this case we’ll change the version of @aws-cdk/core here and will run the aforementioned npm install --save @aws-cdk/core command.
Now, again build and deploy application.
cd ~/environment/sam-app/pipeline
npm run build
cdk deploy
Navigate to AWS CodePipeline and click on your newly created pipeline.
The build step must have failed. But don’t worry this is expected. We haven’t specified the commands to run during the build yet.
Buildspec file:
To provide commands during the build process we need create and code a buildspec file.
Right-click on your sam-app folder and click on new file.
Name your file buildspec.yml This file will contain the build commands we need to provide during the build phase.
Now, Copy and paste the code from the next page into your buildspec.yml file
Take a not that the yml files are based on indentations, so an error in the indent could result in an error. Hence we suggest to carefully paste the code without any errors/changes.
# ~/environment/sam-app/buildspec.yml
version: 0.2
phases:
install:
runtime-versions:
nodejs: 12
commands:
- pip3 install --upgrade aws-sam-cli
- sam --version
- cd hello-world
- npm install
pre_build:
commands:
- npm run test
build:
commands:
- cd ..
- sam build
post_build:
commands:
- sam package --s3-bucket $PACKAGE_BUCKET --output-template-file packaged.yaml
artifacts:
discard-paths: yes
files:
- packaged.yaml
Save the file. It should look like this.
Push the changes to your repository and this time the CodePipeline should automatically get to build and deploy phase.
cd ~/environment/sam-app
git add .
git commit -m "Added buildspec.yml"
git push
Go to CodeBuild and see if your build has succeeded or not. This time, it should
Deploy stage
The Deploy Stage is where your SAM application and all its resources are created an in an AWS account. The most common way to do this is by using CloudFormation ChangeSets to deploy. This means that this stage will have 2 actions: the CreateChangeSet and the ExecuteChangeSet.
Add the code from the next page to your pipeline-stack.ts file
// Deploy stage
pipeline.addStage({
stageName: 'Dev',
actions: [
new codepipeline_actions.CloudFormationCreateReplaceChangeSetAction({
actionName: 'CreateChangeSet',
templatePath: buildOutput.atPath("packaged.yaml"),
stackName: 'sam-app',
adminPermissions: true,
changeSetName: 'sam-app-dev-changeset',
runOrder: 1
}),
new codepipeline_actions.CloudFormationExecuteChangeSetAction({
actionName: 'Deploy',
stackName: 'sam-app',
changeSetName: 'sam-app-dev-changeset',
runOrder: 2
}),
],
});
Please refer the pipeline-stack-3.txt file from the extracted zip package to confirm no errors are made.
Run the following commands within your pipeline directory.
cd ~/environment/sam-app/pipeline
npm run build
cdk deploy
Triggering the release
Navigate to your pipeline and you will see the Deploy stage has been added, however, it is currently grayed out because it hasn’t been triggered. Let’s just trigger a new run of the pipeline manually by clicking the Release Change button from your CodePipeline.
Once the Dev looks like gets succeeded push the changes to your CodeCommit repository
git add .
git commit -m "CI/CD Pipeline definition"
git push
Congratulations! You have created a CI/CD pipeline for a Serverless application!
Step 7: Canary Deployments
A Canary Deployment is a technique that reduces the risk of deploying a new version of an application by slowly rolling out the changes to a small subset of users before rolling it out to the entire customer base.
In this step you will learn how to implement gradual deployments with AWS SAM, AWS CloudFormation and AWS CodeDeploy with just a few lines of configuration.
Update the SAM Template:
Open sam-app/template.yaml and add the following lines under HelloWorldFunction properties section.
AutoPublishAlias: live
DeploymentPreference:
Type: Canary10Percent5Minutes
Please check the indentation as it is very important in YAML file
Validate the SAM template
Run the following command on your terminal:
cd ~/environment/sam-app
sam validate
If the template is correct, you will see template.yaml is a valid SAM Template. If you see an error, then you likely have an indentation issue on the YAML file. Double check and make sure it matches the screenshot shown above.
Finally, In the terminal, run the following commands from the root directory of your sam-app project.
cd ~/environment/sam-app
git add .
git commit -m "Canary deployments with SAM"
git push
Define a Cloudwatch alarm
Add the following lines to the DeploymentPreference section of the HelloWorldFunction definition.
Alarms:
- !Ref CanaryErrorsAlarm
And then add the following alarm definition to the template.yaml file in the Resources section after the HelloWorldFunction definition.
CanaryErrorsAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmDescription: Lambda function canary errors
ComparisonOperator: GreaterThanThreshold
EvaluationPeriods: 2
MetricName: Errors
Namespace: AWS/Lambda
Period: 60
Statistic: Sum
Threshold: 0
Dimensions:
- Name: Resource
Value: !Sub "${HelloWorldFunction}:live"
- Name: FunctionName
Value: !Ref HelloWorldFunction
- Name: ExecutedVersion
Value: !GetAtt HelloWorldFunction.Version.Version
Please refer the template-yaml.txt file from the extracted zip package to confirm no errors are made especially in the indentations.
Validate the SAM template Run the following command on your terminal:
cd ~/environment/sam-app
sam validate
The output should look like this
If you see any error, that’s probably because of the indentation and we recommend to copy the whole file from template-yaml.txt which is in the zip package you downloaded from Prerequisites.
Finally, In the terminal, run the following commands from the root directory of your sam-app project.
cd ~/environment/sam-app
git add .
git commit -m "Canary deployments with SAM"
git push
Verify in CodeDeploy
Wait for your pipeline to get to the deployment stage (ExecuteChangeSet) and when you see it In Progress. Navigate to the CodeDeploy console to watch the deployment progress.
Navigate to the AWS CodeDeploy console and after a couple of minutes, you should see a new deployment in progress. Click on the Deployment to see the details.
The deployment status shows that 10% of the traffic has been shifted to the new version (aka The Canary). CodeDeploy will hold the remaining percentage until the specified time interval has ellapsed, in this case we specified the interval to be 5 minutes.
Shortly after the 5 minutes, the remaining traffic should be shifted to the new version:
Rollbacks Introduce an error on purpose: Lets break the Lambda function on purpose so that the CanaryErrorsAlarm gets triggered during deployment. Update the lambda code in sam-app/hello-world/app.js to throw an error on every invocation, like this:
let response;
exports.lambdaHandler = async (event, context) => {
throw new Error("This will cause a deployment rollback");
// try {
// response = {
// 'statusCode': 200,
// 'body': JSON.stringify({
// message: 'hello my friend with canaries',
// })
// }
// } catch (err) {
// console.log(err);
// return err;
// }
// return response
};
Make sure to update the unit test, otherwise the build will fail. Comment out every line in the sam-app/hello-world/tests/unit/test-handler.js file:
// 'use strict';
// const app = require('../../app.js');
// const chai = require('chai');
// const expect = chai.expect;
// var event, context;
// describe('Tests index', function () {
// it('verifies successful response', async () => {
// const result = await app.lambdaHandler(event, context)
// expect(result).to.be.an('object');
// expect(result.statusCode).to.equal(200);
// expect(result.body).to.be.an('string');
// let response = JSON.parse(result.body);
// expect(response).to.be.an('object');
// expect(response.message).to.be.equal("hello my friend with canaries");
// });
// });
Push the changes
cd ~/environment/sam-app
git add .
git commit -m "Breaking the lambda function on purpose"
git push
Wait for the deployment to start
While the deployment is running, you need to generate traffic to the new Lambda function to make it fail and trigger the CloudWatch Alarm. In a real production environment, your users will likely generate organic traffic to the canary function, so you may not need to do this.
In your terminal, run the following command to invoke the Lambda function:
sudo yum install -y jq
aws lambda invoke --function-name \
$(aws lambda list-functions | jq -r -c '.Functions[] | select( .FunctionName | contains("sam-app-HelloWorldFunction")).FunctionName'):live \
--payload '{}' \
response.json
This will be the output
There will be a new file response.json created. It contains the response of the lambda invocation. If you open it, you may see the the response of the old Lambda version, or you may see the new one that causes an error.
Remember: During deployment, only 10% of the traffic will be routed to the new version. So, keep on invoking your lambda many times. 1 out of 10 invocations should trigger the new broken lambda, which is what you want to cause a rollback.
Here is a command that invokes your function 15 times in a loop. Feel free to run it in your terminal.
counter=1
while [ $counter -le 15 ]
do
aws lambda invoke --function-name \
$(aws lambda list-functions | jq -r -c '.Functions[] | select( .FunctionName | contains("sam-app-HelloWorldFunction")).FunctionName'):live \
--payload '{}' \
response.json
sleep 1
((counter++))
done
Since 1/10 invokes go to the broken lambda, out of 10 you’d see one response with a Unhandled exception
Go to your CodePipeline and confirm that the Deployment actually failed.
Step 8: Clean up.
Empty and delete both the buckets created by SAM CLI and CodePipeline.
Now that the buckets are empty, we can delete the Cloudformation stacks:
Delete the CF stacks in following sequence and please wait for each to complete before going to next one.
Delete sam-app
aws cloudformation delete-stack --stack-name sam-app
Delete sam-app-cicd
aws cloudformation delete-stack --stack-name sam-app-cicd
Delete aws-sam-cli-managed-default
aws cloudformation delete-stack --stack-name aws-sam-cli-managed-default
You can also delete the CF Stacks by navigation to the Cloud Formation management console and deleting the stacks.
Finally Delete the cloud 9 environment.