Lambda Function To Stop Long Running EC2 Instances

Background

While working in my personal AWS account, sometimes I forget to shut EC2 instances down. Most of these instances do not cost much to run, however the charges keep adding up and no one likes to waste money.

A Lambda function is a great way to monitor your account for long running instances and can be used to shut down those instances to save money.

So if you forget to shutdown instances, the lambda function will notify you and shut it down.

If for some reason, you wanted to keep the server up for some more time, you could change the tag associated with the instances which the lambda function looks at.

I decided to deploy a Lambda Function which can use instance tags to determine if an instance has gone past its allotted runtime. The function will notify by sending an e-mail when it detects that an instance has been selected to be stopped.

Requirements

Make sure you have installed the Serverless Framework on your workstation.

Ensure that the workstation from where you are going to run the serverless framework has all the necessary permissions to publish your Lambda function. The workstation should have a ‘IAM Role’ or ‘Access Keys’ configured correctly.

We need a SNS Topic which I have shown how to setup below.

Procedure

SNS

Let us go ahead and create a SNS topic with an e-mail subscription first.

Run these two commands from your linux shell.


topicARN=$(aws sns create-topic --name site-monitor --output text)
aws sns subscribe --topic-arn $topicARN --protocol email --notification-endpoint <your email>

Make sure you edit line and set your e-mail for SNS Subscription.

Once you run these commands, you will receive an e-mail as shown below.

SNS Subscription
SNS Subscription

Go ahead and click on the link provided in the e-mail to confirm your SNS subscription.

So far, we have created:

  • A SNS topic called site-monitor
  • Setup an e-mail subscriber to the SNS topic

Lambda Function

Now it is time for us to setup our Lambda function using the Serverless Framework.

To create the template for our function, I run the commands as shown below.


sls create --template aws-python3 --path stop-aws-instances
cd stop-aws-instances

Here you will see two files serverless.yml and handler.py, I are going to update these files with our code as shown below.

The file serverless.yml contains information about our Lambda function and the IAM role that will be needed by the function.

File handler.py is where we will write the code which accomplishes our task.

Let us walk through the handler.py file.

But first a few important edits in this file.

  • Make sure you edit your account number on line #5 to ensure your topic ARN is valid.
  • Line #9 sets the max hours that an instance is allowed to run.

Following is a brief description and workflow of the function, without going into too much details.

Method check_and_stop_ec2

The method starts by making a describe instances call to find all running instances that contain a tag AllowStop.

A for loop will loop through all the instances returned in the response object.

For each of the instances, I am looking for the AllowStop tag and the value associated with it.

If the value is greater than the max_hours specified then it is replaced.

If the value is 0 or less than 0, I am changing the threshold value to a low number so that the instance can be shutdown right away.

serverless.yml

Let us take a look at the serverless.yml file now.

Few important edits in this file.

  • Line #8 specify the name of your deployment bucket.
  • Line #10 set a tag value that makes sense to you.
  • Line #36 sets the interval at which this Lambda runs and examines the running instances.

In this file, we are setting the function runtime information, the bucket where serverless will deploy the code too. The IAM role needed for the function to query tags and stop EC2 instances.

We also specify the memory allocated to our function, the timeout period and the architecture. In this example I set the architecture to arm64 to save even more on Lambda execution costs.

The events section in the file specifies that this function should run every 5 minutes. Serverless will take care of setting up the Event and its frequency to invoke your function accordingly.

Deployment

It is now time to deploy our function. The deployment can be done by running the command sls deploy as shown below:


sls deploy
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Ensuring that deployment bucket exists
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service stop-aws-instances.zip file to S3 (3.48 kB)...
Serverless: Validating template...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
....................
Serverless: Stack create finished...
Service Information
service: stop-aws-instances
stage: prd
region: us-east-1
stack: stop-aws-instances-prd
resources: 6
api keys:
  None
endpoints:
functions:
  save-money: stop-aws-instances-prd-save-money
layers:
  None

This will trigger the process of creating the CloudFormation scripts which in turn will create a Lambda Function, an IAM role for your function and a CloudWatch Events Rule.

After the resource creation completes, go to CloudFormation to see the stack information to confirm the details. Note: This is not needed but good to lookup. If there are errors in the execution, you will most likely see them on the command line itself.

If you make any changes to either of the two files, you can simply run ‘sls deploy’ again to push the latest code up to your account.

Testing

Let us go ahead and set a tag on one of the instances we want our Lambda function to shutdown.

aws ec2 create-tags --resources i-someinstanceid --tags Key=AllowStop,Value=1

The above command will set a tag on your instance and the Lambda will stop this instance if it has been running for more than an hour.

You can either wait for the CloudWatch Events rule to trigger your function,, or to Test this function immediately from your workstation you can run this CLI command with your account number set (command and output listed):


aws lambda invoke --function-name arn:aws:lambda:us-east-1:<your account>:function:stop-aws-instances-prd-save-money response.json
{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}

See my other posts to get an idea of how to do a test run of a Lambda function.

Note: With the default setting, the function will not allow an instance to run more than 10 min (600 seconds) if the AllowStop tag is set to a value lower than 1. This can be changed by editing line #38 of handler.py.

Remove Resources

If you no longer wish to keep this function, you can remove the resources that were created by running this command.

sls remove

This will remove all the resources that were created. Remember you are responsible for all charges incurred. Leaving a Lambda function with a CloudWatch Event rule enabled, will cost you even if there are no servers to terminate.

You may also have to cleanup your CloudWatch Logs and S3 buckets where this code was deployed.

To ensure you do not retain too many CloudWatch logs, see my post to set retention period for your logs.

Improvements

The code shown here is meant for demonstration purpose.

Review exception handling, adding comments etc. to make it more meaningful for you.

Use fractions or seconds or other time units to specify runtime duration.

Further Reading:

Photo Credit:

Photo by Sandy Millar on Unsplash

Leave a Reply