AWS Lambda Layer example in python

By
S Bali
-

Lambda Layers was one of the most exciting news out of AWS re:Invent 2018 for me. Using layers it is now possible to move runtime dependencies out of your function code by placing them in a layer.

Many of my lambda function need pymysql to get access to a RDS instance and it was quite a hassle to include the dependency in every function. In this post, I am going to show how to add pymysql in a lambda layer and then use it in a function.

Note: You may also be interested in my other post which shows how to set Time To Live in DynamoDB using Lambda.

Create the layer for pymysql:

  • On a Linux server with pip already installed create a temp folder and add a python folder to it as shown below.
cd $HOME 
mkdir -p temp/python 
cd temp/python

pip install pymysql in this folder.

pip install pymysql -t .

Package up the library and publish the layer.

cd .. 
zip -r9 ../pymysql.zip . 
aws lambda publish-layer-version --layer-name pymysql \
     --description "pymysql for mysql access" \
     --zip-file fileb://../pymysql.zip \
     --compatible-runtimes python3.6

Note:

  1. Make sure your user has the correct IAM to publish the layer.
  2. If you navigate to the Layers you should see your layer like this. I had to do this twice to get it right, but in your case, you should see it as version 1.
Layers

Now that our layer is ready we can create a test Lambda function and add the layer to it.

Adding pymysql layer to function:

I am assuming that you already know how to create a Lambda function and will not be listing the steps here. But if you need help, this is a good place to start with lambda.

Create a function called test-rds-with-layer using the ‘Create Function’ button and selecting the ‘Author from scratch’ option. I am going to call our function test-rds-with-layer.

Note:

  1. Make sure you set up your function with appropriate Execution Role.
  2. Your function runs in a VPC which can access your RDS.
  3. Set the security group that will allow access to RDS.

Open the Lambda function in AWS console, you will see layers as shown below.

layers

Click on layers and then on ‘Add a layer’ button.

layers

In the drop-down select your layer and the version and click on ‘Add’ button.

layers

Here is some sample lambda code to use layer and connect to a database.


import pymysql 
import json 
import sys 
def lambda_handler(event, context):
     rds_host='MyHost'
     name='user'
     password='password'
     db_name='dbname'
     try:
         conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
     except Exception as e:
         print(str(e))
         print("ERROR: Unexpected error: Could not connect to MySql instance.")
         sys.exit()
     print("SUCCESS: Connection to RDS mysql instance succeeded")
     with conn.cursor() as cur:
        select_statement = 'select now(), DATE_SUB(NOW(), INTERVAL 4 DAY); '        
        cur.execute(select_statement)
        for doc in cur:
           print( doc )
                 
     return {
         "statusCode": 200,
         "body": json.dumps('Hello from Lambda with Layer!')
     }

Save your function and create a test event for testing. The event data can be default as our function is not really going to use it.

layers

Test Function:

Click on ‘Test’ your lambda function and use the test event you created earlier.

At runtime, the function will import the pymysql library form your layer and your output should be similar to what I have shown below.

Execution Result: 
{   
   "statusCode": 200,
   "body": "\"Hello from Lambda with Layer!\""
} 
Log output
START RequestId: xxxxxxxx-xxxx-xxxx-xxxxx-xxxxxxxxxxx Version: $LATEST SUCCESS: Connection to RDS mysql instance succeeded
select now(), DATE_SUB(NOW(), INTERVAL 4 DAY);  (datetime.datetime(2018, 11, 30, 16, 23, 39), datetime.datetime(2018, 11, 26, 16, 23, 39))
END RequestId: xxxxxxxx-xxxx-xxxx-xxxxx-xxxxxxxxxxx
REPORT RequestId: xxxxxxxx-xxxx-xxxx-xxxxx-xxxxxxxxxxx Duration: 4.10 ms Billed Duration: 100 ms  Memory Size: 128 MB Max Memory Used: 25 MB

Further reading and improvements:

RELATED ARTICLESMORE FROM AUTHOR

Leave a Reply