AWS Lambda Layer for Python package requests

December 29, 2018

In my previous post, I made a layer for pymysql to access an RDS from Lambda. In this post, I will show how to make a layer for the Python package requests. This will make it easy to make curl like requests from our Lambda Function. To read more about this package see here.

Background

Requests is an Apache2 licensed HTTP library which makes it very easy to make HTTP requests from Python. It is designed to be used in a simple manner. As the documentation for it says there is no need to manually add query strings to your URLs, or to form-encode your POST data.

Procedure

I am going to use my AWS Cloud9 environment to build the layer, but you can do this on any Linux server with python installed and which has the necessary IAM role or permissions to publish your layer.

Make sure you have a working Python 3.x environment available on your Cloud9. If not you can follow steps listed here this to install pip3.

Create a temporary directory on your Cloud9 environment to install the package locally to make it into a layer.

 sbali:~/environment $ which pip3
 ~/.local/bin/pip3
 sbali:~/environment $ mkdir -p temp/python && cd temp/python

Next step is to install the package locally using pip3 and package
up the directory in a zip file and publish the layer.

pip3 install requests -t . 
cd ..
zip -r9 ../requests.zip .
aws lambda publish-layer-version --layer-name requests \
      --description "requests package" \
      --zip-file fileb://../requests.zip \
      --compatible-runtimes python3.6

if your function was published correctly, then you should see it appear in the AWS Console under Lambda Layers.

Note:

If you were not able to publish you function then you are missing some IAM permissions. See this page for more help with fixing your IAM role or permissions.

Next step is to create a Lambda Function, where we will include our layer and make some HTTP requests. If you need more information on how to get started with Lambda, read more here.

One of the things I love most about using Layers is that I can write simple function directly in the AWS Lambda Console. Before Layers came along if you ever needed a package in your code, you would have been forced to work on your function in a Linux server and update code via zip file or other means for every update.

Navigate to the AWS Lambda Console and click on ‘Create function’ button and go with the Author from scratch option. Give your function a name, select the Runtime as Python 3.6 and choose the appropriate role.

If you do not have any existing roles, maybe because this is your first function, then select the ‘Create a new role from one or more templates’ option and follow the steps.

Hit the ‘Create function’ button and your function will be created. Follow steps from my earlier post to add the newly created layer in this function. Make sure you hit the ‘Save’ button to save these changes. Also make sure to create a test event so that you can do a ‘Test’ of your function from the console.

Now let us go ahead and add some test code to our Lambda Function.

import json
import requests

url = 'https://blog.skbali.com'

def lambda_handler(event, context):
    r = requests.get(url)
    print(r.status_code)
    print(r.headers.get('content-type'))
    return {
         'statusCode': 200, 
         'body': json.dumps('All Done')
    }

Save your function, select your test Event and hit the ‘Test’ button. In the log output, expand ‘Details’, you should see the HTTP status code and the content type in the ‘Log output’ similar to the output shown below

START RequestId: xxx-xxx-xxx Version: $LATEST
 200
 text/html; charset=UTF-8
 END RequestId:  xxx-xxx-xxx 
 REPORT RequestId:  xxx-xxx     Duration: 246.93 ms Billed Duration: 300 ms     Memory Size: 128 MB Max Memory Used: 28 MB

That is all there is to this example now. You can use the requests package as you wish in all of your Lambda Functions that need to make some form of HTTP request.