AWS Lambda isn't truly "serverless", your code must run somewhere, right? Don't answer that, it's a rhetorical question! 😅
Yes! AWS lambda runs on servers!
But the word "serverless" means "I don't really care where it runs as long as it runs! And the actual servers aren't my concern therefore I won't have access to them".
Pay close attention to the last part as sometimes you would need some packages/libraries depending on your language of choice to use in your lambda function (actually most of the time 😂).
For me I use Python, and there is no way to "pip install ..." into the lambda function the packages I need as I don't have access to the servers that runs the function. See the issue here?! 😕
Enter: AWS Lambda Custom Layers
Custom layers are AWS' solution to that problem ☝️. With custom layers you can include the extra "Umph" your amazing Lambda function needs to do its magic. You can read all about them form the AWS Docs.
But I'm not here to direct you to the docs. I wanted to write this to all the busy cloud enthusiasts out there that are too short on time to read the docs and share my experience creating a python custom layer in a short and clear set of steps. (Actually, you must read the related docs always! Even if you are remotely interested in developing cloud workloads 🤷♂️)
(I'm on a windows machine and I use Ubuntu WSL for this. AWS recommends the use of Cloud9 to do that as your custom layer must be compatible with Amazon Linux to function correctly on your lambda. But I find WSL satisfactory in MOST cases)
For the demo, I'll just be following the example listed in the docs as it's very clear but I'll remove the clutter and share my own experience 😉.
1- I'll make one directory to group everything I need for this task as follows:
my-lambda
|_ venv-python3.9
|_ python
|_ lambda-layer.zip
This is how it WILL look like but first I'll create my root directory and "cd" into it.
mkdir my-lambda
cd my-lambda
2- You must use the same python version in your local environment as in your lambda function. My Lambda uses Python 3.9 and I'll use that too. You can use whatever Python environment management solution that makes sense for you. I use Conda but pyenv is good.
This is how you do it using Conda.
conda create -p ./venv-python3.9 python=3.9 -y
This will install python in venv-python3.9 directory in my-lambda. After the installation, you must activate this environment as follows:
conda activate ./venv-python3.9
3- I'll be implementing this functions that uses requests package and that's what I'm making the custom layer for
import requests
def lambda_handler(event, context):
response = requests.get("https://www.example.com/")
print(response.text)
return response.text
This function just requests the "https://www.example.com" website and prints the response. To see what we would expect, use the following command:
curl https://www.example.com/
And we should get something like that.
4- I'll pip install the requests package but I'll change its download path using the --target param so that I don't download it in the default site-packages directory and just download it in the my-lambda directory
pip install --target=./python requests
After executing this command, you will see the packages directory containing all the dependencies for the requests package to work.
5- I'll zip the python directory and that's what I'll use as my custom layer. AWS Lambda extracts whatever is in the zip file in the /opt directory and the lambda python runtime automatically supports /opt/python in the PATH env var.
zip -r lambda-layer.zip python
This will produce the lambda-layer.zip file that I'll upload.
6- Now for the AWS stuff, in order to upload your custom layer, we will do the following:
1- Navigate to Layers form the left navigation bar then press create layer
2- Fill the Create Layer form and upload the zip file we have created.
3- Then press create.
4- After your zip file is uploaded and if everything goes well, you will get the following message.
5- Navigate again to your lambda function, scroll down and click Add a lawer.
6- Select a Custom Layer.
7- Choose your recently uploaded layer.
8- Select your version (generally it will be version 1 unless you have previously made changes to it)
9- Press Add
10- To test the new layer, press Test.
11- Configure your test event, the default values will be fine.
12- Press Test again and this time you will see similar results as the curl command
[Optional] Clean up our local environment
In order to clean up, we will have to do the follwing:
1- Deactivate the python3.9 env
conda deactivate
2- Remove the environement
conda env remove -p ./venv-python3.9
3- Delete the my-lambda directory
cd ..
rm -fr my-lambda
And that's it! Now you know how to create custom AWS Lambda Layers 😉.
Top comments (1)
Hi @fadygrab, I tried to import the PyYAML library as a custom layer, but I received an error:
{
"errorMessage": "Unable to import module 'lambda_function': No module named 'yaml'",
"errorType": "Runtime.ImportModuleError",
"requestId": "88449d3f-4587-4528-b522-f363e83f13da",
"stackTrace": []
}
How can I solve this issue? I have followed your steps.