Unlocking Lambda's Full Potential: How to Resolve Layer Size Limits and Optimize Your Serverless Experience!

A while ago me and my team were facing some issues regarding the limit size of the layer in our lambda apps, it was because the library that we used are to big for layer to handle. As you can see in bellow image the size that you can uploaded is only 262 MB.

At that time we were using tensor library and its related library to be used in our project because we were aiming to build an artificial intelegence apps using that lib. For your information before you go dive into this article you should know that not all project are applicable using lambda, for alternative you can use ECS or some kind of docker, but for our project we want to make pricing as efficient as possible, because when you use a service such as ECS you will be billed monthly, it was so much different than lambda which was billed for every access that was hit by apps by user.

Here i will list and explain what did we do in order to fix that issue.

we choose which library that we need and eliminate the rest

We are successfully optimized it and shrink it size by a half, but we still haven't succeed, so we go to the next step.

We uploaded the whole project into AWS S3

And dont forget to reshape your project to be applicable in the lambda, make its format into lambda style, dont make it as python project only *example, for example you change its main function name into

def lambda_handler(event, context):

when we used this method we can exceed its size into almost 400 MB for the whole project and the limit of that zip file should not exceed more than 170 MB for its final size after you zipped it. But unfortunately our project are much bigger than that.

But after we try above method, finally we were decide to move the code into our separate server because the size of project are so big that lambda cannot handle, i think in that case we could learn that lambda are only suitable for small project that doesn't need big size library, but you can still tried above method if your project aren't as big as ours. But if your project is using tensor i fully recommended that you use service like ECS, docker, and etc.