Looking into the AWS ecosystem of serverless services, AWS Step Functions is one of my personal most favorite services. I recently had a chat with ...
For further actions, you may consider blocking this person and/or reporting abuse
I'm having an issue with making dynamoDB calls in a step function vs lambda - I am seeing queries taking 7-10 milliseconds in the lambda vs about 250-775 milliseconds in the step function - any thoughts as to what would cause this? Any input would be appreciated
Hard to guess. Do you see some throttling on DynamoDb that could cause the increased latency?
The issue turned out to be xray - when we disabled it the calls in the stepfunction took as long as the calls in the lambda
It's weird to me that you're using step function as lambda replacement. Both possibly can do the same task but step function is a much better interface for scheduling i.e. "What should we execute next? When?" and microservice orchestration and especialy good in passing along messages among compute services. While lambda focus on general computing services.
As you can see, you have managed to "put" then "get" to s3 with both lambda and step function. However, the focus here should be in step function, you have 2 computing service, one is get, the other one is put, and you execute one after the other. Whereas in lambda, you possibly have one code base with aws sdk to access the s3 that does the same thing.
Frankly, given the same task, I will simply use lambda as it will be the cheaper option. However, if you are benchmarking this way, you have misunderstood step function power.
Hi @artidata thanks for your reply. Step Functions gives you an interesting option when orchestrating services or a workflow. Both services are awesome and have it's use cases that have quite some overlaps. As I stated: the comparison is not meant to be to argue excplicit for or against a given services. It is more to offer more options for some use cases.
I will look deeper into a cost comparison in an upcoming benchmark.
I think it's important to point out that the lambda benchmark might be different for other programming languages.
I've noticed that SDK calls from golang lambda functions are faster than nodeJS functions. I just wanted to add this dimension so that we don't just blame lambda for the latency but consider other factors
Absolutely. There is a lot you can improve on your Lambda function code. The thing is: you have to know all these details if you have to improve for performance efficiency. It seems like the state machine in the stepfunction simply works pretty well without maintaining a bunch of code or configuration.
Optimizing your lambda function can be a really complex task.
Does the conclusion hold when setting the env var
AWS_NODEJS_CONNECTION_REUSE_ENABLEDin the Lambda function?docs.aws.amazon.com/sdk-for-javasc...
That is worth to test in an upcoming version. As well as increasing memory configurations on the lambda. First indications are already discussed on Twitter:
twitter.com/marekq/status/14536579...
I've been using step function for a while and I've loved it.
One question, how do you set the things inside the step function block that calls the S3 API? For example, how do we set the content of the PutObject action?
Note: I've not seen the step function after this great update.
@bigluxio you should play around with this awesome new feature :D
With regards to your question: in the first step I wanted to keep it simple. So I just write the execution id from the context object in S3. You get the whole state machine definition from here:
github.com/cremich/aws-sf-lambda-b...
My recommendation would be: start with the new Workflow Editor to setup your workflow, export the state machine definition and provision it using your favorite infastructure as code tool.
Analyzing the performance difference between step functions and lambdas is like examining the performance of the Vespas versus mopeds. 😬
So which service is the vespa and which one the moped? :D
I don't think it matters. I drive a freight train. ;-)
Lambda, and things built on top of that, are great for small/medium projects. But, once running at enterprise/Internet scale, one needs to switch to services that directly are built on top of EC2. Even Fargate managed container service is orders of magnitude more scalable (horizontally and vertically) than anything that depends on Lambda.
For me, a project crosses the border from small/medium to enterprise/internet scale at around a sustained load of 10s of API requests / second, or an SLA > 99.9%, whichever comes first. This is the moment when I see dependencies on Lambda fall apart.
It would be interesting to also take cost into consideration
Definitely. I will prepare a second part of this focusing on tuning the lambda settings but also looking into the affect of costs.
It's an interesting article, thanks for sharing it.
I was just wondering about something, did you use step functions in an asynchronous way and lambda synchronous from API gateway?
Looking at the codebase, it is an Express state machine executed synchronously:
github.com/cremich/aws-sf-lambda-b...
github.com/cremich/aws-sf-lambda-b...
Also lambda is executed synchronously.
Thanks for the feedback @sirmomster
I will give the article an update to be more precise in the general setup section.
Yes, like @gerardketuma already wrote: both lambda and the state machine are executed synchronously. In this case I used an express state machine.
I wonder how the lambda code impacts this result? Would a rust lambda compare more favouribly?
It is likely to have an impact. Like I tried to scratch without going too much in detail as this is worth a complete article on its own: