Hi Matt, congrats on the article. It was clear and the companion source code and video were helpful.
While looking at the step function of AWS I left with the impression that it has some expectations that can limit its adoption - or force changes on how you would model your process.
For me, Sagas or the process Manager patterns are used when you have long running processes, which are distributed by nature, and want to have the transactional context. So the SEC or Process Manager is a state machine that reacts to the results from the previous state to move to the next one. This means triggering the execution of the next step.
If we look at the example you - and Caite/others - used there is a clear relationship. The stepfunction provides this state machine and the lambdas are the steps being executed.
All good so far, but what if the action being taken by this step is long-running on its own? Imagine that in an e-commerce solution you would have the steps of payment, shipment, and sending the email in that order. While payment and sending emails can likely be executed in seconds, the shipment is a long-running that could have a compensating aspect if, for example, the item is found to be out of stock or can't be sent (found to be damaged upon inspection).
In a more 'traditional' saga implementation, I would ask for the shipment to happen and simply go to sleep until I receive a message (event) with the ShipmentSent or ShipmentCancelled to resume and go to the next step.
With step functions it feels I would have to essentially break the process in two. One that would go payment, then trigger the shipment and stop. Another that would be triggered with the shipment event and have the compensating step for the payment or the successful path to send the email. And since they are different processes the state would have to be somehow recreated as I can't reuse from the previous process.
I wonder what would be your assessment on the subject because having this breakdown feels artificial but seems like the only solution if one still wants to use AWS step function.
Software Architect working on enabling engineers to rapidly deliver serverless-first solutions in a Fortune 100 organisation - passionate about Serverless, AWS, dev.to/cdkpatterns, TCO, and CI/CD
Pausing the step function flow until a longer running process or human approval process happens is totally possible. AWS just doesn't seem to shout about it enough. Here is an example - docs.aws.amazon.com/step-functions...
Hi Matt, congrats on the article. It was clear and the companion source code and video were helpful.
While looking at the step function of AWS I left with the impression that it has some expectations that can limit its adoption - or force changes on how you would model your process.
For me, Sagas or the process Manager patterns are used when you have long running processes, which are distributed by nature, and want to have the transactional context. So the SEC or Process Manager is a state machine that reacts to the results from the previous state to move to the next one. This means triggering the execution of the next step.
If we look at the example you - and Caite/others - used there is a clear relationship. The stepfunction provides this state machine and the lambdas are the steps being executed.
All good so far, but what if the action being taken by this step is long-running on its own? Imagine that in an e-commerce solution you would have the steps of payment, shipment, and sending the email in that order. While payment and sending emails can likely be executed in seconds, the shipment is a long-running that could have a compensating aspect if, for example, the item is found to be out of stock or can't be sent (found to be damaged upon inspection).
In a more 'traditional' saga implementation, I would ask for the shipment to happen and simply go to sleep until I receive a message (event) with the ShipmentSent or ShipmentCancelled to resume and go to the next step.
With step functions it feels I would have to essentially break the process in two. One that would go payment, then trigger the shipment and stop. Another that would be triggered with the shipment event and have the compensating step for the payment or the successful path to send the email. And since they are different processes the state would have to be somehow recreated as I can't reuse from the previous process.
I wonder what would be your assessment on the subject because having this breakdown feels artificial but seems like the only solution if one still wants to use AWS step function.
regards
Pausing the step function flow until a longer running process or human approval process happens is totally possible. AWS just doesn't seem to shout about it enough. Here is an example - docs.aws.amazon.com/step-functions...
Thank you Matt. I will take a look in details to see if it solves the issue.