DEV Community

Cover image for Coordinating Nested Step Functions

Coordinating Nested Step Functions

forstmeier profile image John Forstmeier ・2 min read

As of the time of this writing, AWS does not currently support the pattern of nesting Step Functions within Step Functions.

For example, you might have a series of "child" steps that you want to be kept separate from the "parent" Step Function definition but want to be included as part of the "parent" execution. While it's not officially supported (yet), it is possible to implement.

Using the GetActivityTask API method call you can retrieve a task token from an Activity in the Step Function which will block until told to progress. This token from the "parent" workflow is then passed into the "child" Step Function and the last State in this "child" definition is responsible for making a SendTaskSuccess or SendTaskFailure API call back to the "parent."

Here's an example in Java of what that would look like within the last task in the child step function.

    String parentToken = requestJSON.getString("parent_token");

    SendTaskSuccessRequest request = new SendTaskSuccessRequest();

    SendTaskSuccessResult result = client.sendTaskSuccess(request);

Enter fullscreen mode Exit fullscreen mode

In order to retrieve the task using the GetActivityTask API call, both the AWS::StepFunctions::Activity and a "poller" AWS::Lambda::Function were placed into a Parallel state in the state machine definition - AWS's recommended pattern involves including the poller as a separate thread in the worker, but we opted out of this since we've trying to remain as stateless as possible in our architecture. I'm approaching this as if everything's being defined in a CloudFormation template.

There are several caveats to this structure with the most important being that AWS places an API call limit on GetActivityTask which you'll need to be aware of depending on how much activity your step function sees.

I'm sure there are other ways of coordinating Step Functions (and it sounds like there may be something under development internally at AWS to that effect) but this is the pattern our team went with.

Discussion (0)

Forem Open with the Forem app