Mastering AWS Step Functions: Unleashing the Power of Nested Workflows

Amazon Step Functions is a powerful service provided by AWS that enables you to coordinate and manage multi-step workflows involving various AWS services. One of the key features of Step Functions is the ability to trigger child workflows from a parent workflow. This allows you to break down complex processes into smaller, more manageable steps.

In this blog post, I will guide you through the process of setting up a parent workflow in Step Functions that triggers the execution of a child workflow. I will provide detailed explanations and examples to help you get started.

Introduction to Step Functions

Amazon Step Functions is a serverless orchestration service that allows you to coordinate and manage workflows. Workflows are defined as state machines, where each state represents a step in the process. States can be tasks for AWS services or actions defined by AWS Lambda functions.

Defining the Parent Workflow

Start by creating a new Step Functions state machine or editing an existing one. Define the steps of the parent workflow, including the step that will trigger the child workflow.

The parent workflow is the higher-level orchestration that manages the overall process. It may consist of multiple steps, including the step that triggers the child workflow.

Step Function Definition for Parent Workflow

{
  "Comment": "Parent Workflow",
  "StartAt": "Step1",
  "States": {
    "Step1": {
      "Type": "Task",
      "Resource": "arn:aws:states:::states:startExecution.sync",
      "Parameters": {
        "StateMachineArn": "arn:aws:states:us-east-1:123456789012:stateMachine:ChildWorkflow",
        "Input.$": "$.inputForChildWorkflow"
      },
      "Next": "Step2"
    },
    "Step2": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:MyLambdaFunction",
      "End": true
    }
  }
}

Explanation:

• Comment: A descriptive comment for the state machine.
• StartAt: The initial state of the state machine.
• States: Defines the individual states and their properties.

Task to Trigger the Child Workflow

Let me explain a bit more on the Task state that is responsible for triggering the child workflow in the above state machine definition.

You can see that I have set that I have set Resource as arn:aws:states:::states:startExecution.sync and that will basically say that I want to execute another state machine in this step.

StateMachineArn variable defines what state machnie I want to execute, you can set it to the arn of the particular ChildWorkflow state machine. You can go many nested levels as this and you can break down complex processes into smaller, more manageable nested flows.

"Step1": {
  "Type": "Task",
  "Resource": "arn:aws:states:::states:startExecution.sync",
  "Parameters": {
    "StateMachineArn": "arn:aws:states:us-east-1:123456789012:stateMachine:ChildWorkflow",
    "Input.$": "$.inputForChildWorkflow"
  },
  "Next": "step2"
}

Passing input variables to the Child Workflow

The $.inputForChildWorkflow syntax is a placeholder indicating where to find the input data within the parent workflow's input.

Child Workflow

Create a separate Step Functions state machine for the child workflow. Define the steps of the child workflow, including any tasks or actions you want it to perform.

Step Function Definition for Child Workflow

Here is an example of a simple state machine that invoke a lambda function, we can use this as our child workflow.

{
  "Comment": "Child Workflow",
  "StartAt": "Step1",
  "States": {
    "Step1": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:ChildLambdaFunction",
      "End": true
    }
  }
}

Make sure you give add the arn of the this child state machine in your parent orchestrator state machine definition accordingly as I mentioned above.

Testing and Execution

When you start the parent workflow, it will progress through its steps. When it reaches the state that triggers the child workflow, it will use the startExecution API to start the child workflow.

Monitoring and Managing Workflows

You can monitor the progress of both the parent and child workflows in the AWS Step Functions console. This allows you to track the execution of each step and ensure that the overall workflow is functioning as expected.

By following above steps, you can build a blue print of an orchestration of complex workflows using Amazon Step Functions. Feel free to adoot this to any of your automations and I’m looking forward to see your creativity.

Happy orchestrating!