AWS Step Function Background

AWS Step Functions is one of the AWS services which allows us to define workflow by integrating the existing AWS services such as Lambda, Fargate, SNS and etc. Each workflow is based on the concept of a state machine and it is defined using Amazon States Language (ASL) (JSON-based).

Workflow on Step Functions is made up of series of steps, with the output of one step acting as input into the next step. Thus, you can perform a series of actions to the workflow input. Due to the limitation of data passed between each steps, it is best to use a separate service like AWS S3 to transfer the large files (Limit).

The benefit of using Step Function is that the application logic is implemented within the workflow and it is separated from the implementation of individual component (step). This actually promotes the reusability of each component for other workflows.

A simple state machine is defined as below. StartAt and States are the compulsory building blocks. StartAt defines the entry point of your workflow and States defines the steps for your customised workflow. There are other optional components that you can have within the state machine.

{
  "Comment": "A Hello World example of the ASL.", // Optional
  "StartAt": "HelloWorld", // Compulsory
  "States": {              // Compulsory
    "HelloWorld": {
      "Type": "Pass",
      "Result": "Hello World!",
      "End": true
    }
  }
}

To define the steps within the workflow, these are the available types of states provided by ACL now:

Sample Step Functions Workflow

This article is going to cover some useful features of ASL and how to create a simple workflow by using AWS Lambda functions. Sample AWS Lambda functions are available here. Before starting, you will need to deploy the AWS Lambda functions from the sample code. Do check out here if you need some help on the Lambda deployment.

The sample Step Functions workflow diagram is as below:

The basic idea of this workflow is to:
1) Configure the random-number-generator-lambda to generate a random number from a number range.
2) Then, trigger the random-number-generator-lambda to generate a random number.
3) Based on the output of the random-number-generator-lambda, the next step is to check if the output is above or below number 5.
4) If it is above 5, SNS notification will be triggered to send out SMS.
5) If it is below 5, another Lambda power-of-number-lambda will be triggered to raise the previous input to the power of configured exponents.
6) Then, the workflow will end after that.

Here is the explanation of ACL features being used for this sample workflow:
1) random-number-generator-lambda-config - To configure the random-number-generator-lambda by using the Pass state.

"random-number-generator-lambda-config": {
  "Comment": "To configure the random-number-generator-lambda.",
  "Type": "Pass",
  "Result": {
      "min": 1,
      "max": 10
    },
  "ResultPath": "$",
  "Next": "random-number-generator-lambda"
},

2) random-number-generator-lambda - To trigger a Lambda function by using the Task state.

"random-number-generator-lambda": {
  "Comment": "Generate a number based on input.",
  "Type": "Task",
  "Resource": "${random-number-generator-lambda-aws-arn}",
  "Next": "send-notification-if-less-than-5"
},

3) send-notification-if-less-than-5 - To check if the output from previous step is less than 5 using the Choice state.

"send-notification-if-less-than-5": {
  "Comment": "A choice state to decide to send out notification for <5 or trigger power of three lambda for >5.",
  "Type": "Choice",
  "Choices": [
    {
        "Variable": "$",
        "NumericGreaterThanEquals": 5,
        "Next": "power-of-three-lambda"
    },
    {
      "Variable": "$",
      "NumericLessThan": 5,
      "Next": "send-multiple-notification"
    }
  ]
},

4) power-of-number-lambda - If output is more than 5, trigger another Lambda function using Parameters to customise the input JSON object.

"power-of-three-lambda": {
  "Comment": "Increase the input to power of 3 with customized input.",
  "Type": "Task",
  "Parameters" : {
    "base.$": "$",
    "exponent": 3
  },
  "Resource": "${power-of-number-lambda-aws-arn}",
  "End": true
},

5) send-multiple-notification - A Parallel state to trigger multiple actions.
6) send-sms-notification - To trigger SNS notification to send SMS.
7) send-email-notification - To trigger SNS notification to send Email. (Not include as part of the sample)

"send-multiple-notification": {
  "Comment": "Trigger multiple notification using AWS SNS",
  "Type": "Parallel",
  "End": true,
  "Branches": [
    {
     "StartAt": "send-sms-notification",
     "States": {
        "send-sms-notification": {
          "Type": "Task",
          "Resource": "arn:aws:states:::sns:publish",
          "Parameters": {
            "Message": "SMS: Random number is less than 5 $",
            "PhoneNumber": "${valid-handphone-number}"
          },
          "End": true
        }
     }
   },
   {
    "StartAt": "send-sns-topic",
     "States": {
       "send-sns-topic": {
          "Type": "Task",
          "Resource": "arn:aws:states:::sns:publish",
          "Parameters": {
            "Message": "Email: Random number is less than 5: $",
            "TopicArn": "${aws-sns-topic-to-send-out-email}"
          },
          "End": true
        }
     }
   }  
  ]
}

You can trigger the workflow by using the Start Execution button within the Step Functions console. Each invocation allows JSON object as input to the workflow, but it is not compulsory. Or Cloudwatch Schedule event can be configured to trigger the workflow.

Before that, it is also important to ensure that you have granted the appropriate permission to the IAM role for the Step Function execution. For this sample workflow, you will need the following

• AWS Managed Policy - AmazonSNSFullAccess - To trigger SNS notification SMS or Email.
• AWS Lambda Invocation Permission.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction",
                "lambda:InvokeAsync"
            ],
            "Resource": "*"
        }
    ]
}

You should see something as below if the execution is successful. Do try a few more rounds to see you get a SMS from the SNS notification!

Not all states are being covered in the above sample workflow, more can be found from official doc on the different ASL features.

Hope this is useful for your exploration on Step Functions.

Reference:

• AWS Step Functions
• AWS State Language Official Documentation
• AWS State Language - Different States