How to use intrinsic functions within AWS Step Functions

I recently had a use case for extending a state machine publish to SNS topic step to include a unique identifier on a per-execution basis. The input of the execution for the state machine was going to be the same across all of them, so creating an execution with a unique identifier inside of the input wasn’t really an option.

This seemed like it could be quite a tricky problem to solve, but to my delight, AWS supported just the thing I needed directly in Amazon State Language (ASL) - enter intrinsic functions!

❓ What the function?

Intrinsic functions are essentially little middleware helper functions that ASL supports natively to provide some low-level basic data transformation and generation.

Per the documentation:

The Amazon States Language provides several intrinsic functions, also known as intrinsics, that help you perform basic data processing operations without using a Task state

There are a plethora of different functions that they provide out of the box, but I’m just going to give an example of how I’ve used one recently to generate a UUID at the time of execution.

🤖 State Machine Definition

Going back to my use case, I wanted to create a unique identifier that I could pass along with the content that gets published to the SNS topic. After it had published to SNS, I then added an additional step to push an item into a DynamoDB table, for some internal tracking purposes.

So my initial state machine definition looked something like:

{
  "Comment": "My state machine that does not use intrinsic functions",
  "StartAt": "SNS Publish",
  "States": {
    "SNS Publish": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sns:publish",
      "Parameters": {
        "TopicArn.$": "$.input.snsTopicArn",
        "Message": {
          "content.$": "$.input.content"
        }
      },
      "ResultPath": "$.sns.output",
      "Next": "DynamoDB PutItem"
    },
    "DynamoDB PutItem": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:putItem",
      "Parameters": {
        "TableName": "internal-state-management",
        "Item": {
          "content": {
            "S.$": "$.input.content"
          }
        }
      },
      "End": true
    }
  }
}

I wanted to extend this state machine to generate a UUID per execution and pass it through to the various steps for internal tracking purposes. Some of the additional metadata is omitted, but the definition remains the same.

💡 How?

Firstly, intrinsic functions can be used within the Parameters object in a Pass state within ASL. Pass states can be used to pass input from one state to the next, but it can also be used to transform the input that it receives and mutate it to suit your needs.

In our case, we can leverage a Pass state and its parameters to enrich our input with a UUID that we can use to uniquely identify the execution for tracking purposes across the SNS topic publish message and in our DynamoDB table.

Looking at the AWS documentation for intrinsic functions within ASL there is an option to generate a UUID using the States.UUID() syntax.

Critically however, the documentation calls out that we need to utilise the .$ notation in our key for it to register the fact that it is a function:

To use intrinsic functions you must specify .$ in the key value in your state machine definitions

Our Pass state can look like the following:

"Generate UUID": {
 "Type": "Pass",
 "Next": "SNS Publish",
 "Parameters": {
   "uuid.$": "States.UUID()"
 },
 "ResultPath": "$.identifier"
}

A few critical things are taking place here:

• We’re leveraging the Parameters object that a Pass state can contain to generate our uuid and assign it to a variable.
• You’ll also notice that we’re using the ResultPath property on the state to actually specify that we want to add the output of this to a new path (identifier), meaning that the original input will also remain as part of the output.

Once this step completes in the execution, we will have our initial input alongside our newly created identifier object that contains the uuid within it — this forms the output of this step, something like:

{
  "input": {
    "snsTopicArn": "<topicArn>",
    "content": "Hello world!"
  },
  "identifier": {
    "uuid": "70b0d816-abfe-4675-a670-ab04c5634aee"
  }
}

💫 Utilising the UUID

Now that we have our computed uuid that can form as our unique identifier for this entire execution of the state machine, we can reference it where we require it using the same JSONP syntax.

In our case, we can update the SNS publish call as well as the DynamoDB PutItem call to include this identifier — resulting in our state machine definition now looking like:

{
  "Comment": "My state machine that now uses intrinsic functions!",
  "StartAt": "Generate UUID",
  "States": {
    "Generate UUID": {
     "Type": "Pass",
     "Next": "SNS Publish",
     "Parameters": {
       "uuid.$": "States.UUID()"
     },
     "ResultPath": "$.identifier"
    },
    "SNS Publish": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sns:publish",
      "Parameters": {
        "TopicArn.$": "$.input.snsTopicArn",
        "Message": {
          "uuid.$": "$.identifier.uuid",
          "content.$": "$.input.content"
        }
      },
      "ResultPath": "$.sns.output",
      "Next": "DynamoDB PutItem"
    },
    "DynamoDB PutItem": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:putItem",
      "Parameters": {
        "TableName": "internal-state-management",
        "Item": {
          "uuid": {
            "S.$": "$.identifier.uuid"
          },
          "content": {
            "S.$": "$.input.content"
          }
        }
      },
      "End": true
    }
  }
}

🎥 Action!

Let's execute our state machine and see the result! The input I'm using for this execution is the same as what has been used above:

{
  "input": {
    "snsTopicArn": "<topicArn>",
    "content": "Hello world!"
  }
}

Whenever we kick off our state machine execution, it completes as expected. Let's have a look at each block then to verify it's doing what we expect:

Above, we can see the output from the UUID generation step. We have successfully utilised the intrinsic function to generate a UUID and append it onto the initial input using our ResultPath syntax.

Verifying that it is passed through to our other steps then, you can see that the identifier object has been updated and the result passed to the next state. This means that it can be used and referenced in the block, just like we are doing.

Finally, you can see that the input for the final DynamoDB task includes all of the original input, along with the output from the SNS task.

Success!

✅ Conclusion

There are a number of different instrinsic functions and operations you can perform, for all various use cases. I thought it would be worth putting together a small example to showcase my specific use case for it.

These help to simplify your state machines, reducing the number of Task states you might need to simpler data transformation operations. Pretty cool!

Make sure you read up on all of the functions available on the AWS documentation here—but they are all used in a similar way.

Happy functioning!