Introduction to AWS Step functions by Automated Image-Processing Using Amazon Rekognition

Hi everyone,

We are implementing an automated image-processing workflow using state machines in step functions. It will cover the basics of step functions workflow

Before jumping into the topic let me introduce the step functions and terminology we are going to use in this article

**Step functions: **It’s a managed service that helps to combine AWS services to create workflows. It will provide a visual workflow editor. We can just drag and drop the services and combine them to create a workflow

Terminology:

State: Each step in the workflow is defined as a state. A state will have input, outputs, and error-handling

State Machine: A workflow is defined as a state machine. It contains the states. By default, the editor will provide **start **and **end **states

Overview: Here we will create 4 lambda functions and connect them using the step functions to automate image processing using the AWS Rekognition service

API-receiver: This lambda function receives the image through API-Gateway and triggers the state machine

Image-uploader: This function will upload the image to s3 bucket and return the image data

Object-receiver: This function will receive the image data pass that info to the AWS recognition service and return that data

Store-to-dynamo-db: This function will take the image info and store that data in the Dynamo Db

Let’s start

Step 1: Create a lambda function to upload images to S3

Here is the code for the lambda function

import json
import base64
import boto3
from datetime import datetime


def lambda_handler(event, context):

    ##rececing base 64 image and data and decoding it
    image_data = base64.b64decode(event['base64Image'])

    s3 = boto3.client("s3")

    ##generating the name for the image to upload to s3
    now = datetime.now()
    date_time_string = now.strftime("%Y-%m-%d %H:%M:%S")

    bucket_name = "images-rek-test"
    object_key = f"{date_time_string}.png"
    region = "us-west-2"



    try:
        ##uploading image to s3
        s3.put_object(Bucket=bucket_name, Key=object_key, Body=image_data)

        response = {
            "bucket_name":bucket_name,
            "object_key":object_key,
            "region":region
        }

        return {
            'statusCode':200,
            'body': response
        }
    except Exception as e:
        print(e)
        return {
            'statusCode': 500,
            'body': json.dumps("Error in uploading image")
        }

Step 2: Create a lambda function for image processing:

Here is the code

import json
import boto3

def lambda_handler(event, context):

    s3_client = boto3.client('s3')
    rekognition_client = boto3.client('rekognition')

    bucket_name = event["body"]['bucket_name']
    object_key = event["body"]['object_key']


    print(f"keys {bucket_name} and {object_key}")

    ##sending uploaded image info to rekognition service
    response = rekognition_client.detect_labels(
        Image={'S3Object': {'Bucket': bucket_name, 'Name': object_key}},
        MaxLabels=10
        )

    print(response)

    return {
        'statusCode': 200,
        'body': response
    }

Step 3: Create a Lambda function to store data in DynamoDb

• Create a table in the dynamo DB with id as the primary key

Here is the code

import json
import boto3
from datetime import datetime
from decimal import Decimal

##converting floats to decimal as dynamo db won't support
def convert_floats_to_decimal(obj):
    if isinstance(obj, float):
        return Decimal(str(obj))
    elif isinstance(obj, dict):
        return {k: convert_floats_to_decimal(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [convert_floats_to_decimal(v) for v in obj]
    else:
        return obj



def lambda_handler(event, context):

    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table('image_data')##table name in dynamo db


    rekognition_data = convert_floats_to_decimal(event['body'])

    ##generate an id with date and time to store in table
    now = datetime.now()
    date_time_string = now.strftime("%Y-%m-%d %H:%M:%S")

    item = {
        'id': date_time_string, 
        'RekognitionResponse': rekognition_data
    }

    table.put_item(Item=item)##stroing the data to db

    return {
        'statusCode': 200,
        'body': 'Data stored in DynamoDB successfully'
    }

• Now we have the Lambda functions ready, Let’s create the state machine in step function

Step 4: Create a state machine in step functions:

• Visit the step functions service from the AWS search bar

• Click on create state machine and select a blank template

• From the side menu drag and drop the Lambda invoke

• From the right side menu update the lambda configuration, input, and output

• Give a name for the state, From API Parameters select the function

• Leave the input empty

• Transform the output like this to pass info to the next state

. This is the state the machine looks like now

• Drag another Lambda invoke state and keep it under the first function

• Configure the second Lambda function configuration, Input and output like this

• Update the state name, select the lambda function, leave the input as it is, and change the output like the previous state

• Now drag and drop another Lambda invoke function below the second Lambda function

• Update the Lambda function configuration, leave the input as it is, and output like this

• Save the state machine from the top right corner

• While creating the state machine you can test each state by selecting it.

Step 5: Create a Lambda function to trigger this state machine

• Copy the state machine ARN and paste it into the state_machine_arn variable

  import json
  import boto3

  def lambda_handler(event, context):

  client = boto3.client('stepfunctions') 
  
  state_machine_arn ="arn:aws:states:{region}:{accountid}:stateMachine:MyStateMachine-jz7b4j339"
  
  input_data = json.dumps(event)
  
  ##trigger the state machine
  response= client.start_execution(
      stateMachineArn=state_machine_arn,
      input=input_data
      )
  
  return{
      'statusCode':200,
      'body': json.dumps('Successfully started state machine')
  }
  

Step 6: Create an API-Gateway API to trigger the lambda function

• Visit the API-Gateway service from the AWS Search bar and visit the API Gateway service

• Create a POST call click on the integration request and select the lambda function we created above in the mapping template enter the code like this

• From the API Setting enable binary media types and enter image/png as the media type

• Deploy the API and copy the API URL and from the postman trigger that URL

• It will start the State machine and you can see the results stored in the dynamo DB

• You can see the execution flow from the state machine below and see the results in the dynamo db

• That’s it. We successfully created a state machine to process the images

• Go to the step function and visit all the features available, keep on trying other workflows

• There will be another article with more info about workflows

Limitations: Here state machine input accepts only around 250kb of data, so if you want to input bigger images better to do them from S3 Bucket directly.