AWS IoT Core Simplified - Part 4: Rules

We are now able to connect clients to the server, either using a presigned url or a custom authorizer. Clients can send messages to each other but in many cases you want to do something with these messages server side. This can be done with rules.

Queries & Actions

A rule basically consists of 2 parts; first, a query to capture data. Queries use an SQL-like syntax which offer a lot of flexibility. Second, actions can send data to services like Lambda, SQS, DynamoDB, Kinesis and many more.

Rule Example

Creating a rule is not difficult but there are a few pointers. Let's look at this example to get started:

SensorUpdatesRule:
  Type: AWS::IoT::TopicRule
  Properties:
    RuleName: myapp-prod-sensor-updates
    TopicRulePayload:
      Sql: "SELECT * FROM 'updates/sensor/+'"
      AwsIotSqlVersion: '2016-03-23'
      Actions:
        - Sqs:
            QueueUrl: !GetAtt SensorEvents.QueueUrl
            RoleArn: !GetAtt RuleRole.Arn
            UseBase64: False

This will capture all messages published to wildcard topic updates/sensor/+ and sends them to an SQS queue. As we learned before, this includes topics like updates/sensor/room-1 and updates/sensor/room-2.

Permissions

As always, this rule needs the permissions to perform the configured actions, so let's add this:

RuleRole:
  Type: AWS::IAM::Role
  Properties:
    RoleName: myapp-prod-iot-rule
    AssumeRolePolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Effect: Allow
          Principal:
            Service:
              - iot.amazonaws.com
          Action: sts:AssumeRole
    Policies:
      - PolicyName: myapp-prod-iot-rule-policy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - 'sqs:SendMessage'
              Resource:
                  - !GetAtt SensorEvents.Arn

Queue

And obviously we also need the queue we want to write to:

SensorEvents:
  Type: AWS::SQS::Queue
  Properties:
    QueueName: myapp-prod-sensor-updates

Now you can create a Lambda and add it as a trigger to the queue. You can also trigger a Lambda directly from the rule but I personally like to add a queue in between so I have more visibility and control over it (ie: batch size, concurrency, dead letter queue). If triggering Lambda directly has your preference, make sure to give your rule lambda:invoke permissions instead.

Queries

Queries are the most important part of a rule and it's very feature rich.

SELECT

SELECT is always required and defines which data you want to capture.

Capture everything
SELECT * -> {"temperature":21, "moisture": 50}

Select a specific field
SELECT temperature -> {"temperature":21}

Select multiple fields
SELECT temperature, moisture -> {"temperature":21, "moisture": 50}

Alias fields
SELECT temperature as temperature_celsius, moisture -> {"temperature_celsius":21, "moisture": 50}

Hard coded values
SELECT *, 1 as version -> {"temperature":21, "moisture": 50, "version": 1}

Single value
SELECT VALUE temperature -> 21

There are also many functions to transform your data in your query. I use this for instance to get the client ID and timestamp:

SELECT *, clientid() as clientId, timestamp() as timestamp

Another notable function is topic() which can be used to capture a segment of the topic as a variable:

SELECT *, topic(3) as room FROM 'updates/sensor/room-1'
->
{"temperature":21, "moisture": 50, "room": "room-1"}

FROM

The FROM part defines on which topic(s) the query should trigger.

Single topic
updates/sensor/room-1

Wildcard topic
updates/sensor/+ or updates/sensor/#

It's also possible to capture everything using a top level wildcard: #. This could be useful for analytics or debugging purposes but depending on how much data your clients are sending it could generate quite some data (and cost).

WHERE

The WHERE part is optional and allows you to narrow down which messages are captured. It's better to add the WHERE clause here than to discard the data in your Lambda function as it would simply save cost since the action won't run and the Lambda doesn't need to be executed.

Only capture high temperatures
SELECT * FROM 'updates/sensor/+' WHERE temperature > 25

High temperatures with low moisture
SELECT * FROM 'updates/sensor/+' WHERE temperature > 25 AND moisture < 40

All the basic operators you expect are supported out of the box.

Actions

Once you have queried the messages you want, you want to process them. In the example I've showed SQS as action but there are many more to choose from. You can for instance write directly to DynamoDB to query your data or log to CloudWatch to create nice metrics. Something I personally do is to write my events to a Firehose stream so I can easily log chunks of data to S3 per minute. Remember to always give your role the right permissions.

Saving money with Basic Ingest

A notable feature which could save you some money is basic ingest. With basic ingest you can let your clients send messages to the rule directly instead of sending it to the topic the rule is created for. A rule will create its own topic in this format: $aws/rules/{rule_name}. Sending messages to a rule directly instead of a topic saves you the cost of a message, depending on the amount of messages this could be very beneficial. The downside of this method is that other clients can not subscribe to it. Also make sure your client has permissions to publish to this topic, otherwise it will disconnect.

Closing

With this series of posts I hope to made it clear that IoT Core isn't as difficult as it seems. While IoT Core is very cheap (and has a generous free tier in the first 12 months!) there are obviously costs involved. IoT Core is however very cheap so fiddling around with it doesn't break the bank. Let me know in the comments what type of application you've built with it to inspire me and others.