Introduction

In this blog, I will walk you through how we can use AWS IOT thing to decouple the frontend application from the backend.

Basically, the frontend talks to an API Gateway through a rest endpoint. We have two methods one to get all animals in the database. And another method to insert an animal.

This is a configuration walkthrough blog, meaning the frontend app is very minimalistic.
The frontend consists of a simple Angular 10 application.
To checkout the full code, here is the GitHub repo

Architecture

As you can see, the backend consists of:

• an API Gateway with a rest endpoint with two methods
• A DynamoDB table with Streams enabled on it
• An AlertIOTFunction that gets trigged on the STREAMS change
• An IOT topic, that is used by the AlertIOTFunction to publish a message to.

So on a high level, we can imagine a system where a customer does an action, in this case, adds an animal to the database. This insert, triggers a stream that calls a lambda, which can trigger a process for a payment, or a confirmation or something that can take some time ⏳.

In our case, this process only takes the newly added animal, and publishes it to an IOT topic. And we can see it in the client's console and act on it if needed (which is most likely to happen 🙄 )

Code Examples

Frontend

For the frontend everything is in the aws-examples inside the github repo. To run it you can follow the README.

To subscribe to the IOT topic, we are using an AWS library called aws-iot-device-sdk. (we could use the MQTT.js directly if we want.)

To make it work with the frontend application, I have added the following in the package.json:

"browser": {
   "fs": false,
   "tls": false,
   "path": false
},

Without this piece, running the app will result in build errors: ERROR in ./node_modules/aws-iot-device-sdk/common/lib/tls-reader.js
Module not found: Error: Can't resolve 'fs' in '/Users/.../aws-examples/aws-examples/node_modules/aws-iot-device-sdk/common/lib'

Plus, we have to add the following piece in the polyfill.ts:

(window as any)['global'] = window;
global.Buffer = global.Buffer || require('buffer').Buffer;

import * as process from 'process';
window['process'] = process;

without it the browser will complain that index.js:43 Uncaught ReferenceError: global is not defined

The code is pretty straightforward. In the app.component.ts
in the constructor we are connecting to the IOT Topic.

ℹ️ As you know everything that needs access to an AWS service needs credentials. This is why we are using Cognito. We are using it to generate temporary credentials so the application can subscribe to the IOT topic.

// 1
AWS.config.credentials = new AWS.CognitoIdentityCredentials({
    IdentityPoolId: this.AWSConfiguration.poolId
})

const clientId = 'animals-' + (Math.floor((Math.random() * 100000) + 1)); // Generating a clientID for every browser

// 2
this.mqttClient = new AWSIoTData.device({
    region: AWS.config.region,
    host: this.AWSConfiguration.host,
    clientId: clientId,
    protocol: 'wss',
    maximumReconnectTimeMs: 8000,
    debug: false,
    secretKey: '', // need to be send as an empty string, otherwise it will throw an error
    accessKeyId: '' // need to be send as an empty string, otherwise it will throw an error
});

On '1', the IdentityPoolId comes from the backend, where we deploy a template with some Cognito resources, it is explained below 🤓.

On '2', we are trying to connect to the IOT endpoint (explained in the README)

Moving to the ngOnInit, we can see the following:

this.mqttClient.on('connect', () => { // 1
    console.log('mqttClient connected')
    this.mqttClient.subscribe('animals-realtime')
});

this.mqttClient.on('error', (err) => { // 2
    console.log('mqttClient error:', err);
    this.getCreds();
});

this.mqttClient.on('message', (topic, payload) => { // 3
    const msg = JSON.parse(payload.toString())
    console.log('IoT msg: ', topic, msg)
});

this.http.get(`${this.api}get-animals` // 4
)
    .subscribe((data) => {
        console.log('data: ', data)
    });

On '1', we are listening to the connect event, if it is established correctly we are subscribing the the IOT topic created in AWS.

On '2', in case of an error we are calling the getCreds method. It is interesting to know, that the first time we run the app, connecting to the IOT topic will throw an error, because the credentials are not passed to the mqttClient, so in the error event we call the getCreds method to set the credentials correctly.

On '3', we are listening to the messaged that are published to the IOT topic, here we are just console logging it to keep things simple.

On '4', we are just making a request to API Gateway endpoint to get the animals in DynamoDB.

Moving to the getCreds method:

const cognitoIdentity = new AWS.CognitoIdentity(); // 1
(AWS.config.credentials as any).get((err, data) => {
    if (!err) {
        console.log('retrieved identity: ' + (AWS.config.credentials as any).identityId)
        var params = {
            IdentityId: (AWS.config.credentials as any).identityId as any
        }
        // 2
        cognitoIdentity.getCredentialsForIdentity(params, (err, data) => {
            if (!err) {
                // 3
                this.mqttClient.updateWebSocketCredentials(data.Credentials.AccessKeyId,
                    data.Credentials.SecretKey,
                    data.Credentials.SessionToken,
                    data.Credentials.Expiration
                )
            }
        })
    } else {
        console.log('Error retrieving identity:' + err)
    }
})

On '1' we are getting a Cognito Identity instance.
On '2' we are getting the credentials coming from Cognito
On '3' we are updating the mqttClient with the retrieved credentials.

To test this, we have a simple button, when we click it, it will call insertAnimal method that will simply post an animal to the database:

insertAnimal() {
    this.http.post(`${this.api}add-animal`, {
        "name": "cat",
        "age": 1
        // other fields ...
    }
    )
        .subscribe((data) => {
            console.log('data: ', data)
        });
}

After a couple of seconds we will receive a console in the console logs printing: IoT msg: animals-realtime ... 🎉

demo

Backend

The backend code is in /backend/iot
We have the resources defined in the template.yml. We deploy the backend using AWS SAM

To know how to deploy it, please follow the instructions in the project's README.

On a high level, in the template.yml you will find multiple resources:

• AnimalsRealtime the AWS IOT thing
• InsertAnimalFunction, a Lambda function that gets called when calling the api endpoint with /add-animal
• GetAnimalsFunction, a Lambda function that gets called when calling the api endpoint with /get-animals
• AlertIOTFunction, a Lambda function that gets trigger by a DynamoDB Stream
• AnimalsAPI, an API Gateway
• AnimalsTable, the DynamoDB database to store the items
• UserPool & UserIdentity, to give access to the frontend to subscribe to the IOT topic

Conclusion

To sum things up, there are many ways to decouple the frontend from the asynchronous/long-term backend processes. One of these approaches could be leveraging the IOT publish/subscribe methodology. Where a client execute an event, and subscribes to a topic. And when the backend finishes processing the needed tasks, it can publish results/notification to the topic.

In our case it was a simple action, returning the new added animal to the frontend. It can be more complicated than that, such as handling payment, approvals ...

I hope you have found this article useful. Please feel free to leave your remarks/questions in comments 🙏