IoT for Dummies: Building a Basic IoT Platform with AWS

This article will guide you through creating the fundamental functionalities of an IoT platform using AWS, with a practical use case focused on monitoring electrical grid parameters to enhance efficiency and contribute to reducing the carbon footprint.

Understanding the Use Case

In the quest for net-zero emissions, electric distribution companies face immense pressure to reduce their carbon footprint and transition towards sustainable energy solutions. Efficient management of the electrical grid is critical in this endeavor, as it optimizes energy use, integrates renewable energy sources, and reduces emissions associated with electricity generation and distribution.

The use case involves deploying sensors and network monitoring devices within the electrical infrastructure. These devices collect real-time data on energy consumption, renewable energy generation, energy demand, and other relevant parameters. The data will be processed and analyzed to identify opportunities for improving energy efficiency and reducing emissions. By leveraging AWS's advanced analytics capabilities, proactive measures can be taken to optimize grid operation and move towards a sustainable future.

Project Objectives

1. Integrate the New IoT Platform within Corporate Landing Zone Standards and Regulations:

   • The company has a pre-configured landing zone that meets the standards and regulations for a large enterprise with multiple subsidiaries. Each subsidiary has its own organizational unit to enable agile development while adhering to global standards in networking, security, and shared components. The new IoT platform must comply with these requirements.

2. Design and Create a Scalable IoT Platform:

   • The company plans to deploy over 10,000 devices from three hardware providers. The initial version aims to provide a secure Fleet Provisioning capability to simplify the installation of these devices in each substation. Network data must be collected from each device and stored for future analysis and processing.

3. Ensure Automation from Device Provisioning to Data Collection:

   • Automation is key, from provisioning IoT devices to collecting data. This follows infrastructure-as-code principles using Terraform for automation.

Step-by-Step Guide to Building the IoT Platform

1. Connecting Devices

The devices are on-premises, and the first task is to connect them to AWS IoT Core.

• AWS IoT Core: This managed cloud service allows connected devices to interact securely with cloud applications and other devices. It can support billions of devices and trillions of messages, reliably processing and routing those messages to AWS endpoints and other devices.

2. Fleet Provisioning

AWS offers several methods to provision devices and install unique client certificates:

Devices can be connected using three types of provisioning methods:

• Just-in-time provisioning (JITP): If you can securely install unique client certificates on your IoT devices before delivering them to the end user, you should opt for just-in-time provisioning (JITP) or just-in-time registration (JITR).

• Provisioning by trusted user: If it's not feasible to securely install unique client certificates on your IoT devices prior to delivery, but the end user or an installer can use an app to register the devices and install the unique device certificates, the provisioning by trusted user process is suitable.

• Provisioning by claim: If end users cannot use an app to install certificates on their IoT devices, the provisioning by claim process can be used. This method involves your IoT devices having a claim certificate shared by other devices in the fleet. When a device connects for the first time using a claim certificate, AWS IoT registers the device using its provisioning template and issues it a unique client certificate for future access to AWS IoT. This method allows automatic device provisioning upon connection to AWS IoT but poses a higher risk if a claim certificate is compromised. If a claim certificate is compromised, it can be deactivated to prevent future registrations with that certificate, though already provisioned devices will not be affected.

Provisioning by Claim

This method uses a certificate (AWS Private Certificate Authority (PCA) certificate) shared with AWS Resource Access Manager (RAM). It is effective for mass provisioning and managing device credentials securely.

• AWS Private Certificate Authority (PCA): Best practices include regular rotation of certificates and minimizing their scope to reduce the risk if compromised. Isolate your PCA in its own AWS account to minimize unauthorized access risk. Share certificates across AWS accounts securely using AWS RAM. Terraform is used as IaC, here there are examples about how a pca is set up:

• Provisioning Template: Create a template that defines policies and configurations for the devices to ensure consistent security standards.

• Provisioning Flow: The device uses the shared certificate to connect to AWS IoT Core. AWS IoT Core validates the certificate and applies the provisioning template to register and configure the device in the cloud.

1. Present Bootstrap Certificate: Edge devices initially connect to AWS IoT Core using a bootstrap/claim certificate.
2. Birth Policy Execution: The birth policy is executed, which includes a Certificate Signing Request (CSR) that is signed and returned.
3. Official Certificate Payload: The device receives its official certificate payload for secure communications.
4. Send Ownership Token and Specify Provisioning Template: The device sends an ownership token and provisioning template to AWS IoT Core.
5. Execute Provisioning Template:
   • Custom Provisioning Validation: Validates the provisioning request.
   • Activate Certificate: Activates the device's official certificate.
   • Create Thing/Group: Creates the device entity (Thing) or associates it with a group.
   • Assign Policy: Assigns the necessary security policies to the device.
6. Respond with Outcome of Provisioning Transaction: AWS IoT Core confirms the outcome of the provisioning transaction.

3. Data Ingestion and Processing

• Set Up AWS IoT Rules: Create rules to process incoming data, routing it to other AWS services for further processing.
• Sending Data to a Data Lake: Use IoT rules to send data to an Amazon S3 data lake and an analytics platform in another AWS account. This involves setting up a Lambda function to enrich the data and using Amazon SQS to decouple the systems for efficient processing.

A diagram with MQTT connections to AWS IoT Core, its suborditane CA in the same account and the isolate root CA in other account is picture here. Besides, and IoT Rule is added to send infortation to the datalake (S3) and other account to process information (lambda + SQS).

4. Testing the Setup

• Script for Device Testing: Develop a script to simulate data transmission from the device to AWS IoT Core. This ensures communication and data ingestion functionality. There is an example in aws repository: https://github.com/aws/aws-iot-device-sdk-python-v2/blob/main/samples/pubsub.py
• MQTT Test: Use the MQTT test client in AWS IoT Core to publish and subscribe to topics, verifying data flow between the device and AWS IoT Core.

5. Execute Actions on the Devices: AWS Jobs

• Firmware Updates: AWS IoT Jobs facilitate communication from the cloud to devices for tasks such as firmware updates, ensuring all devices remain up-to-date and secure. Use AWS IoT Jobs to manage remote operations for one or multiple devices connected to AWS IoT.

To create jobs, start by defining a job document containing instructions for the remote operations the device should perform. Then, specify the targets for these operations, which can be individual things, thing groups, or both. The combination of the job document and the specified targets constitutes a deployment.

AWS IoT Jobs notifies the targets that a job is available. The target devices then download the job document, execute the specified operations, and report their progress back to AWS IoT. You can track the job's progress for specific targets or for all targets using AWS IoT Jobs commands. Once a job starts, it has an "In progress" status, and devices will report incremental updates until the job is completed, fails, or times out.

Next Advanced Steps

6. Analyzing Data with AWS IoT Analytics

• Create Data Sets: Define data sets in AWS IoT Analytics to process and transform the raw data.
• Run Analyses: Utilize built-in analytics capabilities to run SQL queries and perform machine learning on the data to derive insights.

7. Visualizing Data

• AWS QuickSight: Create dashboards and visualize the data to understand patterns and trends in energy consumption and generation.
• Real-Time Alerts: Set up real-time alerts using AWS IoT Events to notify operators of anomalies or inefficiencies in the grid.

8. Integrating Advanced Machine Learning with Amazon SageMaker and use GenAI with Amazon Bedrock

• Amazon SageMaker: Use Amazon SageMaker to build, train, and deploy machine learning models with the data collected from IoT devices.
• Amazon Bedrock: Leverage Amazon Bedrock to simplify the development and deployment of machine learning models.

Main Benefits of the AWS IoT Platform

• Scalability: AWS IoT services can scale to handle increasing amounts of data and devices.
• Security: Robust security features ensure the data collected and transmitted is secure.
• Flexibility: AWS offers a range of services that can be tailored to specific needs, allowing for a flexible and customizable IoT platform.

Conclusion

By following these steps, you can build a basic IoT platform with AWS that not only monitors electrical grid parameters but also contributes to the path towards net-zero emissions. Leveraging IoT technology and AWS's comprehensive suite of services, electric distribution companies can optimize energy use, integrate renewable sources more effectively, and significantly reduce their carbon footprint. This approach is not limited to this specific use case; it can be adapted and applied to any IoT scenario, demonstrating AWS's versatility in enabling sustainable solutions across diverse industries.