Connect the Open Broadband USP Agent to AWS IoT Core using MQTT5, and Validate the Integration with AWS IoT Core Device Advisor

Contents

1. Introduction
2. Scope
3. Integrate obuspa with AWS IoT Core over MQTT5
4. Verify MQTT 5 Integration/Protobuf Data Publishing
5. Validate MQTT 5 Integration with AWS IoT Core Device Advisor “Qualification Program Test Suite”
6. Conclusion

Introduction

The User Services Platform (USP), also called TR-369, is a standardized protocol for managing, monitoring, upgrading, and controlling connected devices. It is widely used for customer premise equipment (CPE), such as modem routers. This protocol is the next generation follow-up for Broadband Forum’s CPE WAN Management Protocol (CWMP), commonly known as TR-069.

The Broadband Forum specification defines USP as "consisting of a collection of Endpoints (Agents and Controllers) that allow applications to manipulate Service Elements (Objects and Parameters that model a given service, such as network interfaces, software modules, device firmware, remote elements proxied through another interface, virtual elements, or other managed services)".

Generally, Agents run on CPE devices and can communicate with Controllers over a variety of message transport protocols (MTPs), one of which is MQTT. MQTT allows a secure, 24/7 open communication channel between Agents and Controllers. USP has specified the usage of both MQTT 5 and MQTT 3.1.1 from early specification versions.

With the recent general availability of MQTT 5 support in AWS IoT Core, I was eager to test the MQTT 5 integration between the AWS IoT Core MQTT Broker and USP Agents. This post is summarizing my findings.

For testing purposes, I will be using the Open Source, GitHub available, Open Broadband User Services Platform Agent (also referred to as OB-USP-Agent or obuspa), a reference C-based implementation of the USP protocol for Agents.

Scope

In this post, we will cover the following:

1. Configuring the OB-USP-Agent to connect to AWS IoT using MQTT 5.

2. Verifying successful connection to AWS IoT Core over MQTT 5, as well as subscriptions and data publishing.

3. Creating and running an AWS IoT Core Device Advisor “Qualification Program Test Suite”, to validate the MQTT 5 client/broker communication.

For those not familiar with AWS IoT Device Advisor, it is a Cloud-based, fully managed test capability for validating IoT devices during device software development. Device Advisor provides pre-built tests that you can use to validate IoT devices for reliable and secure connectivity with AWS IoT Core, before deploying devices to production. Device Advisor’s pre-built tests help you validate your device software against best practices for usage of TLS, MQTT, Device Shadow, and IoT Jobs. With the recent support for MQTT 5 in AWS IoT Core, MQTT 5 test suites have also been added to Device Advisor.

This blog post is a demonstration that obuspa can be configured to integrated with AWS IoT Core over MQTT 5. It is not a solution to be lifted and shifted for production use. The content also assumes some familiarity with USP, especially overall architecture, getting/setting parameters, and configuring USP subscriptions. If you are not already familiar with USP, these concepts are very well explained in the USP technical specification, provided by the Broadband Forum.

Integrate obuspa with AWS IoT Core over MQTT5

Our first goal is to get obuspa to connect successfully to AWS IoT Core, and perform the pub/sub over MQTT 5.

Before getting started, make sure you have the following list of per-requisites in place:

1. Download and install Docker. It is very convenient to run obuspa in Docker.

2. Make sure you have an AWS account, and select a region where the AWS IoT Device Advisor is available, for example: eu-west-1 (Ireland).

3. Enable AWS IoT Logs, for example via the AWS IoT Console. You can find instructions here.

To this purpose, it is worth pointing out that support for MQTT 5 client-side is already implemented as part of obuspa. For more details, have a look at the mqtt.c file in obuspa. With this in mind, getting obuspa running and connecting to AWS IoT Core using MQTT 5 should be a matter of configuration, not code.

To create and test this configuration, we will be using the test/mqtt folder, the existing shared.sh script (with some minor changes required, which we will discuss later). We will create another test script, inspired by the existing MQTT test scripts. We can call it test-mqtt-aws.sh, and it can look as below:

#!/bin/bash
CURRENT_DIR="$(dirname $0)"

echo $CURRENT_DIR
CERTS_ARGS="-t tests/mqtt/certs-iot.pem -a tests/mqtt/client-iot.pem"
source "$CURRENT_DIR"/shared.sh || exit 1

#CERTS_ARGS="-t /etc/ssl/certs"

rm -rf "$LOG_FILE"
rm -rf "$GREP_FILE"
rm -rf "$DB_FILE"

trap cleanup EXIT
trap cleanup SIGINT

stop_obuspa
sleep 1
start_obuspa $CERTS_ARGS

techo
techo "Preparing USP Agent for AWS IoT MQTT V5.0."

add_client 1
verify_client 1

add_local_agent_mtp 1 || fail
add_local_agent_controller 1  || fail
add_local_agent_controller_mtp 1 || fail

set_parameter_client "ClientID" "uspTestAgent"
set_parameter_client "ProtocolVersion" "5.0"

configure_client "1" "<YOUR_AWS_IOT_CORE_ENDPOINT>" "8883"

obuspa_cmd -c set "Device.MQTT.Client.1.TransportProtocol" "TLS"

# Enable boot notify (Controller 1)

obuspa_cmd -c add Device.LocalAgent.Subscription
obuspa_cmd -c add Device.LocalAgent.Subscription.1

obuspa_cmd -c set "Device.LocalAgent.Subscription.1.NotifType" "ValueChange"
obuspa_cmd -c set "Device.LocalAgent.Subscription.1.ReferenceList" "Device."
obuspa_cmd -c set "Device.LocalAgent.Subscription.1.Enable" "true"
obuspa_cmd -c set "Device.LocalAgent.Subscription.1.Recipient" "true"


# Finally, enable client.
enable_client 1

sleep 2
# Log and wipe grep file
cat "$GREP_FILE" >> $LOG_FILE
echo "" > "$GREP_FILE"

techo "USP Agent Ready"
wait

Before you can run this script, the following extra steps are needed:

1. Replace the placeholder YOUR_AWS_IOT_CORE_ENDPOINT with the endpoint from your AWS account. You can read about the IoT endpoint here.

2. Create the following AWS IoT Resources:

   • AWS IoT Device Policy (allowing iot:Connect,iot:Subscribe, iot:Publish, iot:Receive on the correct topics),
   • Your AWS IoT Thing, with thingName: uspTestAgent, a new certificate, and attach the newly created IoT Policy, either via the AWS Console, CLI, or SDK.

3. You then need to save the device certificate, private key, as well as the root CA, in 2 separate files, which you will reference when starting the obuspa agent.

   Note: certs-iot.pem must contain the root CA, and client-iot.pem the private key, followed by unique certificate, like this:
   

   -----BEGIN RSA PRIVATE KEY-----
   <body>
   -----END RSA PRIVATE KEY-----
   -----BEGIN CERTIFICATE-----
   <body>
   -----END CERTIFICATE-----
   

4. In the shared.sh script, in the add_local_agent_mtp(), the ResponseTopicConfigured parameter is configured to '/usp/endpoint/#’. According to the TR-181 Data Model specification, the value MUST NOT contain any wild card characters (“+”, “#”). So, you need to modify the Set cli call, and remove the # in the topic (Not doing this will force an MQTT client disconnect):
   

    obuspa_cmd -c set "Device.LocalAgent.MTP.$1.MQTT.ResponseTopicConfigured" "/usp/endpoint"
   

   Additionally, set the QoS 1 on publish:
   

    obuspa_cmd -c set "Device.LocalAgent.MTP.$1.MQTT.PublishQoS" "1"
   

5. As a last step, to be able to start up the test script as you run your Docker container, you can replace the existing Docker file, with the following:
   

FROM ubuntu:focal

ENV MAKE_JOBS=8
ENV OBUSPA_ARGS="-v4"

# Add mosquitto latest ppa
RUN apt-get update && \
    apt-get install -y software-properties-common && \
    add-apt-repository ppa:mosquitto-dev/mosquitto-ppa

# Install dependencies
RUN apt-get update &&\
    apt-get -y install \
        libssl-dev \
        libcurl4-openssl-dev\
        libsqlite3-dev \
        libc-ares-dev \
        libz-dev \
        autoconf \
        automake \
        libtool \
        libmosquitto-dev \
        libwebsockets-dev \
        pkg-config \
        make \
        &&\
        apt-get clean

# Copy in all of the code
# Then compile, as root.
COPY . /obuspa/
RUN cd /obuspa/ && \
    autoreconf -fi && \
    ./configure && \
    make -j${MAKE_JOBS} && \
    make install

CMD cd obuspa &&  tests/mqtt/test_mqtt_aws.sh

Build and run your Docker image, and your test will run and start obuspa, connected to AWS IoT Core, using MQTT 5.

Verify MQTT 5 Integration/Protobuf Data Publishing

Once you have successfully run obuspa in Docker, you should see in the log file of the Agent log statements around the successful transition of the MQTT client state:

SendingConnect → [[ Connect sent ]] → AwaitingConnect, to AwaitingConnect → [[ Connect Callback Received ]] → Running

On the AWS IoT Core MQTT Broker side, you can validate that the USP agent is successfully connected, subscribed and publishing data, by looking at the AWS IoT Logs in Amazon CloudWatch, and searching for the following events:

CONNECT:

{
  "timestamp": "xxx",
  "logLevel": "INFO",
  "traceId": "xxx",
  "accountId": "xxxx",
  "status": "Success",
  "eventType": "Connect",
  "protocol": "MQTT",
  "clientId": "uspTestAgent",
  "principalId": "xxxx",
  "sourceIp": "xxxx",
  "sourcePort": 26259
}

PUBLISH:

{
"timestamp": "xxx",
"logLevel": "INFO",
"traceId": "xxx",
"accountId": "xxx",
"status":"Success",
"eventType":"Publish-In",
"protocol": "MQTT",
"topicName": "/usp/controller",
"clientId": "uspTestAgent",
"principalId": "xxx",
"sourceIp": "54.240.197.234",
"sourcePort": 26259
}

SUBSCRIBE:

{
"timestamp": "xxx",
"logLevel": "INFO",
"traceId": "xxx",
"accountId": "xxx",
"status": "Success",
"eventType": "Subscribe",
"protocol": "MQTT",
"topicName": "/usp/endpoint",
"clientId": "uspTestAgent",
"principalId": "xxx",
"sourceIp": "xxx",
"sourcePort": 26259
}

You can verify that the USP agent has successfully subscribed to the topic configured in your modified shared.sh(/usp/endpoint), by looking out for the following log statements in the agent logs:

Subscribe: Sending subscribe to /usp/endpoint 0 0
LogCallback: MQTT Debug: Client uspTestAgent sending SUBSCRIBE (Mid: 1, Topic: /usp/endpoint, QoS: 0, Options: 0x00)

Because of the USP subscription to changes in the Device. data model parameters, you should see MQTT publishing happening from the USP agent, for instantiated data model parameters changing frequently, such as Device.DeviceInfo.UpTime or Device.Time.CurrentLocalTime Examples of successful publishing in the agent logs will look like this:

NOTIFY sending at time 2022-12-08T11:27:23Z, to host <YOUR_AWS_IOT_CORE_ENDPOINT> over MQTT
LogCallback: MQTT Debug: Client uspTestAgent sending PUBLISH (d0, q1, r0, m8, '/usp/controller', ... (164 bytes))
Sending NotifyRequest (ValueChange for path=Device.DeviceInfo.UpTime)
....
LogCallback: MQTT Debug: Client uspTestAgent received PUBACK (Mid: 4, RC:0)
PublishCallback: Sent MID 4

If you log into your AWS account, navigate to AWS IoT, select the test MQTT client, and subscribe to the /usp/controller topic, you should see data coming in. Please note that the data is protobuf-encoded, as specified by the USP message encoding specification.

You can, for now, stop the Docker container in which obuspa is running. This will also ensure the termination of the MQTT connection.

Validate MQTT 5 Integration with AWS IoT Core Device Advisor “Qualification Program Test Suite"

After seeing successful MQTT connections, you can use AWS IoT Core Device Advisor, with its “Qualification Program Test Suite”, to validate MQTT 5 functionality and set-up.

Note: To use AWS IoT Core Device Advisor, you will need to confgure your device with the IoT Endpoint of Device Advisor, which is different than the production IoT Core Endpoint. Hence, this should be done at test phases in your development.

This step is extremely valuable during device software development, before on-boarding devices to AWS IoT Core. AWS IoT Core Device Advisor also allows you to create your own test suite, not as part of the qualification program, and configure the appropriate tests for your use case. We will however not cover custom suites in this blog post.

Let’s walk through the steps for setting up AWS IoT Core Device Advisor to run the test suite against the USP agent.

1. Navigate to the AWS Console, to IoT Core Device Advisor. Use one of the regions where Device Advisor is supported, such as Ireland (eu-west-1).

2. Under Test Suites, create a new Qualification Test Suite:

   

3. Choose the “AWS IoT Core qualification test suite” for MQTT 5, and click Next:

   

4. On the Configuration page, verify that all the tests in the suite are present (they should be there by default, so no action), and click Next.

5. Create a new Role for AWS IoT Core Device Advisor to perform the needed MQTT actions, and make sure you configure the correct Action permissions, on the correct resources:

   

6. Review the test suite, and click Create Test Suite.

7. Your newly create test suite should appear in the Test Suites section, and you can select it:

   

8. Open it, and click Run test suite under Actions:

   

9. Select your IoT thing, and copy the IoT Endpoint, as outlined below:

   

10. In your obuspa test script test-mqtt-aws.sh, replace the IoT endpoint with the Device Advisor endpoint, and rebuild your Docker image.

11. You can now click Run test, and immediately after run your USP Agent Docker container.

    

    You will notice that Cloud-side, the test suite is running, and also, that in the obuspa logs, MQTT tests are being executed.

    

    Be patient until the obuspa's MQTT client managed to successfully connect to the IoT endpoint of the Device Advisor, and observe the test suite execution steps, progress, and look at the logs.

    If all the tests in the suites have passed, the validation has been successful. At the end, you can also download the report with the test results.

    

Conclusion

This post shows how the Open Source reference implementation of an USP Agent, obuspa, can be configured to connect, subscribe and publish data, over MQTT 5, to AWS IoT Core, following the release of MQTT 5 support for IoT Core. It also shows how to create and configure a “Qualification Program” MQTT 5 test suite, and run it against the agent during development phases.

For more information around USP, have a look at the specification released by the Broadband Forum, and the Open Source USP Agent reference implementation. If you would like to dive deeper into AWS IoT Core Device Advisor, here is a link to the developer guide. To understand better MQTT 5 support in AWS IoT Core, have a look at the technical documentation.

If you have any feedback about this post, or you would like to see more IoT related content, please reach out to me here, or on Twitter or LinkedIn.