Using SiriDB for IoT applications

The Internet of Things has long been one of the most popular technology trends and for good reason. It offers many possibilities and advantages. This includes monitoring data, such as the temperature or air quality in a home. But this data must be stored somewhere for later use. Preferably by using a very efficient solution, because IoT devices often do not have a lot of computing power. An extremely suitable solution that can be used for this is SiriDB. A super fast and very robust time series database.

In this post, we'll show you how to easily build a weather station that uses SiriDB as the database for all its measurements. For this weather station we use a Raspberry Pi (3 Model B) in combination with a Sense HAT.

Prerequisites

Hardware

• Raspberry Pi
• Sense HAT

Software

You will need the latest version of Raspberry Pi OS, which already includes the following software packages:

• Python 3
• Sense HAT for Python 3

If for any reason you need to install the Sense HAT package manually, execute the following command on your Raspberry Pi:

$ sudo apt-get install sense-hat

Set up

Now that our Raspberry Pi is ready to go, we need to follow a few steps to install SiriDB.

First, install the required SiriDB dependencies:

$ sudo apt install libcleri-dev libpcre2-dev libuv1-dev libyajl-dev uuid-dev

Download the source code of SiriDB:

$ wget https://github.com/SiriDB/siridb-server/archive/refs/tags/2.0.46.zip

Note: Change the version in the URL to get the latest version of SiriDB.

Now unzip the downloaded ZIP file:

$ unzip 2.0.46.zip

After that, compile the source code:

$ cd siridb-server-2.0.46/Release/
$ make clean
$ make

Then install SiriDB:

$ sudo make install

Finally, install the SiriDB Connector for Python with pip (recommended), which will be used in a later step:

$ pip install siridb-connector

Configuration

Great! SiriDB is now installed. But in order for SiriDB to start automatically on startup, the file below named siridb-server.service needs to be added to the /etc/systemd/system folder:

[Unit]
Description=SiriDB Server
After=network.target

[Service]
Environment="SIRIDB_HTTP_API_PORT=9020"
ExecStart=/usr/bin/siridb-server
StandardOutput=journal
LimitNOFILE=65535
TimeoutStartSec=10
TimeoutStopSec=300

[Install]
WantedBy=multi-user.target

Note: The environment variable SIRIDB_HTTP_API_PORT must be added because if the HTTP API port is not set (or 0), the API service of SiriDB will not start. Checkout the documentation for more information about using environment variables with SiriDB.

Now we need to reload the daemon, to make the system aware of the newly added service:

$ sudo systemctl daemon-reload

Let’s enable our service so that it doesn’t get disabled when the Raspberry Pi restarts:

$ sudo systemctl enable siridb-server.service

And now let's start our service:

$ sudo systemctl start siridb-server.service

The SiriDB server is now running in the background.

The only thing left to do in terms of configuration is to create a database in SiriDB. For this we use the SiriDB HTTP API. SiriDB has a default service account with the username sa and password siri, which we will use. For our tutorial we only need a second precision database. We also select a shard duration of 6 hours for this database because our measurement interval will be only a few seconds. Sometimes you may want to store one value per measurement per hour or even per day. In that case, your database will perform better by using a larger shard duration.

Create the database on the SiriDB server running on port 9020 using cURL with basic authentication:

$ curl --location --request POST 'http://localhost:9020/new-database' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic c2E6c2lyaQ==' \
--header 'Content-Type: text/plain' \
--data-raw '{
    "dbname": "iot",
    "time_precision": "s",
    "buffer_size": 8192,
    "duration_num": "6h",
    "duration_log": "3d"
}'

Note: The value passed to the header parameter Authorization, is the word Basic word followed by a space and a base64-encoded string username:password. See the documentation for more information about authentication in SiriDB.

Collecting data

To collect the data we use Python 3 together with the Sense HAT package.

The Python script below called sense.py contains the minimal code necessary to collect some key measurements using the Raspberry Pi Sense HAT and then write them to SiriDB:

import asyncio
import time

from sense_hat import SenseHat
from siridb.connector import SiriDBClient


async def collect_data(siri):

    # Start connecting to SiriDB.
    await siri.connect()

    # Initialize the Raspberry Pi Sense HAT.
    sense = SenseHat()

    try:
        # Make sure the script runs indefinitely.
        while True:

            # Get the current temperature and round it to 2 decimal places.
            temp = round(sense.get_temperature(), 2)

            # Get the current humidity and round it to 2 decimal places.
            humi = round(sense.get_humidity(), 2)

            # Get the current pressure and round it to 2 decimal places.
            pres = round(sense.get_pressure(), 2)

            # Show the current temperature on the Sense HAT.
            sense.show_message('%s C' % temp)

            # Get the current time in seconds since the Epoch.
            ts = int(time.time())

            # Add the collected data to the SiriDB serie: "temperature".
            await siri.insert({'temperature': [[ts, temp]]})

            # Add the collected data to the SiriDB serie: "humidity".
            await siri.insert({'humidity': [[ts, humi]]})

            # Add the collected data to the SiriDB serie: "pressure".
            await siri.insert({'pressure': [[ts, pres]]})

            # Wait 10 seconds before retrieving new measurements.
            await asyncio.sleep(10)

    finally:
        # Close all SiriDB connections.
        siri.close()

# Initialize the SiriDB client.
siri = SiriDBClient(
    username='iris', # Default username
    password='siri', # Default password
    dbname='iot', # The name of the database we created earlier
    hostlist=[('localhost', 9000)],  # Multiple connections are supported
    keepalive=True 
)

loop = asyncio.get_event_loop()
loop.run_until_complete(collect_data(siri))

Add the script to your Raspberry Pi and start it with:

$ python sense.py

To verify that everything is working properly you should now see a message on your Sense Hat every 10 seconds:

Note: Our camera couldn't capture the message from the LED matrix well, but it does show the correct temperature. (Assuming you believe us of course 🙂)

Visualization

That was all! Some key measurements are now automatically collected every 10 seconds and written to SiriDB.

However, it would be even more valuable if some insights could also be obtained from this data. This can be done using a visualization application such as Grafana, for which SiriDB has made a ready-to-use plugin: grafana-siridb-http-datasource.

How Grafana can be used in combination with SiriDB is explained here. But below you can already see what the measured data would look like in this application:

Conclusion

In this blog post we have shown step by step how easy and effective it is to deploy SiriDB for IoT applications. Now you have enough information to get started with SiriDB yourself!

If you want to learn more about SiriDB, you can visit the official documentation page at https://docs.siridb.net.