DEV Community

Cover image for SmartHome - your home data in real time!
marco-bertelli
marco-bertelli

Posted on

SmartHome - your home data in real time!

Overview of My Submission

hi, today we are going to see a project that collect your home temperature and humidity in real time!

base app overview
before talking in the detail about the app is important to understand the architecture.

architecture

as we can see there are many different parts:

  • Backend: write using node js and TS, handle the redis integration using redis OM, MongoDb integration, Aws Iot real time MQTT connection and Tensorflow js model utilization. Use redis for reading the real time sensor messagges over the MQTT channel "machines" and try to find a sensor associated with it, if found insert a datalog of the received data in both MongoDb and redis for rapid real time data into the frontend graph. the alarm logic is associated to the datalogs, if when a datalog is created trigger an alarm rule an active alarm is created and will be closed only when an alarm below the trashold arrive, all the alarms are stored and not deleted. Tensorflow is used to read a very simple regression model and use it for the next hour predictions. the most cool features of the backend are:
    • Aws Iot integration
    • Tensorflow Js integration to parse an existing ML model and use it into JS
    • Redis integration to cache data and use redis JSON and redis search to get real time data
    • Redis model generator that take an existing Mongoose model and parse it into a redis one with no code logic!
    • the alarms rules that are stored into the database!, an example is:
{
    "_id" : ObjectId("6300e639830e983ff546f7af"),
    "sensorId" : ObjectId("62fce2bc830e983ff546f775"),
    "rule" : {
        ">" : [
            {
                "var" : "temp"
            },
            25.0
        ]
    },
    "name" : "Bedroom High Temperature",
    "type" : "rule"
}
Enter fullscreen mode Exit fullscreen mode

as you can see the rule is a sort of logic!, in fact this is possible by the JSON logic module.

  • Frontend: retrieve the data from the backend API to make a graph based on a custom range time, in the graph you can choose the sensor who data you want to see, there is also the average temperature and humidity cards in the bottom of the dashboard that show you the Average in the selected range. the alrams are real time and retrived with only 1 minute cache to don't lose the new alarms, as explained before the alrams are created only when a value trigger an alarm trashold. there is also a home map to see the sensors disposition and have an idea of the space, the "SUN" card show you the actual values and not the average. the last card show you the prediction fo the next hour. All the components support the multi theme feature to personalize the user experience.

theme

frontend also have mobile support but only Horizontal not vertical!

  • Sensors: send real time temperature and humidity data every 5 minutes, integrated with Aws-Iot. this sensor are made using Mongoose Os, a really cool it framework that help to increase speed using simil JS code. for the code see the Firmware repo

sensor

  • Redis: used to store the cache and the partial datalogs data to speed up the responses time and keep retrive graphic data simple.

  • MongoDb: used to store all other data like sensor and alarms

Previews

Live Preview: Site
after enter into the site select a range and enjoy it!

now that we know the base of the app is time to talk about the redis integration, it help a lot for the graph real time data, because as we say before the data are partial stored into redis to take i for graphs, before redis with only mongo the response time was 300 ms, after 120 ms what a great thing!. i also use redis cache to store the most important and slow enpoint (like the tensorflow one) and it was very helpfull, here we go too see a list of endpoints and used cache:

  • datalogs/:
    • /predict: predict the next hour data, using redis cache to keep it for 15 minutes
    • /ambient/now: actual data
    • /:sensorId/:interval: get datalogs of a sensor use redis JSON and redis Search!
    • /ambient/humidity/:interval: Average humidity
    • /ambient/temperature/:interval: Average temp
  • alarms/:
    • /active: get active alrams, use redis cache to store it for 1 minutes
    • /finished: get finished alarms: use redis cache to keep it for 3 minutes
  • sensors/: get the sensor list, use redis cache to keep it for 30 minutes.

so a full list of frontend feature you can try in the live demo!

  • real time graph data from my house sensors on 1 or more sensor in an interval
  • 2D map to take an idea of sensor position
  • active alarms up to now!
  • average temp and humidity in the selected range
  • card with actuals info
  • next hours prevision using ML!

futures feature

the only thing that i don't have time to make was a complete ML model, in fact this simple one use a regression but a complete one is a timeseries prevision model!

Submission Category: MEAN/MERN Mavericks

video overview of the project Video

Language Used

Node js / TS for the backend
Angular for the frontend
Mongoose Os for the firmware code

Link to Code

Live Preview: Site
after enter into the site select a range and enjoy it!
BackeEnd repo:

GitHub logo marco-bertelli / iotSensorBackend

Backend with node js and TS for redis hackathon

SmartHome

this is a project that get real time temperature and humidity data from esp32 with DHT22 AM2302 temperature sensor. after this the data is stored in both redis and MongoDb and processed from the frontend (see the other repo for more frontend info)

Embed your YouTube video

Embed your YouTube video

Embed your YouTube video

Embed your YouTube video

Overview video

Here's a short video that explains the project and how it uses Redis:

Embed your YouTube video

How it works

before start is important to understand the app architecture:

Embed your YouTube video

as we can see there are many different parts:

  • Backend: write using node js and TS, handle the redis integration using redis OM, MongoDb integration, Aws Iot real time MQTT connection and Tensorflow js model utilization. Use redis for reading the real time sensor messagges over the MQTT channel "machines" and try to find a sensor associated with it if found insert a datalog of the received data in both MongoDb and redis for rapid real time data into…


FrontEnd repo:

GitHub logo marco-bertelli / iotSensorFrontend

Angular ngx-admin frontend for redis challenge

SmartHome

this is a project that get real time temperature and humidity data from esp32 with DHT22 AM2302 temperature sensor. after this the data is stored in both redis and MongoDb and processed from the frontend (see the other repo for more frontend info)

Embed your YouTube video

Embed your YouTube video

Embed your YouTube video

Embed your YouTube video

Overview video

Here's a short video that explains the project and how it uses Redis:

Embed your YouTube video

How it works

before start is important to understand the app architecture:

Embed your YouTube video

as we can see there are many different parts:

  • Backend: write using node js and TS, handle the redis integration using redis OM, MongoDb integration, Aws Iot real time MQTT connection and Tensorflow js model utilization. Use redis for reading the real time sensor messagges over the MQTT channel "machines" and try to find a sensor associated with it if found insert a datalog of the received data in both MongoDb and redis for rapid real time data into…


Firmware repo:

GitHub logo marco-bertelli / iotSensorFirmware

the firmware code for the esp32 sensors

SmartHome

this is a project that get real time temperature and humidity data from esp32 with DHT22 AM2302 temperature sensor. after this the data is stored in both redis and MongoDb and processed from the frontend (see the other repo for more frontend info)

Embed your YouTube video

Embed your YouTube video

Embed your YouTube video

Embed your YouTube video

Overview video

Here's a short video that explains the project and how it uses Redis:

Embed your YouTube video

How it works

before start is important to understand the app architecture:

Embed your YouTube video

as we can see there are many different parts:

  • Backend: write using node js and TS, handle the redis integration using redis OM, MongoDb integration, Aws Iot real time MQTT connection and Tensorflow js model utilization. Use redis for reading the real time sensor messagges over the MQTT channel "machines" and try to find a sensor associated with it if found insert a datalog of the received data in both MongoDb and redis for rapid real time data into…


Tensorflow Model: if you want more info of tensorflow visit the model here: colab

closure

if you like the project or have any questions leave a like and a comment!
all the shares will be appreciated!


Top comments (14)

Collapse
 
tqbit profile image
tq-bit

This project looks super cool. FE has a sleek design & I like the mongooseMapper function you implemented.

May I ask what tool you used to draw the app's architecture? your image looks like it comes straight from a pitch deck.

Collapse
 
marcobertelli profile image
marco-bertelli

thank you a lot!, the tool name of the architecture is cloudcraft: cloudcraft.co/

Collapse
 
matteg profile image
MattG

Amazing, i think that will be a usefull upgrade for my smart home

Collapse
 
decker67 profile image
decker

Seems very complicated to me. What's the use of Tensorflow? Do you know NodeRed?

Collapse
 
marcobertelli profile image
marco-bertelli

yeah i know NodeRed, but i am not so pratical to use it!
Instead Tensorflow is a Google framweork to Machine Learning Neural Network development, i use it to import an existing machine learning model in Node Js

Collapse
 
marcobertelli profile image
marco-bertelli

thank you!

Collapse
 
manuela_durizzi profile image
Manuela Durizzi

Really cool

Collapse
 
marcobertelli profile image
marco-bertelli

thanks!

Collapse
 
leonardo_marinai_2031a5eb profile image
Leonardo Marinai

Beautiful!!!

Collapse
 
marcobertelli profile image
marco-bertelli

thanks!

Collapse
 
gozzo00604789 profile image
Gozzo

I want this in my home 🐸🐸🐸

Collapse
 
marcobertelli profile image
marco-bertelli

sure!

Collapse
 
teresamerigo profile image
Teresa Merigo

I love It!

Collapse
 
marcobertelli profile image
marco-bertelli

thanks!