Unleashing the Power of Dozer Lambda Runtime for Real-time and Event-driven Data Apps

Real-time data processing is essential for modern applications, as it ensures that insights are timely and accurate. The open-source project Dozer now provides a runtime environment that allows developers to execute lambda functions or custom user-defined functions based on specific data conditions.
This new functionality allows for more dynamic and event-driven applications, as Lambda functions can be triggered based on specific data change conditions. Just imagine triggering a Twilio notification based on a SQL statement (data change condition)!

In this blog post, we will discuss how you can use Dozer to respond to real-time data changes by triggering functions based on data conditions.
In this example, we will create a React app that generates a real-time graph based on room temperature data (room temperature monitoring). We will use the pydozer_log module to react to data change conditions and trigger a Twilio message if the room temperature goes above 22.9°C.

Dozer Lambda Runtime: A Glimpse

The Dozer Lambda runtime is a major enhancement to the Dozer project, enabling developers to write Lambda functions in JavaScript and Python that react to Dozer events.

To help you better understand the Dozer Lambda runtime, we will examine the following state diagram:

DCC: Data Change Condition

• Data Source: Your application's data source, such as a database, sends data to the Dozer pipeline.
• Dozer Pipeline: The data is processed through the Dozer pipeline, which can apply various transformations and aggregations based on the configuration you provide.
• Lambda Runtime: The Dozer Lambda runtime monitors the pipeline for specific data change conditions that you define. If a data change condition is met, the corresponding Lambda function is triggered.
• Action/Trigger: The triggered Lambda function performs the desired action, such as sending a Twilio notification or updating a database record.

The reponse of trigger can be used to write back top dozer, for example using gRPC service.

JavaScript and Python Bindings for Reading Dozer Logs

Dozer now provides bindings for both JavaScript and Python to read logs, making it easier for developers to work with the languages they are most comfortable with. These bindings are essential components of the Dozer Lambda runtime, as they enable developers to create Lambda functions that respond to
events.

React App Example: Room Temperature Monitoring

In this example, we will create a React app that generates a real-time graph based on room temperature data. We will use the pydozer_log module to react to data change conditions and trigger a Twilio message if the room temperature goes above 22.9°C.

Here are the steps to run the app:

1. Clone the Dozer Samples repository and install dependencies:

git clone https://github.com/getdozer/dozer-samples.git
cd dozer-samples/usecases/lambda/
pip install -r requirements.txt

1. Remove any existing Dozer configuration and start the Dozer:

rm -rf ./.dozer && dozer -c config/dozer-config.yaml

1. Run the Python scripts to read data from the pipeline:

Note: make sure to replace the Twilio configuration in reader.py, with your account id and auth token. To learn more about Twilio SMS setup checkout this getting started guide.

python reader.py
python main.py

1. Navigate to the React app folder, install the necessary packages, and start the development server:

cd react
yarn install && yarn start

Once the app is running, you should see a real-time graph of the room temperature data. If the temperature exceeds 22.9°C, a Twilio message will be triggered, notifying the relevant parties of the situation.

Conclusion 🎯

By using the Dozer Lambda runtime, you can create event-driven applications that automatically respond to changes in your data, making your applications more efficient, scalable, and responsive to user needs. By leveraging the Python and JavaScript libraries provided by Dozer, developers can easily build applications that respond to real-time data changes, making it an invaluable resource for modern applications.

Happy coding, Happy Data APIng! 🚀👩‍💻👨‍💻