Building a Real-time Data App with Dozer, React, and PostgreSQL

In today's data-driven world, having access to real-time data is crucial and gaining a value from the data is more cumbersome. Another challenge developers face when working with real-time data is efficiently and quickly fetching, processing, and displaying the insights from the data.

Enter Dozer, it simplifies the process of creating low-latency data APIs (gRPC and REST) from any data source easily with OpenAPI specification. Dozer is a real-time data engineering tool built with Rust, which enables seamless integration of data streams(sources), efficient data processing, and lightning-fast response times. This allows developers to focus on building applications without worrying about the complexity of managing real-time data.

To know more about it's capabilities and features you can checkout Dozer repository.

In this blog, we'll walk you through the process of creating a real-time flight data application using Dozer, React.
For this particular blog, we are using PostgreSQL as our data source, but it's important to note that there can be multiple data sources that can be connected to Dozer and subsequently used by your React application. By using Dozer, you can quickly create efficient data APIs that enable seamless integration with your frontend applications.
PostgreSQL is connected to Dozer via PostgreSQL connector and fetches data from multiple Postgres tables that has a flight data and combines them in real time based on the queries and produces fast READ APIs.

On the client side, React app is being used to display the flight data on map coming from Dozer. The dozerjs and dozer-react libraries are used to make it easy to interact with Dozer and access real-time data without having to hardcode API endpoints in the code.
The dozerjs library manages gRPC requests between the frontend and the Dozer app, while the dozer-react library provides React hooks for easy integration with React components.

So let's dive in and learn how to harness the power of Dozer to create amazing real-time data-driven applications! 💪✨

🐘 Setting Up PostgreSQL:

1. Clone the sample repository and navigate to the appropriate directory:

git clone https://github.com/getdozer/dozer-samples.git
cd dozer-samples/usecases/pg-flights

1. Checkout the react branch:

git checkout react

1. In the usecases/pg-flights/ directory, run the following command to start PostgreSQL using Docker Compose:

docker-compose up

This command will create a PostgreSQL container with the necessary configuration and sample flight data as shown below:

To test the connection to the PostgreSQL container, you can use psql or any PostgreSQL client. Here's an example using psql:

psql -h localhost -p 5437 -U postgres -d flights

This command connects to the flights database using the postgres user on the default port 5437. Once connected, you can run SQL queries to explore the sample flight data.

With PostgreSQL set up and running, we can now proceed to build the React application and configure Dozer.

⚛️ Building & Running the React App 🚀:

1. Open a new termincal and navigate to the /usecases/react directory in the cloned dozer-samples repository:

cd dozer-samples/usecases/react

This directory contains a sample React app that displays flight data on a map component.

1. Install the required packages for the app using yarn install:

yarn install

1. In the project directory, run the following command to start the development server:

yarn start

This command runs the app in development mode.

Open http://localhost:3000 to view it in your browser. The page will automatically reload when you make changes, and you may also see any lint errors in the console. On this page, you will the Dozer logo, now navigate to http://localhost:3000/airports to see the map UI for respective ariport and it's flight booking data as follow:

This React app uses the @dozerjs/dozer and @dozerjs/dozer-react libraries to interact with Dozer, enabling seamless integration and real-time data updates.

Since, we have not configured the Dozer with PostgreSQL yet, there is not data on the map.
Let' quickly setup the Dozer and see the magic happening!

🔌 Configuring Dozer and PostgreSQL Connector:

1. Start by installing Dozer. In for this blog, we'll use Homebrew for the installation. To learn about other installation methods, visit the official Dozer installation documentation.

open a new terminal, ensuring you're in the same directory as your React project, and run:

brew tap getdozer/dozer
brew install dozer

1. Once the installation is complete, create a dozer-config.yaml file in your project directory and paste the following configuration into it:

# dozer-config.yaml content

app_name: flight-microservices
connections:
  - name: flights_conn
    config: !Postgres
      user: postgres
      password: postgres
      host: 0.0.0.0
      port: 5437
      database: flights

sql: |
  select f.arrival_airport as airport, a.coordinates as coordinates, COUNT(t.ticket_no) as tickets
  INTO airports_count
  from tickets t
  join ticket_flights tf on t.ticket_no = tf.ticket_no
  join flights f on tf.flight_id = f.flight_id
  join airports a on f.arrival_airport = a.airport_code
  group by f.arrival_airport, a.coordinates;

  select extract(HOUR FROM f.window_start) as start, count(f.window_start) as dep_count
  INTO departures_count
  from TUMBLE(flights, scheduled_departure, '4 HOURS') f
  group by extract(HOUR FROM f.window_start)

sources:
  - name: tickets
    table_name: tickets
    columns:
    connection: !Ref flights_conn

  - name: flights
    table_name: flights
    columns:
    connection: !Ref flights_conn

  - name: ticket_flights
    table_name: ticket_flights
    columns:
    connection: !Ref flights_conn

  - name: airports
    table_name: airports
    columns:
    connection: !Ref flights_conn

  - name: airports_flights_schema
    table_name: airports
    columns:
    schema: flights_schema
    connection: !Ref flights_conn

endpoints:

  - name: tickets
    path: /bookings/tickets
    table_name: tickets
    index:
      primary_key:
        - ticket_no

  - name: flights
    path: /bookings/flights
    table_name: flights
    index:
      primary_key:
        - flight_id

  - name: airports
    path: /bookings/airports
    table_name: airports
    index:
      primary_key:
        - airport_code


  - name: ticket_flights
    path: /bookings/ticket_flights
    table_name: ticket_flights
    index:
      primary_key:
        - ticket_no
        - flight_id

  - name: airports_count
    path: /airports_count
    table_name: airports_count
    index:
      primary_key:
        - airport
        - coordinates

  - name: departures_count
    path: /departures_count
    table_name: departures_count
    index:
      primary_key:
        - start

This configuration defines the PostgreSQL connector and sets up the API endpoints that our React application will use to fetch data.
For more information about the PostgreSQL connector, refer to documentation.
Dozer supports multiple connectors, allowing you to integrate various data sources. Check out the list of available connectors for more options.

1. With the configuration in place, run Dozer in action by executing the following command:

dozer -c dozer-config.yaml

Now, you should see Dozer logs indicating that the endpoints is ready to be queried by the React app.

You can now access your application at http://localhost:3000/airports to see the real-time flight data populating on the map component as shown below:

Additionally, You can also query the API using gRPC and REST endpoints to test the data. In this case, you can use the REST endpoint using curl or Postman http://localhost:8080/airports/

Additionally, you can check the OpenAPI specification documentation that Dozer generates for endpoints by requesting http://localhost:<port>/<endpoint_path>/oapi as shown below.

🌟 Conclusion:

Congratulations! You've successfully built a real-time flight data app using Dozer, React, and PostgreSQL. This powerful combination enables you to create interactive and data-driven applications with ease.
Feel free to expand on this example and explore other data sources and connectors to enhance your app's capabilities. Enjoy exploring the application and customizing it further to suit your needs!

Happy coding, Happy Data APIng! 🚀👩‍💻👨‍💻