Real-time frontends with Kalix and Laminar

Events are very useful for keeping systems in sync. Here, I'm keeping the backend and the various frontends (web, mobile) synchronised using events and Scala everywhere.
Having everything synchronised at all times is what I call real-time frontends here.
Events are broadcasted and pushed to the frontends through web sockets. Events are replayed with event sourcing in the backend and the frontend.

For everything to be correctly synchronised, it's important to replay events in the same way on both sides. To do this, in this proposal we suggest using the same event definition and the same scala code for replay between the backend and the frontends.
This sharing not only allows things to be synchronised but also avoids having to redefine the same thing at frontend level. Very often we define the data model in the backend and then the same thing in the different frontends. Here, we'd like to avoid this and define it once in Kalix and reuse it in the Laminar frontends (web, mobile).
Reusing exactly the same scala code for the replay is really important. This avoids problems of desynchronisation. If you have different codes (probably also different languages) for the replay between the backend and the frontend, you quickly start having big problems, bugs and data problems. Think of a mobile application that you have to redeploy to correct these problems (mobile deployment isn't easy).
To illustrate my point, I've created a small example. The domain is crowdfunding. The replay logic is very simple and minimal, but it's just an example.
Code can be found here: https://github.com/domschoen/real-time-fes-kalix-laminar-example
The repository contains 3 projects:

• Kalix project
• Laminar project for the web
• Laminar + Capacitor project for mobile app (not yet available)

Note: For more information on how to run the example, see the readme of the different projects.

Description of the example

The application manage crowdfunding projects. A project has:

• Project ID
• Title
• Description
• Goal: The amount of money you want to raise with your crowdfunding project.
• Funded: The amount of money raised so far.

You manipulate the projects with 2 actions:

• Modify project details (title, description, goal)
• Invest: donate money to the project

What is shared ?

We share:

• The definition of events as a protobuf
• The arguments of the REST entry point as a protobuf
• The state (result of event replay) as a protobuf
• The event replay logic as a Scala file

We simply have unix links from Kalix project to Laminar projects.

Selection of shared parts

On the frontend, we are in a different context and we cannot share the entire Kalix protobuf directory or the entire file containing the replay. We have to isolate the parts that interest us and restructure Kalix a little by moving what we need into separate files.
This denature a little the Kalix project but may be, in future version of the project, we could come up with a better solution where we share things as they are and find a way in the frontend to ignore what we don't need.

Keep everything in sync

I've identified 2 solutions:

1. When Kalix emit an event, it is send to Kafka. Each frontend register with Kafka and replays the event received.
2. A frontend emit a command to Kalix. If Kalix digests the command successfully, it sends the same events created in Kalix to the frontend server via websocket. The frontend server broadcasts it to all the frontends via websocket.

The first solution is more elegant and work also with commands issued directly to Kalix (not just from the frontends) but I've chosen the second just to avoid Kafka for the moment and come up with an example quickly.

Laminar

The heart of the frontend application is a Laminar EventBus:

• EventBus receives events and snapshots (more on snapshots below) either from Kalix or from event broadcasts from other frontends.
• From the EventBus, we go through the replay, which outputs a Signal containing a list of all the projects. Laminar is perfect for this, as you can see from this code:

  val $projects: Signal[List[ProjectData]] = 
    eventBus.events.foldLeft(
      initial = List.empty[ProjectData])
      ((acc, ev) => {
        ev match {
          case evt: ProjectState =>
            replayEvent(evt.projectId, evt, true, acc, snapshotReplay)
          case evt: ProjectDetailsChanged =>
            replayEvent(evt.projectId, evt, false, acc, ProjectReplay.projectDetailsChanged)
          case evt: MoneyInvested =>
            replayEvent(evt.projectId, evt, false, acc, ProjectReplay.moneyInvested)
          case _ =>
            println("Bus no managing event " + ev)
            acc
        }
      }
    )

List of all projects in a Laminar Signal

The structure of the projects in the list managed in the signal is twofold:

• The state
• The list of events

The state is always present, but the list of events is optional.
When we fetch data from kalix, we have two use cases which has been chosen for performance reason:

1. When we display the list of all projects (AppProjectsPage), we retrieve a list of states (= snapshots in the event replay). Why do we do this? Because it would be too expensive to retrieve all the events from all the projects and replay all these events. In this case, each project structure only contains the state as a current snapshot, and not the part relating to events.
2. When we inspect one project (in the Project Dashboard page), we retrieve all events of the project and replay them. Here, the project structure contains the state (result of the replay) and the events.

You might ask yourself why it would be interesting to have the list of events in the first place? The answer is this: We can display project events for the user and this represents the history of the project (audit trail).

This is valid for data extracted from the backend, but what happens when an event is received from another frontend? We update the state using the replay logic and add the event to the list of events in the project structure.

Conclusion

Laminar really helps to stream events across frontends and inside frontends and makes it easy to synchronise all frontends.
This solution should save you development time and make synchronisation between the backend and frontends more reliable.
There are a number of areas for improvement:

• How to manage lots of data which don't fit in the frontends ?
• In a larger application, with several entities, you need to separate your events.
• How do you manage events that are not received? In a web application, the user can always refresh the page to get fresh data, but that is not really a solution, and mobile application may retain the state for longer.

Feel free to send me your comments at: dschoenenberger@mac.com