How to dataflow merge on Observable

In this series I will explore programming techniques for the notebook platform Observable. Today I am looking at how to merge multiple reactive Dataflow streams into a single stream. This article assumes you are familiar with Observable's non-linear reactive program flow already.

A common situation is this: you have a notebook that performs some useful task, and you want to offer a few different ways to start that task. For example, starting something based on configuration obtained from either URL parameters, local storage or a manual UI. It's a little trickier than you would hope because, if each of these methods are in their own cell [1], then we have multiple distinct data flow pathways converging into a single flow.

[1] and they probably should be because of separation-of-concerns

The core difficulty is if there is no data in a source, its dataflow won't tick, so if the business logic depends on all the input sources, the business logic will not tick either. So this pattern described here will joins the business logic to the sources a different way so that if any of them tick, the business logic will tick too.

In ReactiveX terms Observable's dataflow is a combineLatest across cell streams, but we want a merge.

So here is how you can do it with views and Inputs.bind.

Step 1. Create an unresolved view for the business logic to depend on

We can quickly create an unresolved view with Inputs.input(). We will fan-in sources to this view.

If the business logic extracts parameters from the config variable, then it will not run until it is set to a non-undefined value (note grey bar on the left of the cell after notebook startup).

Our example business logic will flash the notebook in a color set by the config.

2. Setup all the sources to be views

Next we need the sources to be emitting their config to a view's value.

Often, a source is already a view:

But even if it isn't, it is trivial to create a view from a dataflow variable by writing into an Inputs.input()

3. Bind the source views to the fan-in view backwards

Now, if either of the sources fire, the business logic is run. Give it a try on the live notebook on at @tomlarkworthy/merge-dataflow

I hope you find that useful and that you find other useful ways to manipulate dataflow!