Use the well known RxJS operators to manipulate arrays or iterables in a blazing fast way using the new tiny library rxjs-transducer

Most JavaScript developers by now have learned to use Array’s builtin methods like filter, map, reduce, some, and every to manipulate arrays of data in a functional programming style. This style of programming has the advantage of being easier to reason about than imperative loop style programming. It does have a number of drawbacks, however:

• The operations only work for arrays, not iterables.

• There is a fairly limited set of operations. Notable omissions are operations like take, skip, first, last, single.

• Bad performance: When chaining multiple operations, they will each create an intermediate array and thus iterate the array as many times as there are operators. E.g:

Will create 3 intermediate arrays and then iterate the last array to reduce it to a string, a total of 4 array iterations. Not exactly efficient. Of course this is not an issue when the source contains 10 or 100 elements. But if there are millions of elements, it could be a problem.

Using RxJS operators instead

When thinking of RxJS, you usually think asynchronous stream processing, but RxJS will in fact process streams synchronously when possible. This means that we can use RxJS to create a synchronous stream of values from an array or other iterable using the from function:

This snippet will only iterate the array once, transforming and filtering values as it goes along. It is however a bit clunky to have to use the from, pipe, and subscribe keywords, so I have written a small transducer library that reduces the snippet above to:

The cool thing about this transducer is that it supports iterables such as infinite sequences so you can do stuff like:

Furthermore it is written in TypeScript, so it will give you full TypeScript support in your IDE:

Performance

So, how does it perform you say?

Let us make a test using console.time where we map, filter and reduce an array of 10,000,000 random numbers:

310ms vs 47ms! So in this case the rxjs-transducer is more than 6 times as fast as standard array chaining. Your mileage may vary, but in almost all cases it will be quite a lot faster than array chaining.

How to get it

npm install rxjs-transducer (< 1KB GZipped)

Check out the code on GitHub:

npm install rxjs-transducer --save

import { transducer } from 'rxjs-transducer';
import { map, filter, reduce, skip, take } from 'rxjs/operators';
const source = ['a', 'ab', 'abc', 'abcd', 'abcde'];
// Works as standard array

Playground

I have created a StackBlitz playground for you to try it out in your browser:

Let me know what you think.

https://twitter.com/rasmusvhansen