Andrea Simone Costa

Posted on Sep 8, 2019 • Edited on Apr 13, 2021

JavaScript Iterators and Generators: Asynchronous Iterators

#javascript #intermediates #asynchronous #iterators

Hi, nice to meet you 😄!
You can find this article in my personal blog here.

Top comments (11)

Chris Achard • Sep 8 '19

Oh wow, this is next level! Also TIL about for-await-of; neat. Do you have some examples of where this can be really helpful in real world code? I'm wondering when I should think to maybe reach for this instead of cobbling something else together 😃 Thanks!

Andrea Simone Costa • Sep 8 '19

Thanks, I'm glad you liked the article 😃

Generally, the async iteration is well suited for all those circumstances where consumers should have control over the flow of data.
Next time I'll show some more concrete examples thanks to async generators, but for now a good starting point to satisfy your needs surely are Node.js Readable streams. WHATWG Streams are async iterables too.

Pasquale Mangialavori • Sep 9 '19

I was thinking pagination could be a use case candidate for async iteration.

const page = await pagination[Symbol.asyncIterator]().next();

Andrea Simone Costa • Sep 9 '19

Yes of course. I've already briefly mentioned it into the 'The consumer pressure problem' section 😃

Pasquale Mangialavori • Sep 9 '19 • Edited

I think we can also write it down in this succinct way. But correct me if I'm wrong

const pagination = {
  async *[Symbol.asyncIterator]() {
    let i = 0;
    while (i < 10) {
      await Promise.resolve(true);
      i++;
      yield i;
    }
  }
};

Andrea Simone Costa • Sep 9 '19

Yes, but async gens are the argument of the next article eheheheh

Pasquale Mangialavori • Sep 9 '19

SPOILER ALERT then :D eheh

RENARD Laurent • Sep 19 '19

A while back I also wrote a library to manipulate async stream with async iterables. You might find it interesting. It was in the context of this article.
It is a very nice serie you wrote here. However I think you should stress more on the fact that closing iterable is a leaky abstraction(by Reginald “Raganwald” Braithwaite). You did a bit in the second article but I think it deserves more attention. It is usually better to use built in consumers (such for await statement) to avoid problems

Andrea Simone Costa • Sep 19 '19

It could seem a stupid answer but I strongly believe in well-commented programs, so what is not immediately understandable should be highlighted with a proper comment/explanation.

I am with you, I think that custom implementations of interfaces like these should be kept to a minimum but...there is always an exception.
Like here: github.com/nodejs/node/blob/master...
One of the Node.js streams maintainer told me that an async gen would be too slow and hard to implement and mantain, so they've chosen to manually implement the async iteration interfaces.

RENARD Laurent • Sep 20 '19 • Edited

I don't think we are talking about the same thing. I think it is totally fine to implement these interfaces yourself: usually when you implement a producer in a "low level" way you know what you are doing (cf Nodejs core maintainers).

I am talking about the code which consumes it, depending on the way you consume it, you might create leaks.

Consider the following producer (from my article I linked above):

const wait = delay => new Promise(resolve => {
    setTimeout(() => resolve(), delay);
});
const counterGen = async function * (limit = 10, delay = 100) {
 let iter = 1;
 try {
  while (true) {
   if (iter > limit) {
    break;
   }
   await wait(delay);
   yield iter;
   iter++;
  }
 } catch (e) {
  console.log('oops something is wrong');
  throw e;
 } finally {
  console.log('I have been released !!!');
 }
};

You could have written it by implementing the interfaces, it does not matter.

Just note it is doing some cleaning ("I have been released !!!"), it could be release file handle or whatever.

Now let's say you want to consume it and sum the 3 first values.

You can do it in a naive way

const sum = async iterator => {
    let i = 0;
    let sum = 0;
    while (i < 3) {
        const next = await iterator.next();
        if (next.done) {
            break;
        }
        sum += next.value;
        i++;
    }

    // VERY IMPORTANT IF YOU DO NOT WANT TO CREATE A LEAK
    // iterator.return();

    return sum;
};

sum(counterGen())
    .then(console.log);

And this code creates a leak if you don't pay attention and do not explicitly call return (you will not see the release message)

Whereas if you decide to go for "native construct" like for await statement you are safe

const sum = async iterator => {
    let i = 0;
    let sum = 0;
    for await (const v of iterator){
        if(i>=3){
            break;
        }
        sum+= v;
        i++
    }
    return sum;
};

sum(counterGen())
    .then(console.log);

I think tutorials and articles on iterators (and async iterators) do not stress enough that eventual issue.

This was my point :)

Andrea Simone Costa • Sep 21 '19

Uh I've totally misunderstood your previous message!

Yes you are right, the for-of and the for-await-of should be always preferred 🙂

View full discussion (11 comments)

DEV Community

JavaScript Iterators and Generators: Asynchronous Iterators

Top comments (11)

Read next

Python beats Javascript, Next.js Leap & the AI Coding Wars

From Concept to Code: Building a Simple Endless Runner Game

The No-Fluff Guide to OpenGraph Images That Actually Work 🎯

Testing Troubles with Jest & ESM and how to fix it