Discussion on: Concurrent Iteration

View post

I have often seen the need for a Promise.map when one has to deal with concurrency of promises. As the native Promise.all doesn't allow for controlling how many promises are executed.

I also find using the async for of loop a pretty neat way of adding concurrency to an array of promises.

Concurrency = 1

const studentDetails = [{ name: 'kj' }, { name: 'ab' }]; // an array of sync values

for (const student of studentDetails) {
  const data = await getPerformanceRecordOfStudent(student);
  // do whatever
}

Concurrency = N

const studentDetails = [{ name: 'kj' }, { name: 'ab' }]; // an array of sync values

const concurrency = 10;
let batch = [];
for (const student of studentDetails) {
  if (executeAll.length < concurrency) {
    batch.push(student);
    continue; // continue the loop
  }
  // runs only a batch of promises i.e. concurrency
  const batchResult = await Promise.all(
    batch.map(getPerformanceRecordOfStudent)
  );

  // do whatever
}

The fancy for-await-of

You can also use for-await-of but remember it works for async iterables, you can use them for an array of promises, but then you aren't really using them to full extent.

Here's how you would create an async iterable.

const fetchNextPage = page => Promise.resolve(page);

async function* asyncGen() {
  let page = 0;
  while (page < 10) yield fetchNextPage(page++);
}

for await (const g of asyncGen()) {
  console.log(g);
}

The best part about this approach is again concurrency, the async generator asyncGen only creates a new promise when asked for, and the for-await-of automatically awaits at the start of for loop, resolves it and puts it as the const g.

Arijit Bhattacharya • Jun 17 '18 • Edited

Hey Kushan

The batching idea is neat. Didn't occur to me that you can play with the degree of concurrency.

Async iterables are pretty awesome. Just discovered them. I guess they officially landed in ES2018.

Thanks for the addendum.

Fun Fact: I was there for your talk at last year's ReactFoo. It proved helpful for my own redux saga journey.