Suppose we need to perform some I/O on items of an array, like fetching the owners of cats from the cats' IDs using some API.
const catIDs = [132, 345, 243, 121, 423];
Let's say we decide to use our newly acquired async/await skills to do the job. Async/await gets rid of the need for callbacks (in most cases), making asynchronous code look similar to synchronous code. But if we forget that we're still just dealing with asynchronous code, we might make a mistake that defeats the entire purpose of having concurrency.
We might be tempted to do something like this:
async function fetchOwners(catIDs) {
const owners = [];
for (const id of catIDs) {
const cat = await fetchCat(id);
const owner = await fetchOwner(cat.ownerID);
owners.push(owner);
}
return owners;
}
What Was Our Oopsie? π€·ββοΈ
Disclaimer: If you know what the oopsie was, then you probably already know what you're doing. You might know a use case for this behaviour, so I guess it's a little unfair to call it an "oopsie". This article is just to familiarise people with this async/await behaviour.
We run the code and all seems to be working alright. But there's a glaring problem in how we've used async/await. The problem is that we used await
within a for-loop. This problem is actually indicative of a common code smell, which is the ".push
to an output array within a for-loop" method of performing an array transformation, instead of using .map
(we'll get to this later).
Because of the await
within the for-loop, the synchronous-looking fetchOwners
function is performing a fetch for the cats sequentially (kind of), instead of in parallel. The code await
s for the owner of one cat before moving on to the next for-loop iteration to fetch the owner of the next cat. Fetching the owner of one cat isn't dependent on any other cat, but we're acting like it is. So we're completely missing out on the ability to fetch the owners in parallel (oopsie! π€·ββοΈ).
Note: I mentioned "kind of" sequentially, because those sequential for-loop iterations are interleaved with other procedures (through the Event Loop), since the for-loop iterations await
within an async
function.
What We Should Be Doing π
We shouldn't await
within a for-loop. In fact, this problem would be better off solved without a for-loop even if the code were synchronous. A .map
is the appropriate solution, because the problem we're dealing with is an array transformation, from an array of cat IDs to an array of owners.
This is how we'd do it using .map
if the code were synchronous.
// catIDs -> owners
const owners = catIDs.map(id => {
const cat = fetchCatSync(id);
const owner = fetchOwnerSync(cat.ownerID);
return owner;
});
Since the code is actually asynchronous, we first need to transform an array of cat IDs to an array of promises (promises to the cats' owners) and then unpack that array of promises using await
to get the owners. This code doesn't handle rejected promises for the sake of simplicity.
// catIDs -> ownerPromises -> owners
async function fetchOwners(catIDs) {
const ownerPromises = catIDs.map(id => {
return fetchCat(id)
.then(cat => fetchOwner(cat.ownerID));
});
const owners = await Promise.all(ownerPromises);
return owners;
}
To further flex our async/await skills, we could pass an async
callback to the map method and await
all intermediate responses (here, fetching a cat to get its owner's ID) within that callback. Remember, an async
function returns a promise, so we're still left with an array of promises as the output of .map
. This code is equivalent to the previous one, but without the ugly .then
.
async function fetchOwners(catIDs) {
const ownerPromises = catIDs.map(async id => {
const cat = await fetchCat(id);
const owner = await fetchOwner(cat.ownerID);
return owner;
});
const owners = await Promise.all(ownerPromises);
return owners;
}
What is .map
actually doing?
.map
invokes the callback (in which we make an I/O request) on each cat ID sequentially. But since the callback returns a promise (or is an async function), .map
doesn't wait for the response for one cat to arrive before shooting off the request for the next cat.
The requests are shot sequentially, but travel in parallel, and their responses arrive out of order (imagine throwing a bunch of boomerangs with one hand behind your back).
So we're now fetching the cat owners in parallel like we intended to π! Oopsie undone!
This was my very first post. Not just my first on DEV, but my first blog post ever. Hope you liked it.
Top comments (15)
Calling this an oopsie or an error is a bit unfortunate.
Sometimes you want to make requests serially. This is not a mistake.
Maybe you want to fail or interrupt the whole process based on the result of one of the fetches. You can do that easily in a for loop.
Consider if you had 100 fetches to do - you don't want to fire them all off simultaneously and flood the services you're using. You could take serial chunks of the work in a for loop and fire those off in parallel.
So, good tip with map & Promise.all, but serial execution is not a mistake in and of itself
Thank you for this comment. I should have stated that this is only something people should be aware of. I went with a clickbaity title just to get it out there. I'll update the article to include this.
I was wandering on social media when this article poped up. But had no time to finish this off, therefore this page was opened for about 3 weeks long and now I had the time. I finished it! I don't regret! Very useful article (boomerangs, right? :)). Thank you your effort!
Thank you for your support!
I've always had some trouble with async/await, some unexpected stuff always happened when using them with map/filter/reduce.
Found some solutions but this is by far the most instructive. Great examples and great first post!
Thank you! It feels great to hear from someone who found the post helpful
hey Neil, great article, can you recommend me a good best practice article or books about something like this and to learning the mern stack thank you
Thank you! I haven't worked on JS in a while now, so I don't recall any best practice article or books at the moment. But I did learn a lot from Eloquent JavaScript and Andrew Mead's Udemy courses
thanks bud, hope that you find your true life purpose as a good and be better human being.. always remember allah, buddy.
Thanks for the tip. Could you have used generators, instead, to do the job?
Async await are based on generators but I haven't used generators themselves as they're not as popular as async await. It would be very interesting to see how generators would be used for async code
Good tip! Congratulations on your first post!
Thank you!
Oh man. This is some good stuff. I feel like I'm a serial async/await abuser.
Great post. Really interesting to read this approach to async/await and getting code running in parallel. This helps solidify the power of an async/await approach to this kind of process.