loading...

Don't Make This Async/Await Oopsie!

mebble profile image Neil Syiemlieh Updated on ・3 min read

Suppose we need to perform some I/O on items of an array, like fetching the owners of cats from the cats' IDs using some API.

const catIDs = [132, 345, 243, 121, 423];

Let's say we decide to use our newly acquired async/await skills to do the job. Async/await gets rid of the need for callbacks (in most cases), making asynchronous code look similar to synchronous code. But if we forget that we're still just dealing with asynchronous code, we might make a mistake that defeats the entire purpose of having concurrency.

We might be tempted to do something like this:

async function fetchOwners(catIDs) {
    const owners = [];
    for (const id of catIDs) {
        const cat = await fetchCat(id);
        const owner = await fetchOwner(cat.ownerID);
        owners.push(owner);
    }
    return owners;
}

What Was Our Oopsie? 🤷‍♂️

Disclaimer: If you know what the oopsie was, then you probably already know what you're doing. You might know a use case for this behaviour, so I guess it's a little unfair to call it an "oopsie". This article is just to familiarise people with this async/await behaviour.

We run the code and all seems to be working alright. But there's a glaring problem in how we've used async/await. The problem is that we used await within a for-loop. This problem is actually indicative of a common code smell, which is the ".push to an output array within a for-loop" method of performing an array transformation, instead of using .map (we'll get to this later).

Because of the await within the for-loop, the synchronous-looking fetchOwners function is performing a fetch for the cats sequentially (kind of), instead of in parallel. The code awaits for the owner of one cat before moving on to the next for-loop iteration to fetch the owner of the next cat. Fetching the owner of one cat isn't dependent on any other cat, but we're acting like it is. So we're completely missing out on the ability to fetch the owners in parallel (oopsie! 🤷‍♂️).

Note: I mentioned "kind of" sequentially, because those sequential for-loop iterations are interleaved with other procedures (through the Event Loop), since the for-loop iterations await within an async function.

What We Should Be Doing 😎

We shouldn't await within a for-loop. In fact, this problem would be better off solved without a for-loop even if the code were synchronous. A .map is the appropriate solution, because the problem we're dealing with is an array transformation, from an array of cat IDs to an array of owners.

This is how we'd do it using .map if the code were synchronous.

// catIDs -> owners
const owners = catIDs.map(id => {
    const cat = fetchCatSync(id);
    const owner = fetchOwnerSync(cat.ownerID);
    return owner;
});

Since the code is actually asynchronous, we first need to transform an array of cat IDs to an array of promises (promises to the cats' owners) and then unpack that array of promises using await to get the owners. This code doesn't handle rejected promises for the sake of simplicity.

// catIDs -> ownerPromises -> owners
async function fetchOwners(catIDs) {
    const ownerPromises = catIDs.map(id => {
        return fetchCat(id)
            .then(cat => fetchOwner(cat.ownerID));
    });
    const owners = await Promise.all(ownerPromises);
    return owners;
}

To further flex our async/await skills, we could pass an async callback to the map method and await all intermediate responses (here, fetching a cat to get its owner's ID) within that callback. Remember, an async function returns a promise, so we're still left with an array of promises as the output of .map. This code is equivalent to the previous one, but without the ugly .then.

async function fetchOwners(catIDs) {
    const ownerPromises = catIDs.map(async id => {
        const cat = await fetchCat(id);
        const owner = await fetchOwner(cat.ownerID);
        return owner;
    });
    const owners = await Promise.all(ownerPromises);
    return owners;
}

What is .map actually doing?

.map invokes the callback (in which we make an I/O request) on each cat ID sequentially. But since the callback returns a promise (or is an async function), .map doesn't wait for the response for one cat to arrive before shooting off the request for the next cat.

The requests are shot sequentially, but travel in parallel, and their responses arrive out of order (imagine throwing a bunch of boomerangs with one hand behind your back).

So we're now fetching the cat owners in parallel like we intended to 🙌! Oopsie undone!


This was my very first post. Not just my first on DEV, but my first blog post ever. Hope you liked it.

Posted on by:

mebble profile

Neil Syiemlieh

@mebble

Engineer at Gojek. Functional programming and music theory nerd. Picking up cooking and basketball.

Discussion

markdown guide
 

Calling this an oopsie or an error is a bit unfortunate.

Sometimes you want to make requests serially. This is not a mistake.

Maybe you want to fail or interrupt the whole process based on the result of one of the fetches. You can do that easily in a for loop.

Consider if you had 100 fetches to do - you don't want to fire them all off simultaneously and flood the services you're using. You could take serial chunks of the work in a for loop and fire those off in parallel.

So, good tip with map & Promise.all, but serial execution is not a mistake in and of itself

 

Thank you for this comment. I should have stated that this is only something people should be aware of. I went with a clickbaity title just to get it out there. I'll update the article to include this.

 

I've always had some trouble with async/await, some unexpected stuff always happened when using them with map/filter/reduce.
Found some solutions but this is by far the most instructive. Great examples and great first post!

 

Thank you! It feels great to hear from someone who found the post helpful

 

I was wandering on social media when this article poped up. But had no time to finish this off, therefore this page was opened for about 3 weeks long and now I had the time. I finished it! I don't regret! Very useful article (boomerangs, right? :)). Thank you your effort!

 
 
 

Thanks for the tip. Could you have used generators, instead, to do the job?

 

Async await are based on generators but I haven't used generators themselves as they're not as popular as async await. It would be very interesting to see how generators would be used for async code

 

hey Neil, great article, can you recommend me a good best practice article or books about something like this and to learning the mern stack thank you

 

Thank you! I haven't worked on JS in a while now, so I don't recall any best practice article or books at the moment. But I did learn a lot from Eloquent JavaScript and Andrew Mead's Udemy courses

 

thanks bud, hope that you find your true life purpose as a good and be better human being.. always remember allah, buddy.

 

Oh man. This is some good stuff. I feel like I'm a serial async/await abuser.

 

Great post. Really interesting to read this approach to async/await and getting code running in parallel. This helps solidify the power of an async/await approach to this kind of process.