Efficiently read files in a directory with Node.js opendir

Originally published on my blog.

Recently I had to scan the contents of a very large directory in order to do some operations on each file.

I wanted this operation to be as fast as possible, so I knew that if I used the standard fsPromises.readdir or fs.readdirSync which read every file in the directory in one pass, I would have to wait till the entire directoy was read before operating on each file.

Instead, I wanted to instead operate on the file the moment it was found.

To solve this, I reached for opendir (added v12.12.0) which will iterate over each found file, as it is found:

import { opendirSync } from "fs";

const dir = opendirSync("./files");
for await (const entry of dir) {
    console.log("Found file:", entry.name);
}

fsPromises.opendir/openddirSync return an instance of Dir which is an iterable which returns a Dirent (directory entry) for every file in the directory.

This is more efficient because it returns each file as it is found, rather than having to wait till all files are collected.

Just a quick Node.js tip for ya 🪄

Follow me on Dev.to, Twitter and Github for more web dev and startup related content