Recursively Moving Files in a Directory with Node.js and TypeScript

Introduction

Recently, my Marketing team was facing an issue with uploading the files on the server. The folder structure of their files was somewhat like the one below.

Folder A/
 Folder B
    Folder 1
       file 1
       file 2
       file n
    Folder 2
       file 1
       file 2
       file n

They had to select each file one at a time for some processing which was time-consuming as they had to select a folder within a folder, Multiple file selection was still possible but not with folders, they consulted with me for some suggestions and instantly I thought that making the files move to one folder will solve their issue.

Now moving the files from folder to folder recursively was not an easy task as there were around 1k files so I thought making this work programmatically will solve our issue and below is a summary of how I did this.

I used Node.js and Typescript for this as they have all the components which we need and are also very easy to understand.

Prerequisites

To get started, you'll need to have Node.js and npm (Node Package Manager) installed on your machine. If you haven't installed them yet, you can download and install them from the official Node.js website (https://nodejs.org/).

For this article, I am using Node v20.2.0

Step by Step guide

Once you have Node.js and npm installed, follow the step-by-step guide.

Create and Initialize a new Node.js project

Open a terminal, create a new directory where you want to store your project files, and run the following commands:

# Folder name can be anything but do make it easily readable.
mkdir recursive-files && cd recursive-files

# This command will create a new default package.json file in your project directory.
npm init -y

Install TypeScript and ts-node

TypeScript is a typed superset of Javascript that compiles to plain Javascript. ts-node is a tool that allows you to run TypSctipt code directly, without the compile step. Install them as development dependencies using the following command :

 npm i typescript ts-node -D

Create a tsconfig.json file:

The tsconfig.json file is used to specify the root files and the compiler options to compile the project.

For creating a tsconfig.json file use the following command:

# Will create the tsconfig.sjon file wit default options which can be modified as needed
npx tsc --init

Time to write the code:

Create a new .ts file in your project directory. You can use the following command to create a new file.

touch index.ts

Importing necessary modules:

We will be using fs and path modules from Node.js. The fs modules allow us to interact with the file system, whereas the path module allows us to work with file and directory paths.

import fs from 'fs'
import path from 'path'

Defining the moveFiles function:

The below function takes two arguments: sourceDir (the directory from which we want to move the files) and targetDir (the directory where you want to move the files). The function returns a Promise that resolves when all the files have been moved or rejected if there's an error.

function moveFiles(sourceDir:string, targetDir:string): Promise<void> {
    return new Promise((resolve), (reject) => {
        // Code for moving the files
    });
}

Reading the source directory & Iterating over the files:

We will be using fs.readdir function to read the contents of the source directory. This function takes a callback that is called with two arguments: an error (if there is one) and an array of filenames.

// Reading the files from source directory
   fs.readdir(sourceDir, (err, files) => {
      if (err) {
          reject(err)
          return;
      }
    console.log('List of files', files);
);

We're using Array.prototype.forEach to iterate over the array of filenames. For each filename, we're doing the following:

• Constructing the old and new paths: We're using path.join to construct the full paths to the file in the source and target directories.

• Checking if the file is a directory or a regular file: We're using fs.lstatSync to get the stats of the file, and then checking if it's a directory or a regular file with Stats.prototype.isDirectory and Stats.prototype.isFile.

• If the file is a directory, we're calling moveFiles recursively to move the files to the subdirectory.

• If the file is a regular file, we're using fs.rename it to move the file to the target directory. This function takes a callback that is called with an error (if there is one).
  

// Continue from the above code
     console.log('List of Files', files);

      files.forEach((file) => {
        const oldPath = path.join(sourceDir, file);
        const newPath = path.join(targetDir, file);
        const stat = fs.lstatSync(oldPath);

        if (stat.isDirectory()) {
          moveFiles(oldPath, targetDir)
            .then(() => {
              if (files.indexOf(file) === files.length - 1) {
                resolve();
              }
            })
            .catch((err) => reject(err));
        } else if (stat.isFile()) {
          fs.rename(oldPath, newPath, (err) => {
            if (err) {
              reject(err);
              return;
            }

            if (files.indexOf(file) === files.length - 1) {
              resolve();
            }
          });
        }
      });

Handling errors:

If there is an error at any point like reading the directory, getting the stats of a file, or moving a file, Promise will be rejected with the error.

Please note we are using fs.lstatSync it as a synchronous method. For the asynchronous version, we can use fs.lstat to avoid blocking the event loop.

Calling the moveFiles function:

moveFiles('SourceDir', 'TargetDir')
.then(() => console.log('All files moved successfully'))
.catch((err) => console.log(err))

Conclusion

The above script will recursively read all directories and files under the specified directory and will successfully move each file into one target folder as specified.

Please note the above code will not work for the files with the same name in different directories, the files will be overwritten and we need to add some logic to check if a file with the same name already exists in the target directory.

I hope you have learned something new as I did. If so, kindly like it or share it with others so they can also see it.

References

You can find the gist of the sample code here