Beginner Topic: File Upload With Multer In Nodejs

I generally don't love writing about topics already covered extensively on the internet. Still, I decided to write this article due to a question one of my mentees asked. He wanted to know how to programmatically direct uploads to different destinations based on the file type using multer. "Instead of just answering him, why not just write a blog post about it?" I thought. So here goes nothing! If you're already familiar with Nodejs and typescript, you can jump to the second to last section for the answer

What is Multer?

For the benefit of those not familiar with it, multer is a Nodejs library for handling file upload. It is one of, if not the most famous library in this category.
How to use multer.
Without further Ado, I'll stop taking it and show you the code.

Because we don't do js, let's set up a simple typescript project.

    mkdir fileupload && cd fileupload && npm init -y && npm install --save-dev typescript @types/node ts-node nodemon && npx tsc --init

The block of code above creates a new directory called "fileupload" in your current directory, navigates to the new fileupload directory, creates a new package.json file with default options, installs some typescript dependencies, and initializes a typescript project.

Let's create a simple HelloWorld program to test our typescript setup.

At this point, you should still be in your file upload folder. Create an index.ts file in that folder and add the code block below.

    console.log('Hello world'); // if you hate semicolons, deal with it ;)

In your package.json, add this key-value pair to your scripts.

    "start:dev": "nodemon index.ts"

Your package.json file should now look like this:

{
  "name": "fileupload",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start:dev": "nodemon index.ts"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@types/node": "^20.8.2",
    "nodemon": "^3.0.1",
    "ts-node": "^10.9.1",
    "typescript": "^5.2.2"
  }
}

Back in your terminal, type npm run start:dev. If everything works fine, you should see "Hello world" printed on the terminal.

Okay, that was a lot to get a typescript project running, but now that all of that is out of the way, let's set up an HTTP server with expressjs

Set Up A Simple HTTP Server In ExpressJS
Install Express: In your terminal, enter these commands.

npm install express cors --save && npm install --save-dev  @types/express @types/cors

The commands above install expressjs and cors and expressjs's typescript definitions. The cors dependency here controls which origin(websites) can talk to our HTTP server.

Create an HTTP Server: In the index.ts file, enter the following lines of code

import express from 'express';
import cors from 'cors';

// create an express app
const app = express();

// enable cors for all HTTP verbs and origins
app.use(cors())

// create a default endpoint that returns a file uploads response
app.get('/', (req, res) => {
    return res.json({ message: 'welcome to file upload'});
})

// a catch-all middleware for unknown endpoints
app.use('*', (req, res) => {
    return res.status(404).json({ message: 'resource not found'})
})

app.listen(3000);

Uploading Single File Uploads with Multer

Of course, first, install multer. In index.ts, add the following lines of code.

 const uploader = multer({ dest: 'tmp/'});

// create an endpoint capable of handling a single file upload at a time
app. post('/single', uploader.single('myupload'), (req, res) => {
    console.log(req.file);
    return res.status(201).json({ message: 'file uploaded successfully'});
});

// create a catch-all middleware that handles unhandled errors.
app.use((err: Error, req: express.Request, res: express.Response, next: express.NextFunction) => {
    if(err instanceof multer.MulterError) {
        console.log('Multer error occurred: ', err.stack);

        return res.status(400).json({ message: 'file upload error occurred'});
    }

    console.log('Unexpected error occurred: ', err.stack);

    return res.status(500).json({ error: true, message: 'Something went wrong'});

});

The code snippet above first adds a route for single file uploads(/single). We use multer to create a middleware that automatically handles upload. The middleware writes the uploaded file into a folder called tmp in the root directory of your project based on the configuration on the first line of the snippet.

Note: Multer renames the file uploaded. For example, if you uploaded a file named myprofilepic.png on the server, the resulting file name would be a random string such as '97ec5eeb74caf50a093d76568c52b87d'. Multer adds the metadata of the uploaded document to the request.file, which contains the following sample metadata.

{
  fieldname: 'myupload',
  originalname: 'package.json',
  encoding: '7bit',
  mimetype: 'application/octet-stream',
  destination: 'tmp/',
  filename: '97ec5eeb74caf50a093d76568c52b87d',
  path: 'tmp/97ec5eeb74caf50a093d76568c52b87d',
  size: 594
}

You can read the original name of the uploaded file from the metadata and rename the upload if you wish.

The following middleware in the code snippet catches all unhandled errors and returns an appropriate response.

Uploading Multiple Files Uploads With Multer

Creating a route for multiple uploads is as simple as making a route for a single upload. Replace uploader.single('myupload') with uploader.array('uploads'). To access the metadata of all uploaded files, read from req.files; rather than req.file.

// create an endpoint capable of handling multiple file upload
app.post('/multi', uploader.array('myuploads'), (req, res) => {
    console.log(req.files);

    return res.status(201).json({ message: 'files  uploaded successfully'});
});

How Do I Upload To Different Folders Depending On File Type With Multer
The question above was the original question that led to the writing of this blog post. It reveals that my mentee still has some knowledge gap about the node standard library, especially the fs standard library.

Already, we know that multer helps us get any file from our client to a specific folder by default, the tmp folder in our case. The next problem we must solve is getting the uploaded file out of the tmp folder to where we want it to be.; This is where the fs library comes into play.

Assuming we would like to move all files to different folders based on their extension, say all JSON files to a directory called json-files and all txt files to a directory called text-files. First, we read the uploaded files' original name from the request object and then act based on the name's extension.

...
const fileNameParts = req?.file?.originalname?.split?.('.');
// if the file has an extension, get the extension; else, make the extension an empty string
const fileExt = fileNameParts?.pop?.() || '';

switch(fileExt.toLowerCase()) {
    case 'json':
        // Move file to json
        break;
    case  'txt':
        // move file to text
        break;
    default:
        // move to others
}

Now, with the help of the fs module, we move the files to their permanent locations.

// create the text-files folder if it doesn't exist
  if (!fs.existsSync("./text-files")) {
        fs.mkdirSync("./text-files");
      }

// Write the uploaded file into the text-files folder
   fs.writeFileSync(
      `./text-files/${req.file!.filename}`,
      fs.readFileSync(`./tmp/${req.file!.filename}`)
    );

//Repeat the same code for JSON files and others

In the snippet above, we first check if a folder called "text-files" exists in the current director. If it doesn't exist, we create it and then write the uploaded file into this folder.

Here's what your final single upload route code would look like:

// create an endpoint capable of handling a single file upload at a time
app.post("/single", uploader.single("myupload"), (req, res) => {
  console.log(req.file);
  const fileNameParts = req?.file?.originalname?.split?.(".");
  // if the file has an extension, get the extension; else, make the extension an empty string
  const fileExt = fileNameParts?.pop?.() || "";

  switch (fileExt.toLowerCase()) {
    case "json":
      // move file to json
      if (!fs.existsSync("./json-files")) {
        fs.mkdirSync("./json-files");
      }

      fs.writeFileSync(
        `./json-files/${req.file!.filename}`,
        fs.readFileSync(`./tmp/${req.file!.filename}`)
      );
      break;
    case "txt":
      // move file to text
      if (!fs.existsSync("./text-files")) {
        fs.mkdirSync("./text-files");
      }
      fs.writeFileSync(
        `./text-files/${req.file!.filename}`,
        fs.readFileSync(`./tmp/${req.file!.filename}`)
      );
      break;
    default:
      // move to others
      if (!fs.existsSync("./other-files")) {
        fs.mkdirSync("./other-files");
      }
      fs.writeFileSync(
        `./text-files/${req.file!.filename}`,
        fs.readFileSync(`./tmp/${req.file!.filename}`)
      );
  }
  return res.status(201).json({ message: "file uploaded successfully" });
});

For multiple uploads, wrap the code above in a loop.

...
for (const file of req.files!) {
    //Copy and paste the code in the single upload  here
}

And that is all it takes.

Final Notes

I mentioned initially that this is a beginner tutorial, so some apparent violations of popular best practices are intentionally left in the code snippets. I did this not to veer away from the primary purpose of the blog post.

You will likely deploy your code on an ephemera server in the cloud, so never upload your file to your server's filesystem. Uploading files like this is only valid for quick file content processing, but 99% of the time, you need a cloud storage bucket like AWS S3 or, in some cases, a separate FTP server. Also, remember to always clean up any temporary folders after processing their content lest you run out of disk space quickly.

Edit: 12/10/2023

If you'd like to play with the code samples in this article, check out this repo