How YOU can learn Node.js I/O, files and paths

Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris

If you are you completely new to Node.js or maybe you've just spun up an Express app in Node.js, but barely know anything else about Node - Then this first part in a series is for YOU.

In this part we will look at:

• Working with file paths, it's important when working with files and directories that we understand how to work with paths. There are so many things that can go wrong in terms of locating your files and parsing expressions but Node.js does a really good job of keeping you on the straight and narrow thanks to built-in variables and great core libraries
• Working with Files and Directories, almost everything in Node.js comes in an async, and sync flavor. It's important to understand why we should go with one over the other, but also how they differ in how you invoke them.
• Demo, finally we will build some demos demonstrating these functionalities

 The file system

The file system is an important part of many applications. This means working with files, directories but also dealing with different access levels and paths.

Working with files is in Node.js a synchronous or an asynchronous process. Node.js is single-threaded which means if we need to carry things out in parallel we need an approach that supports it. That approach is the callback pattern.

 References

• Node.js docs - file system This is the official docs page for the file system
• Overview of the fs module Good overview that shows what methods are available on the fs module
• Reading files Shows all you need to know about reading files
• Writing files Docs page showing to how to writ files
• Working with folders Shows how to work with folders
• File stats If you need specific information on a file or directory like creation date, size etc, this is the page to learn more.
• Paths Working with paths can be tricky but this module makes that really easy.
• Create a Node.js app on Azure Want to know how to take your Node.js app to the Cloud?
• Log on to Azure programmatically using Node.js This teaches you how to programmatically connect to your Azure resources using Node.js

 Paths

A file path represents where a directory or file is located in your file system. It can look like this:

/path/to/file.txt

The path looks different depending on whether we are dealing with Linux based or Windows-based operating system. On Windows the same path might look like this instead:

C:\path\to\file.txt

We need to take this into account when developing our application.

For this we have the built-in module path that we can use like so:

const path = require("path");

The module path an help us with the following operations:

• Information, it can extract information from our path on things such as parent directory, filename and file extension
• Join, we can get help joining two paths so we don't have to worry about which OS our code is run on
• Absolute path, we can get help calculating an absolute path
• Normalization, we can get help calculating the relative distance between two paths.

 Demo - file paths

Pre-steps

1. Create a directory for your app
2. Navigate to your directory cd <name of dir>
3. Create app file, Now create a JavaScript file that will contain your code, the suggestion is app.js
4. Create file we can open, In the same directory create a file info.txt and give it some sample data if you want

Information

Add the following code to your created app file.

const path = require("path");

const filePath = '/path/to/file.txt';
console.log(`Base name ${path.basename(filePath)}`);
console.log(`Dir name ${path.dirname(filePath)}`);
console.log(`Extension name ${path.extname(filePath)}`);

Now run this code with the following command:

node <name of your app file>.js

This should produce the following output

Base name file.txt
Dir name /path/to
Extension name .txt

Above we can see how the methods basename(), dirname() and extname() helps us inspect our path to give us different pieces of information.

Join paths

Here we will look into different ways of joining paths.

Add the following code to your existing application file:

const join = '/path';
const joinArg = '/to/my/file.txt';

console.log(`Joined ${path.join(join, joinArg)}`);

console.log(`Concat ${path.join(join, 'user','files','file.txt')}`)

Above we are joining the paths contained in variables join and joinArg but we are also in our last example testing out concatenating using nothing but directory names and file names:

console.log(`Concat ${path.join(join, 'user','files','file.txt')}`)

Now run this using

node <name of your app file>.js

This should give the following output:

Joined /path/to/my/file.txt
Concat /path/user/files/file.txt

The takeaway here is that we can concatenate different paths using the join() method. However, because we don't know if our app will be run on a Linux of Windows host machine it's preferred that we construct paths using nothing but directory and file names like so:

console.log(`Concat ${path.join(join, 'user','files','file.txt')}`)

Absolute path

Add the following to our application file:

console.log(`Abs path ${path.resolve(joinArg)}`);
console.log(`Abs path ${path.resolve("info.txt")}`);

Now run this using

node <name of your app file>.js

This should give the following output:

Abs path /to/my/file.txt
Abs path <this is specific to your system>/info.txt

Note, how we in our second example is using the resolve() method on info.txt a file that exist in the same directory as we run our code:

console.log(`Abs path ${path.resolve("info.txt")}`);

The above will attempt to resolve the absolute path for the file.

Normalize paths

Sometimes we have characters like ./ or ../ in our path. The method normalize() helps us calculate the resulting path. Add the below code to our application file:

console.log(`Normalize ${path.normalize('/path/to/file/../')}`)

Now run this using

node <name of your app file>.js

This should give the following output:

Normalize /path/to/

 Working with Files and Directories

There are many things you can do when interacting with the file system like:

• Read/write files & directories
• Read stats on a file
• Working with permissions

You interact with the file system using the built in module fs. To use it import it, like so:

const fs = require('fs')

I/O operations

Here is a selection of operations you can carry out on files/directories that exist on the fs module.

• readFile(), reads the file content asynchronously
• appendFile(), adds data to file if it exist, if not then file is created first
• copyFile(), copies the file
• readdir(), reads the content of a directory
• mkdir(), creates a new directory,
• rename(), renames a file or folder,
• stat(), returns the stats of the file like when it was created, how big it is in Bytes and other info,
• access(), check if file exists and if it can be accessed

All the above methods exist as synchronous versions as well. All you need to do is to append the Sync at the end, for example readFileSync().

Async/Sync

All operations come in synchronous and asynchronous form. Node.js is single-threaded. The consequence of running synchronous operations are therefore that we are blocking anything else from happening. This results in much less throughput than if your app was written in an asynchronous way.

Synchronous operation

In a synchronous operation, you are effectively stopping anything else from happening, this might make your program less responsive. A synchronous file operation should have sync as part of the operation name, like so:

const fileContent = fs.readFileSync('/path/to/file/file.txt', 'utf8');
console.log(fileContent);

Asynchronous operation

An Asynchronous operation is non-blocking. The way Node.js deals with asynchronous operations is by using a callback model. What essentially happens is that Node.js doesn't wait for the operation to finish. What you can do is to provide a callback, a function, that will be invoked once the operation has finished. This gives rise to something called a callback pattern.

Below follows an example of opening a file:

const fs = require('fs');

fs.open('/path/to/file/file.txt', 'r', (err, fileContent) => {
  if (err) throw err;
  fs.close(fd, (err) => {
    if (err) throw err;
  });
});

Above we see how we provide a function as our third argument. The function in itself takes an error err as the first argument. The second argument is usually data as a result of the operation, in this case, the file content.

 Demo - files and directories

In this exercise, we will learn how to work with the module fs to do things such as

• Read/Write files, we will learn how to do so in an asynchronous and synchronous way
• List stats, we will learn how to list stat information on a file
• Open directory, here we will learn how to open up a directory and list its file content

Pre-steps

1. Create a directory for your app
2. Navigate to your directory cd <name of dir>
3. Create app file, Now create a JavaScript file that will contain your code, a suggestion is app.js
4. Sample file, In the same directory create a file info.txt and give it some sample data if you want
5. Create a sub directory with content, In the same directory create a folder sub and within create the files a.txt, b.txt and c.txt Now your directory structure should look like this:

app.js
info.txt
sub -|
---| a.txt
---| b.txt
---| c.txt

 Read/Write files

First, start by giving your app.js file the following content on the top:

const fs = require('fs');
const path = require('path');

Now we will work primarily with the module fs, but we will need the module path for helping us construct a path later in the exercise.

Now, add the following content to app.js:

try {
  const fileContent = fs.readFileSync('info.txt', {
    encoding: 'utf8'
  });
  console.log(`Sync Content: ${fileContent}`);
} catch (exception) {
  console.error(`Sync Err: ${exception.message}`);
}

console.log('After sync call');

Above we are using the synchronous version of opening a file. We can see that through the use of a method ending in sync.

Follow this up by adding the asynchronous version, like so:

fs.readFile('info.txt', (err, data) => {
  if (err) {
    console.log(`Async Error: ${err.message}`);
  } else {
    console.log(`Async Content: ${data}`);
  }
})

console.log('After async call');

Now run this code with the following command:

node <name of your app file>.js

This should produce the following output

Sync Content: info
After sync call
After async call
Async Content: info

Note above how the text After sync call is printed right after it lists the file content from our synchronous call. Additionally note how text After async call is printed before Async Content: info. This means anything asynchronous happens last. This is an important realization about asynchronous operations, they may be non-blocking but they don't complete right away. So if the order is important you should be looking at constructs such Promises and Async/await.

 List stats

For various reasons, you may want to list detailed information on a specific file/directory. For that we have stat() method. This also comes in an asynchronous/synchronous version.

To use it, add the following code:

fs.stat('info.txt', (err, stats) => {
  if (err) {
    console.error(`Err ${err.message} `);
  } else {
    const { size, mode, mtime } = stats;

    console.log(`Size ${size}`);
    console.log(`Mode ${mode}`);
    console.log(`MTime ${mtime}`);
    console.log(`Is directory ${stats.isDirectory()}`);
    console.log(`Is file ${stats.isFile()}`);
  }
})

Now run this code with the following command:

node <name of your app file>.js

This should produce the following output

Size 4
Mode 33188
MTime Mon Mar 16 2020 19:04:31 GMT+0100 (Central European Standard Time)
Is directory false
Is file true

Results above may vary depending on what content you have in your file info.txt and when it was created.

 Open a directory

Lastly, we will open up a directory using the method readdir(). This will produce an array of files/directories contained within the specified directory:

fs.readdir(path.join(__dirname, 'sub'), (err, files) => {
  if (err) {
    console.error(`Err: ${err.message}`)
  } else {
    files.forEach(file => {
      console.log(`Open dir, File ${file}`);
    })
  }
})

Above we are constructing a directory path using the method join() from the path module, like so:

path.join(__dirname, 'sub')

__dirname is a built-in variable and simply means the executing directory. The method call means we will look into a directory sub relative to where we are executing the code.

Now run this code with the following command:

node <name of your app file>.js

This should produce the following output

Open dir, File a.txt
Open dir, File b.txt
Open dir, File c.txt

Summary

In summary, we have covered the following areas:

• Paths, we've looked at how we can work with paths using the built-in path module
• Files & Directories, we've learned how we can use the fs module to create, update, remove, move etc files & directories.

There is lots more to learn in this area and I highly recommend looking at the reference section of this article to learn more.