DEV Community

Cover image for 5 examples of reading, writing and pipelining streams in NodeJS
Onelinerhub
Onelinerhub

Posted on

5 examples of reading, writing and pipelining streams in NodeJS

1. Piping input to output stream (file examples)

const fs = require('fs');
let input = fs.createReadStream('/var/www/examples/test.txt');
let output = fs.createWriteStream('/tmp/out.txt');
input.pipe(output);
Enter fullscreen mode Exit fullscreen mode
  • require('fs') - library to work with file system,
  • fs.createReadStream( - create stream to read data from (file in our case),
  • fs.createWriteStream - open file and create writing stream from it,
  • /var/www/examples/test.txt - path to file to stream read,
  • /tmp/out.txt - path to file to write stream to,
  • .pipe( - pipe input stream to output (in our case - read from input file and write to output).

Open original or edit on Github.

2. Preferred way to pipeline streams

const fs = require('fs');
const { pipeline } = require('stream');

let input = fs.createReadStream('/var/www/examples/test.txt');
let output = fs.createWriteStream('/tmp/out.txt');

pipeline(input, output, (err) => console.log(err));
Enter fullscreen mode Exit fullscreen mode
  • require('fs') - library to work with file system,
  • fs.createReadStream( - create stream to read data from (file in our case),
  • fs.createWriteStream - open file and create writing stream from it,
  • /var/www/examples/test.txt - path to file to stream read,
  • /tmp/out.txt - path to file to write stream to,
  • pipeline - pipelines given streams (left to right) and properly destroys all objects after processing,
  • (err) => console.log(err) - last argument of pipeline() is always error callback.

Open original or edit on Github.

3. How to read input stream to Buffer

const fs = require('fs');
let input = fs.createReadStream('/var/www/examples/test.txt');

input.on('data', buf => {
  console.log(buf);
});
Enter fullscreen mode Exit fullscreen mode
  • require('fs') - library to work with file system,
  • fs.createReadStream( - create stream to read data from (file in our case),
  • /var/www/examples/test.txt - path to file to stream read,
  • .on('data' - handle date reading from stream,
  • buf - buffer gets data chunk read from stream.

Open original or edit on Github.

4. How to read input stream to string

const fs = require('fs');
let input = fs.createReadStream('/var/www/examples/test.txt');

const chunks = [];
input.on('data', buf => chunks.push(buf));
input.on('end', () => console.log( Buffer.concat(chunks).toString() ))
Enter fullscreen mode Exit fullscreen mode
  • require('fs') - library to work with file system,
  • fs.createReadStream( - create stream to read data from (file in our case),
  • /var/www/examples/test.txt - path to file to stream read,
  • .on('data' - handle date reading from stream,
  • chunks.push(buf) - push each chunk into array,
  • Buffer.concat(chunks) - join all chunks into single buffer,
  • .toString() - convert Buffer to string,
  • input.on('end' - fire when stream reading was finished.

Open original or edit on Github.

5. How to transform streams using Transform interface

const fs = require('fs');
const { Transform } = require("stream");

let input = fs.createReadStream('/var/www/examples/test.txt');

const my_transform = new Transform({
  transform(chunk, encoding, callback) {
    callback(null, 'TRANSFORMED: ' + chunk.toString());
  },
});

my_transform.on('data', buf => console.log(buf.toString()));

input.pipe(my_transform);
Enter fullscreen mode Exit fullscreen mode
  • require('fs') - library to work with file system,
  • fs.createReadStream( - create stream to read data from (file in our case),
  • new Transform( - create new stream that transforms data,
  • transform(chunk, encoding, callback) - chunk transformation function,
  • 'TRANSFORMED: ' + chunk.toString() - return transformed chunk to callback (we add TRANSFORMED: text to chunk as transformation example),
  • .on('data' - handle date reading from stream,
  • .pipe( - pipe object to the given stream (read from file and pipe to transforming stream).

Open original or edit on Github.

Top comments (0)