Node.js 101 - Streams

Whats are Streams?

Streams are used to process (read and write) data piece by piece (chunks) without completing the whole read and write operation and at the therefore without keeping all the data in memory.

Youtube or Netflix are good example of streams. You don't have to wait for video to fully load. The process is done piece by piece (chucks). So you can then start watching even if the entire file is not yet download

In Node.js there are Readable Streams and Writable Streams. Readable Streams can for example be a file read or a http request data.

Writable Streams is the opposite of Readable Streams so for example a http responses or file to send

Here an example of a large data file read

const fs = require('fs')
const server = require('http').createServer()

server.on('request', () => {
    // No need to load the entire file to memory
    // fs.readFile('data.txt', (err, data) => {
    //    if (err) console.log(err)
    //    res.end(data);
    // })

    // Create a Readable Streams
    const readable = fs.createReadStream('data.txt')

    // Pipe the Stream chunk to a writable Stream
    readable.pipe(res);
})

The readable.pipe() method attaches a Writable stream to the readable, causing it to switch automatically into flowing mode and push all of its data to the attached Writable. The flow of data will be automatically managed so that the destination Writable stream is not overwhelmed by a faster Readable stream.

Conclusion

That's it for today. Tomorrow the journey continue. Stay tune!

Follow me on Twitter: Follow @justericchapman