If you're working with Node.js streams, you've probably come across the
highWaterMark parameter. But what is it, and why is it important? In this post, we'll explore the basics of
highWaterMark and how it affects your stream pipelines.
highWaterMark is a parameter that sets a limit on the amount of data that can be stored in the internal stream buffer. When the buffer reaches this limit, Node.js stops asking for more data until the buffer is emptied. This helps prevent your program from consuming too much memory and crashing.
However, it's important to remember that
highWaterMark is a threshold, not a strict limit. You can keep manipulating data even when the buffer size exceeds
highWaterMark. But be aware that the data will still be buffered in memory, which could cause your memory to fill up and you to lose all the benefits of Node.js streams.
If you're using the
writable.write() method, it returns a
boolean value that indicates whether it's safe to continue writing data to the stream. Specifically, if the internal stream buffer size is bigger than
highWaterMark, it'll return
false. In this case, you need to wait for the
'drain' event to be emitted before you can resume writing data to the stream. If the buffer size is smaller than
highWaterMark, then it'll return
true, and you can continue writing data without waiting.
highWaterMarkis a parameter that sets a limit on the amount of data that can be stored in the internal stream buffer.
This limit is a threshold, not a strict limit, so you can keep manipulating data beyond it if you need to.
However, exceeding the threshold could cause your memory to fill up, so it's important to be mindful of your memory usage.
If you're using the
writable.write()method, check the boolean value it returns before keep writing data to the stream.
highWaterMark and how it works, you can write more efficient and robust code that takes advantage of Node.js streams. Happy streaming! 🌊