Welcome back to NodeJS Event loop series. In this post, I’m going to talk about how I/O is handled in NodeJS in detail. And I hope to dig deep into the implementation of the event loop and how I/O work in conjunction with other async operations as well. If you miss any of the previous articles in this series, I highly recommend you to go through them which I’ve listed in the following Roadmap section. I have described many other concepts in NodeJS event loop in the previous 3 posts.
- Event Loop and the Big Picture
- Timers, Immediates and Next Ticks
- Promises, Next-Ticks, and Immediates
- Handling I/O (This article)
- Event Loop Best Practices
- New changes to timers and microtasks in Node v11
We are talking a lot about asynchronous I/O when it comes to NodeJS. As we discussed in the first article of this series, I/O is never meant to be synchronized.
In all OS implementations, they provide event notification interfaces for asynchronous I/O (epoll in linux/kqueue in macOS/event ports in solaris/IOCP in Windows etc.). NodeJS leverages these platform level event notification systems in order to provide non-blocking, asynchronous I/O.
As we saw, NodeJS is a collection of utilities which eventually are aggregated into the high performant NodeJS framework. These utilities include,
- Libuv — For Event Loop with Asynchronous I/O
- c-ares — For DNS Operations
- other add-ons such as ( http-parser , crypto and zlib )
In this article, we’ll talk about Libuv and how it provides asynchronous I/O to Node. Let’s look at the event loop diagram again.
Let’s recap what we learned so far about the event loop:
- Event loop is started with executing handlers of all expired timers
- Then it will process any pending I/O operations, and will optionally wait for any pending I/O to complete.
- Then it will move on to consume setImmediate callbacks
- Finally, it will process any I/O close handlers.
Now, let’s try to understand how NodeJS performs I/O in its event loop.
What is I/O?
Generally, any work which involves external devices except the CPU is called I/O. The most common abstract I/O types are File Operations and TCP/UDP network operations.
A kind notice!
I recommend you to read the previous articles of this series if you do not have a basic understanding of the event loop. I might omit certain details here for brevity because I’d like to focus more on I/O in this article
I might use some code snippets from libuv itself, and I’ll only use Unix-specific snippets and examples only to make things simpler. Windows-specific code might differ a bit, but there shouldn’t be much difference.
I would assume you can understand a small snippet of C code. No expertise needed, but a basic understanding of the flow would be adequate.
As we saw in the previous NodeJS architecture diagram, libuv resides in a lower layer of the layered architecture. Now let’s look at the relationship between the upper layers of NodeJS and the phases of libuv event loop.
As we saw in diagram 2 (Event loop in a nutshell) previously, there were 4 distinguishable phases of the event loop. But, when it comes to libuv, there are 7 distinguishable phases. They are,
- Timers — Expired timer and interval callbacks scheduled by setTimeout and setInterval will be invoked.
- Pending I/O callbacks — Pending Callbacks of any completed/errored I/O operation to be executed here.
- Idle handlers — Perform some libuv internal stuff.
- Prepare Handlers — Perform some prep-work before polling for I/O.
- I/O Poll — Optionally wait for any I/O to complete.
- Check handlers — Perform some post-mortem work after polling for I/O. Usually, callbacks scheduled by setImmediate will be invoked here.
- Close handlers — Execute close handlers of any closed I/O operations (closed socket connection etc.)
Now, if you remember the first article in this series, you may be wondering…
- What are Check handlers? It was also not there in the event loop diagram.
- What is I/O Polling? Why do we block for I/O after executing any completed I/O callbacks? Shouldn’t Node be non-blocking?
Let’s answer the above questions.
When NodeJS is initialized, it sets all setImmediate callbacks to be registered as Check handlers in libuv. This essentially means that any callback you set using setImmediate will eventually land in Libuv check handles queue which is guaranteed to be executed after I/O operations during its event loop.
Now, you may be wondering what I/O polling is. Although I merged I/O callbacks queue and I/O polling into a single phase in the event loop diagram (diagram1), I/O Polling happens after consuming the completed/errored I/O callbacks.
But, the most important fact in I/O Polling is, it’s optional. I/O poling will or will not happen due to certain situations. To understand this thoroughly, let’s have a look at how this is implemented in libuv.
Ouch! It may seem a bit eye-twisting for those who are not familiar with C. But let’s try to get a glimpse of it without worrying too much about it. The above code is a section of
uv_run method of which resides in core.c file of libuv source. But most importantly, this is the Heart of the NodeJS event loop.
If you have a look at diagram 3 again, the above code will make more sense. Let’s try to read the code line by line now.
uv__loop_alive— Check whether there are any referenced handlers to be invoked, or any active operations pending
uv__update_time— This will send a system call to get the current time and update the loop time (This is used to identify expired timers).
uv__run_timers— Run all expired timers
uv__run_pending— Run all completed/errored I/O callbacks
uv__io_poll— Poll for I/O
uv__run_check— Run all check handlers (setImmediate callbacks will run here)
uv__run_closing_handles— Run all close handlers
At first, event loop checks whether the event loop is alive, this is checked by invoking
uv__loop_alive function. This function is really simple.
uv__loop_alive function simply returns a boolean value. This value is true if:
- There are active handles to be invoked,
- There are active requests (active operations) pending
- There are any closing handlers to be invoked
Event loop will keep spinning as long as
uv__loop_alive function returns true.
After running callbacks of all expired timers,
uv__run_pending function will be invoked. This function will go through the completed I/O operations stored in pending_queue in libuv event. If the pending_queue is empty, this function will return 0. Otherwise, all callbacks in pending_queue will be executed, and the function will return 1.
Now let’s look at I/O Polling which is performed by invoking
uv__io_poll function in libuv.
You should see that
uv__io_poll function accepts a second timeout parameter which is calculated by
uv__io_poll uses the timeout to determine how long it should block for I/O. If the timeout value is zero, I/O polling will be skipped and the event loop with move onto check handlers (setImmediate) phase. What determines the value of the timeout is an interesting part. Based on the above code of
uv_run, we can deduce the follows:
- If the event loop runs on
UV_RUN_DEFAULTmode, timeout is calculated using
- If the event loop runs on
uv_run_pendingreturns 0 (i.e,
pending_queueis empty), timeout is calculated using
- Otherwise, timeout is 0.
Let’s not try to worry about different modes of the event loop such as
UV_RUN_ONCEat this point. But if you are really interested in knowing what they are, check them out here.
Let’s now have a peek at
uv_backend_timeout method to understand how timeout is determined.
- If the loop’s
stop_flagis set which determines the loop is about to exit, timeout is 0.
- If there are no active handles or active operations pending, there’s no point of waiting, therefore the timeout is 0.
- If there are pending idle handles to be executed, waiting for I/O should not be done. Therefore, the timeout is 0.
- If there are completed I/O handlers in
pending_queue, waiting for I/O should not be done. Therefore the timeout is 0.
- If there are any close handlers pending to be executed, should not wait for I/O. Therefore, the timeout is 0.
If none of the above criteria is met,
uv__next_timeout method is called to determine how long libuv should wait for I/O.
uv__next_timeout does is, it will return the value of the closest timer’s value. And if there are no timers, it will return -1 indicating infinity.
Now you should have the answer to the question “ Why do we block for I/O after executing any completed I/O callbacks? Shouldn’t Node be non-blocking? ”……
The event loop will not be blocked if there are any pending tasks to be executed. If there are no pending tasks to be executed, it will only be blocked until the next timer goes off, which re-activates the loop.
I hope you are still following me !!! I know this might be too much detail for you. But to understand this clearly, it is necessary to have a clear idea of what’s happening underneath.
Now we know how long the loop should wait for any I/O to complete. This timeout value is then passed to
uv__io_poll function. This function will watch for any incoming I/O operations until this timeout expires or system-specified maximum safe timeout reaches. After the timeout, event loop will again become active and move on to the “check handlers” phase.
I/O Polling happens differently on different OS platforms. In Linux, this is performed by
epoll_wait kernel system calls, on macOS using kqueue. In Windows, it’s performed using GetQueuedCompletionStatus in IOCP(Input Output Completion Port). I wouldn’t dig deep into how I/O polling works because it’s really complex and deserves another series of posts (which I don’t think I would write).
So far, we didn’t talk about the thread pool in this articles. As we saw in the first article in this series, threadpool is mostly used to perform all File I/O operations, getaddrinfo and getnameinfo calls during DNS operations merely due to the complexities of File I/O in different platforms (for a solid idea of these complexities, please read this post). Since the size of thread pool is limited (default size is 4), multiple requests to file system operations can still be blocked until a thread becomes available to work. However, size of the thread pool can be increased up to 128 (at the time of this writing) using the environment variable
UV_THREADPOOL_SIZE, to increase the performance of the application.
Still, this fixed-size thread pool has identified to be a bottleneck for NodeJS applications because, File I/O, getaddrinfo, getnameinfo are not the only operations carried out by the thread pool. Certain CPU intensive Crypto operations such as randomBytes, randomFill and pbkdf2 are also run on the libuv thread pool to prevent any adverse effects on the application’s performance but, by which also makes available threads an even scarce resource for I/O operations.
As of a previous libuv enhancement proposal, it was suggested to make thread pool scalable based on the load, but this proposal has eventually been withdrawn in order to replace it with a pluggable API for threading which might be introduced in the future.
Some parts of this article are inspired by the presentation done by Saúl Ibarra Corretgé at NodeConfEU 2016. If you’d like to learn more about libuv, I would highly recommend you watch it.
In this post, I described how I/O is performed in NodeJS in detail, diving into libuv source code itself. I believe the non-blocking, event-driven model of NodeJS makes more sense to you now. If you have any questions, I’d really like to answer them. Therefore, please don’t hesitate to respond to this article. And if you really like this article, I’d love it if you can clap and encourage me to write more. Thanks.
- Official Libuv Documentation http://docs.libuv.org/
- NodeJS Guides https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/
- Libuv Github https://github.com/libuv
Background Image Courtesy: https://i.imgur.com/JCVqX0Vr.jpg