Usually unless there is a spike in traffic the memory usage of a server should be regular, and stable. Memory usage of a server that continuously increases over time / per request could indicate a memory leak bug.
This post describes how to use "Chrome Dev Tools for Node" to profile memory of a NodeJs App, and identify the source of a memory leak.
Setup
git clone git@github.com:maybebored/codesamples.git
cd memory-leak-example
# Build
yarn build
# Run the app with --inspect so it can be profiled
node --inspect ./dist/index.js
# This will output the debugger port to connect ex:- Debugger listening on ws://127.0.0.1:9229/a2fc3802-d5d9-497b-b3f7-9fe439414e13
Use the dedicated DevTools for Node on Chrome Devices Page. Navigate to the Memory tab, and check that the Node process shown matches the debugger port of your app. If it doesn't show there, go to Connection page and add the connection as localhost:{Your debugger port}, example mine is localhost:9229.
Profiling
There are 3 different profiling types shown here. This post will utilise the Heap snapshot. Perhaps a future post will explore the other 2 types in more detail.
Heap Snapshot
A heap snapshot is what it sounds like, a snapshot of all the objects stored in the heap, how much memory they are using, what created them, and some more details.
I found that comparing snapshots overtime provided the most useful hints to find the source of the leak.
Click on the record button to take a snapshot before firing any requests. This serves as the baseline. Then fire requests and take a snapshot. Repeat this a few times until the analysis reveals something useful.
# fire n requests
ab -n 1000 http://localhost:4000/info
Snapshot 1
After taking the first snapshot, by default a Summary view is displayed, which shows some important data. The different columns are explained in detail in Chrome's developer docs
One thing to note here is the difference between the column Shallow Size, and Retained Size. Shallow size is memory held by the the Constructor alone, Shallow size is memory held by the Constructor indirectly because it holds a reference to another object. By default the Constructors are sorted by their retained size.
Snapshot 2
This snapshot was taken a few seconds after firing 6000 requests. It is important to remember each snapshot does garbage collection before the snapshot. This means, the memory usage shown in the snapshot is that which is currently in usage.
In this snapshot we notice the memory usage is 24.3 MB. We also notice that in 6th place is a class defined in the app's code: Logger, holding roughly 14 MB - over 50% of the heap memory. This already provides some useful clues about where the issue could be, yet it does not point exactly where the issue is.
Compare snapshot 2 to snapshot 1
Comparison view shows "Size Delta" in Constructors between Snapshot 2 and 1. This allows us to scope down into the problem. When sorting by "Size Delta" it is clear that almost 13 MB of strings were created.
Clicking on (strings), reveals all the strings that were created, and clicking on each opens a detailed view of its contents. At the bottom, in Retainers tab, a hierarchical view of the string's references are shown. Over here, it is noticeable a variable named buffer that belongs to our Logger class is referencing it.
Every other string also points to the same variable buffer. With this information, it is clear enough to debug the code to fix the memory leak.
Note that the hierarchy of references extends all the way until express.js handler functions, and then to NodeJs internal event loop functions. This is the reason why Summary view of a snapshot is not very useful as internal closures' retained memory also increases.
Summary
- Chrome DevTools allows for 3 types of memory profiling NodeJs Apps
- Heap snapshot types allows taking many snapshots and comparing them.
- Summary view of a snapshot does not give an immediate clue as to where the problem is because library functions / internal classes can also appear there.
- Comparison view shows objects created between 2 snapshots, allowing to scope down to the problem area.
Top comments (0)