With different types of performance tests, both basic and advanced application errors can be detected easily. Among them are memory leaks. What are they?
Wikipedia defines memory leaks as:
“In computer science, a memory leak is a type of resource leak that occurs when a computer program incorrectly manages memory allocations in a way that memory which is no longer needed is not released. A memory leak may also happen when an object is stored in memory but cannot be accessed by the running code.”
This means that a bug in the code causes objects to be retained in memory that we no longer need. By doing so, the memory fills up to the point of exhaustion, which causes it to be missing for other needed processes.
Garbage Collections
To understand the bug more closely, let’s talk about garbage collection. This is a complicated topic, to which an entire article would have to be devoted. So let’s just take a cursory look at it.
To perform operations in programming languages, objects are used. These are variables, functions and classes. To access them quickly, they are stored in the heap, which is an area of RAM used for dynamic allocation. Most often, a variable in program code is a pointer to the address of an object in memory. When a variable is used, its value is read from memory, so we can operate on it.
Some of the low-level languages as well as high-level languages need to allocate and release memory from the code level in order to use it for other operations. However, there are languages that do this automatically for the programmer. The name of this method is Garbage Collection. Languages that use it include Java, for example.
Real life scenario
Let’s take a simple application written in the flask framework that causes memory leaks to check the occurrence of the error on a live example. This is a simple script in which a global variable named global_unsafe_var is defined. Then let’s create an endpoint named “memoryleak”, which will add 40,000 new values to the global list when a GET request is made on it.
import memory_profiler as mp
from flask import Flask, cli
import logging
# define flask application
app = Flask(__name__)
# disable the display of flask (requests for example) data in the console
cli.show_server_banner = lambda *args: None
logging.getLogger("werkzeug").disabled = True
# our dangerous variable
global_unsafe_var = []
# defined function tracking
@mp.profile
def leak_memory():
global global_unsafe_var
# memory leak
global_unsafe_var.append([12, "unsafe", 42, True] * 10000)
return {'Is it dangeours?': True}
# define endpoint to which we will make dangerous requests
app.add_url_rule(rule='/memoryleak', endpoint='memoryleak', view_func=leak_memory, methods=['GET'])
# launch the flask application
if __name__ == '__main__':
app.run()
Since global_unsafe_var is a global variable instead of an instance variable, it fills up indefinitely, and it is not removed from memory itself. This causes a memory overflow after full time, and thus a memory leak.
Someone might ask, what about when the memory fills to capacity? Depending on the application, it may behave differently. Most often, however, its reset occurs.
K6 script
For the purpose of visualizing the memory leak, we have prepared a script written in K6. After running it, we will send a GET request to the faulty endpoint for one minute. Before that, the load will increase to five threads in 10 seconds.
import http from 'k6/http';
import { sleep } from 'k6';
export let options = {
stages: [
{ duration: '10s', target: 5 },
{ duration: '1m', target: 5 },
{ duration: '10s', target: 0 }
]
};
const API_BASE_URL = 'http://localhost:5000';
export default function () {
http.get(`${API_BASE_URL}/memoryleak`);
sleep(1);
};
During load, we will be able to see infinite memory growth in the application. But before we can do that, we need to monitor the resources used by the application in some way.
Profiler
To monitor resource consumption, profilers are used. One of them includes python’s memory-profiler module. After installing it, start the flask application via the command:
python -m memory_profiler app.py
This launches the flask application, and the console will only show data from the profiler.
Filename: app.py
Line # Mem usage Increment Occurrences Line Contents
=============================================================
16 37.6 MiB 37.6 MiB 1 @mp.profile
17 def leak_memory():
18 global global_unsafe_var
19
20 # memory leak
21 37.8 MiB 0.3 MiB 1 global_unsafe_var.append([12, "unsafe", 42, True] * 10000)
22 37.8 MiB 0.0 MiB 1 return {'Is it dangeours?': True}
Next, let’s run the k6 script.
memory-leak-flask> k6 run .\scenario.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: .\scenario.js
output: -
scenarios: (100.00%) 1 scenario, 5 max VUs, 1m50s max duration (incl. graceful stop):
* default: Up to 5 looping VUs for 1m20s over 3 stages (gracefulRampDown: 30s, gracefulStop: 30s)
running (0m11.2s), 5/5 VUs, 29 complete and 0 interrupted iterations
default [====>---------------------------------] 5/5 VUs 0m11.2s/1m20.0s
In the console where we ran the flask application, the memory consumption data is visible. As you can see, it increases indefinitely, which leads to memory leaks.
Filename: app.py
Line # Mem usage Increment Occurrences Line Contents
=============================================================
16 112.7 MiB 112.7 MiB 1 @mp.profile
17 def leak_memory():
18 global global_unsafe_var
19
20 # memory leak
21 113.0 MiB 0.3 MiB 1 global_unsafe_var.append([12, "unsafe", 42, True] * 10000)
22 113.0 MiB 0.0 MiB 1 return {'Is it dangeours?': True}
Top comments (0)