We use python's Lists pretty much everywhere. Since we don't declare it's size and we just add items, Ever wondered how and when the memory allocation and consumption is done under the hood?
Lets analyze the same now, for that we need...
-
sys
package to get the list size -
matplotlib
for plotting graphs
-
Lets start writing code by importing the packages.
import sys
from matplotlib import pyplot as plt
Lets print and see the length and memory of our list for 10 items.
l = [] # our list
size = 10
# here we are just adding some values to our list
for counter in range(size):
l.append(counter)
print("length {}, memory in bytes {}".format(
length[counter], memory[counter]))
We can see the following in console.
length 1, memory in bytes 96 length 2, memory in bytes 96 length 3, memory in bytes 96 length 4, memory in bytes 96 length 5, memory in bytes 128 length 6, memory in bytes 128 length 7, memory in bytes 128 length 8, memory in bytes 128 length 9, memory in bytes 192 length 10, memory in bytes 192
We can see that after about 4 iterations each time, the memory is increased.
Now lets increase the loop counter to 1000 and graph it out...
size = 1000 # increasing the size
length = [] # to hold lengths at each iteration
memory = [] # to hold memory values at each iteration
And then change the for loop
for counter in range(size):
l.append(counter)
# capture and store the length
length.append(len(l))
# Capture and store the memory in bytes
memory.append(sys.getsizeof(l))
Now lets plot the graph using the captured values and see what is shows.
plt.plot(length, memory)
plt.title("Python List length - Memory comparison")
plt.xlabel("Length")
plt.ylabel("Memory in bytes")
plt.show()
We can see that when ever list is running out of size, its getting added with additional memory.
Now, Lets change the code to capture the difference in memory only when there is a change.
# declaring two more lists
counter_at_which_memory_changed = []
changed_memory_difference = []
and changing the for loop as follows
for counter in range(size):
l.append(counter)
# capture and store the length
length.append(len(l))
# Capture and store the memory in bytes
memory.append(sys.getsizeof(l))
if memory[counter] != memory[counter - 1]:
counter_at_which_memory_changed.append(counter)
changed_memory_difference.append(
memory[counter] - memory[counter - 1])
# Lets try checking the first 10 values of length and difference in memory
for counter in range(10):
print("at list length {}, memory increase in bytes {}".format(
counter_at_which_memory_changed[counter],
changed_memory_difference[counter]))
The first 10 values of memory changes is as follows.
at list length 4, memory increase in bytes 32 at list length 8, memory increase in bytes 64 at list length 16, memory increase in bytes 72 at list length 25, memory increase in bytes 80 at list length 35, memory increase in bytes 88 at list length 46, memory increase in bytes 96 at list length 58, memory increase in bytes 112 at list length 72, memory increase in bytes 128 at list length 88, memory increase in bytes 144 at list length 106, memory increase in bytes 160
# And the graph for index - memory difference is as follows
plt.plot(counter_at_which_memory_changed, changed_memory_difference)
plt.title("Counter - Memory difference")
plt.xlabel("Counter")
plt.ylabel("Memory difference")
plt.show()
So, by checking the difference we can see that the memory increase is follows..
8, 8, 8, 8, 16, 16, 16, 16, 16, 24, 24, 32, 32, 32, 40, 48, 48, 56, 64, 72, 88, 88, 104, 112.
Thanks for your time, this is my first post. Comments are welcome :)
Top comments (0)