DEV Community

loading...
Cover image for One Simple Trick to Reduce Your Memory Usage in Python

One Simple Trick to Reduce Your Memory Usage in Python

Connor
ãƒŧ1 min read

Table of Contents

  1. Intro
  2. List Comprehension
  3. Generator Comprehension

Intro

Python is ultimate "getting things done" language, where you can soo easily write code and not worry too much about performance and memory. However once your program becomes large, large memory usage can significantly slow down your program. One easy way to reduce memory usage and speed up your programs is to switch your list comprehensions into generator comprehensions.

Lets explore this with a simple example program to sum up a range of numbers.

List Comprehension

Code:

import sys
my_large_list = [i for i in range(100000)]
print(sum(my_large_list))
print(f"My list is {sys.getsizeof(my_large_list)} bytes")

Enter fullscreen mode Exit fullscreen mode

Output:

4999950000
My list is 824456 bytes
Enter fullscreen mode Exit fullscreen mode

Generator Comprehension

My code is often filled with a lot of list comprehensions, where instead we could use generators instead. Generators operate like lists, except they are evaluated "lazily", so the values are grabbed when needed.

All we need to do is use curly braces on all of our list comprehensions.

Code:

import sys
my_large_generator_list = (i for i in range(100000))
print(sum(my_large_list))
print(f"My generator is {sys.getsizeof(my_large_generator_list)} bytes")
Enter fullscreen mode Exit fullscreen mode

Output:

4999950000
My list is 112 bytes
Enter fullscreen mode Exit fullscreen mode

As we can see both give the same result, however the generator only uses a fraction of the memory (112 bytes instead of 824456). When you have hundreds of lists floating in your code, switching them to generators is an easy way to save on memory and increase your program's speed :).

Discussion (2)

Collapse
juancarlospaco profile image
Juan Carlos
import sugar, math
let my_large_list = collect(new_seq, for i in 0 ..< 100000: i)
echo sum(my_large_list)
echo "My list is ", sizeof(my_large_list), " bytes"
Enter fullscreen mode Exit fullscreen mode

play.nim-lang.org/#ix=3noU
ðŸĪŠðŸ‘

Collapse
iceorfiresite profile image
Ice or Fire

Good tip!