DEV Community

Cover image for How to use map, filter and reduce in Python
Michael Currin
Michael Currin

Posted on

How to use map, filter and reduce in Python

This is part of my Functional Programming post series.

Demonstration of how to use map, filter, reduce.

These functions all built into Python. There are actually not included originally but a contributor with a background in a Functional Programming (Lisp, I think) missed them and added them.

Map

Using a named function.

def square(value):
    return value**2


a = [1, 2, 3]
b = map(square, a)
list(b)
# [1, 4, 9]
Enter fullscreen mode Exit fullscreen mode

Skip to Consuming a generator section to understand by list is needed.

Using a lambda, i.e. an anonymous function. This typically uses x as the variable.

a = [1, 2, 3]
b = map(lambda x: x**2, a)
list(b)
# [1, 4, 9]
Enter fullscreen mode Exit fullscreen mode

For the next two sections, I'll just use the lambda style.

Filter

a = [1, 2, 3]
b = filter(lambda x: x > 2, a)
list(b)
# [3]
Enter fullscreen mode Exit fullscreen mode

Combining map and filter. Note the innermost call is evaluated first - in this case, the map call.

a = [1, 2, 3]
b = filter(lambda x: x > 1, map(lambda x: x**2, a))
b
# [4, 9]
Enter fullscreen mode Exit fullscreen mode

Reduce

Use reduce to do calculations on an list which accumulate along the way and output as a single value like a str or int, not a list.

Here we add all the values together (pretending that sum doesn't exist).

In Python 3, you have to import reduce. But map and filter don't need an import.

from functools import reduce


a = [1, 2, 3]
b = reduce(lambda x, y: x+y, a, 0)
b
# 6
Enter fullscreen mode Exit fullscreen mode

The lambda has two variables - we use x for the value from the previous iteration and y for the current item. We use 0 as previous value on the first iteration.

Breaking down what happens in each iteration:

# prev  current
0     + 1
# result: 1

# prev  current
1     + 2
# result: 3

# prev  current
2     + 3 
# result: 6
Enter fullscreen mode Exit fullscreen mode

We already have the sum function in Python, so what we did above is not novel.

Next, we multiply all values together in a list i.e. 1*5*6*7, using 1 as the starting value.

a = [5, 6, 7]
b = reduce(lambda x, y: x*y, a, 1)
b
# 210
Enter fullscreen mode Exit fullscreen mode

Consuming a generator

Note that in Python 3 that map and filter are generators now. This is more efficient from a memory perspective but requires you to use list or a for loop to consume the values in the generator.

b = map(square, a)

# Show a reference to the object which has computed nothing.
b
# <map object at 0x1019aa310>

# Consume all of the values.
c = list(b)
c
# [1, 4, 9]

# The generator is now empty.
list(b)
# []

# Calculate it over.
b = map(square, a)

# Demonstration with a for loop instead of list(b).
for i in b:
    print(i)
# 1
# 4
# 9
Enter fullscreen mode Exit fullscreen mode

Scoping and safety

One more thing to note about passing a value, to map, reduce or filter is that an actual value is in to the call, not just a reference. This is like closures in JavaScript, where you would use the let keyword.

Here is a standard flow.

a = [1, 2, 3]
b = map(lambda x: x + 1, a)
list(b)
# [2, 3, 4]
Enter fullscreen mode Exit fullscreen mode

But here, we change a in between defining b and consuming the map. Yet we get the same result for b.

a = [1, 2, 3]
b = map(lambda x: x + 1, a)
a = [9, 10]
list(b)
# [2, 3, 4]
Enter fullscreen mode Exit fullscreen mode

What happened is that we only reassigned a to point to [9, 10]. The old value [1, 2, 3] still exists in the scope of the uncalled b call, just without a variable name assigned to it.

This is good for Functional Programming principle. The unconsumed b call has state in the sense that is has a value passed into it that it knows about. But the uncalled b does not depend on an external value of a which could change.

Think how unpredictable it would be if in the middle of the b map computing values in a, then a pointed somewhere else or had a value changed or even changed size.

Python doesn't protect you from changing the size of a list when iterating over it... this for will never get to the last element as each time it iterates it adds one to the end.

a = [1, 2, 3]
for x in a:
    a.append(x+1)
Enter fullscreen mode Exit fullscreen mode

I had to cancel the command after a few seconds and saw that the list had ballooned in size.

a[:20]
# [1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6, 5, 6, 7, 6, 7, 8, 7, 8]
# len(a)
14968136
Enter fullscreen mode Exit fullscreen mode

Fortunately, Python does protect you when it comes to updating a dictionary when iterating over it:

c = {'A': 1}
for k in c:
    c['B'] = k
Enter fullscreen mode Exit fullscreen mode
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration
Enter fullscreen mode Exit fullscreen mode

Links

Interested in more of my writing on Python? I have 3 main places where I collect Python material that is useful to me.

Here are all my Python repos on GitHub if you want to see what I like to build.

Top comments (0)