How to use map, filter and reduce in Python

This is part of my Functional Programming post series.

Demonstration of how to use map, filter, reduce.

These functions all built into Python. There are actually not included originally but a contributor with a background in a Functional Programming (Lisp, I think) missed them and added them.

Map

Using a named function.

def square(value):
    return value**2


a = [1, 2, 3]
b = map(square, a)
list(b)
# [1, 4, 9]

Skip to Consuming a generator section to understand by list is needed.

Using a lambda, i.e. an anonymous function. This typically uses x as the variable.

a = [1, 2, 3]
b = map(lambda x: x**2, a)
list(b)
# [1, 4, 9]

For the next two sections, I'll just use the lambda style.

Filter

a = [1, 2, 3]
b = filter(lambda x: x > 2, a)
list(b)
# [3]

Combining map and filter. Note the innermost call is evaluated first - in this case, the map call.

a = [1, 2, 3]
b = filter(lambda x: x > 1, map(lambda x: x**2, a))
b
# [4, 9]

Reduce

Use reduce to do calculations on an list which accumulate along the way and output as a single value like a str or int, not a list.

Here we add all the values together (pretending that sum doesn't exist).

In Python 3, you have to import reduce. But map and filter don't need an import.

from functools import reduce


a = [1, 2, 3]
b = reduce(lambda x, y: x+y, a, 0)
b
# 6

The lambda has two variables - we use x for the value from the previous iteration and y for the current item. We use 0 as previous value on the first iteration.

Breaking down what happens in each iteration:

# prev  current
0     + 1
# result: 1

# prev  current
1     + 2
# result: 3

# prev  current
2     + 3 
# result: 6

We already have the sum function in Python, so what we did above is not novel.

Next, we multiply all values together in a list i.e. 1*5*6*7, using 1 as the starting value.

a = [5, 6, 7]
b = reduce(lambda x, y: x*y, a, 1)
b
# 210

Consuming a generator

Note that in Python 3 that map and filter are generators now. This is more efficient from a memory perspective but requires you to use list or a for loop to consume the values in the generator.

b = map(square, a)

# Show a reference to the object which has computed nothing.
b
# <map object at 0x1019aa310>

# Consume all of the values.
c = list(b)
c
# [1, 4, 9]

# The generator is now empty.
list(b)
# []

# Calculate it over.
b = map(square, a)

# Demonstration with a for loop instead of list(b).
for i in b:
    print(i)
# 1
# 4
# 9

Scoping and safety

One more thing to note about passing a value, to map, reduce or filter is that an actual value is in to the call, not just a reference. This is like closures in JavaScript, where you would use the let keyword.

Here is a standard flow.

a = [1, 2, 3]
b = map(lambda x: x + 1, a)
list(b)
# [2, 3, 4]

But here, we change a in between defining b and consuming the map. Yet we get the same result for b.

a = [1, 2, 3]
b = map(lambda x: x + 1, a)
a = [9, 10]
list(b)
# [2, 3, 4]

What happened is that we only reassigned a to point to [9, 10]. The old value [1, 2, 3] still exists in the scope of the uncalled b call, just without a variable name assigned to it.

This is good for Functional Programming principle. The unconsumed b call has state in the sense that is has a value passed into it that it knows about. But the uncalled b does not depend on an external value of a which could change.

Think how unpredictable it would be if in the middle of the b map computing values in a, then a pointed somewhere else or had a value changed or even changed size.

Python doesn't protect you from changing the size of a list when iterating over it... this for will never get to the last element as each time it iterates it adds one to the end.

a = [1, 2, 3]
for x in a:
    a.append(x+1)

I had to cancel the command after a few seconds and saw that the list had ballooned in size.

a[:20]
# [1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6, 5, 6, 7, 6, 7, 8, 7, 8]
# len(a)
14968136

Fortunately, Python does protect you when it comes to updating a dictionary when iterating over it:

c = {'A': 1}
for k in c:
    c['B'] = k

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration

Links

Interested in more of my writing on Python? I have 3 main places where I collect Python material that is useful to me.

• Python topic on Learn to Code - good for beginners but the resources list will useful for all levels.
• Python Cheatsheets
• Python Recipes

Here are all my Python repos on GitHub if you want to see what I like to build.