Demystifying Python Decorators in Less Than 10 Minutes

Please note: this post originally appeared on adrianperea.dev

In this quick tutorial, we will learn all about Python decorators.

The Python Decorator is syntactic sugar for writing function transformations. This excerpt from PEP 318 describes the motivation for it:

The current method for transforming functions and methods … is awkward and can lead to code that is difficult to understand. Ideally, these transformations should be made at the same point in the code where the declaration itself is made.

Understanding decorators, then, is a matter of understanding what function transformations are. Once we understand them, we’ll easily digest decorators.

So, what are function transformations? The name sounds daunting, but they’re quite simple. By definition, function transformations are no more than functions that extend the functionality of other functions, without modifying their original behaviors. They are like toppings on frozen yogurt: they add an extra zing and oomph, but don’t take away the vanilla flavor. They wrap and decorate our functions so that they adhere to a particular representation.

They are like toppings on froyo: they add an extra zing and oomph but don’t take away the vanilla flavor. Source: Pixabay

This will all clear up once we start going through some examples. Let’s get started.

Higher-order Functions

Before jumping straight into function transformations, we should first learn about higher-order functions. We will see that function transformations make a lot of use of this concept.

Higher-order functions are functions that have the following properties:

• They accept functions as arguments

• They return functions as results

How can higher-order functions be applied to Python? It’s straightforward: functions in Python are first-class objects. This means that like any other objects, functions can:

• be assigned to variables

• be passed as function arguments

• be returned from other functions

Therefore, we don’t need any special procedures. Python supports higher-order functions right off the bat!

Let’s see an example:

def say_hello(name):
    print(f'Hello, {name}!')


def say_goodbye(name):
    print(f'Goodbye, {name}!')


def say_to_bob(fun):
    fun('Bob')


say_to_bob(say_hello)
say_to_bob(say_goodbye)

Here, say_to_bob is accepting a function parameter fun. We can then use this by passing say_hello and say_goodbye as arguments to say_to_bob.

Seeing this in action:

Hello, Bob!
Goodbye, Bob!

Note that when we pass functions as arguments, we don’t include the parentheses. If we do, Python evaluates the function before it gets passed. Often it is easy to spot, since calling the function without the proper arguments will lead to errors:

>>> say_to_bob(say_hello())
TypeError: say_hello() missing 1 required positional argument: 'name'

This satisfies the first condition for a higher-order function.

Now, let’s change the code above to satisfy the second condition:

def say_hello(name):
    print(f'Hello, {name}!')


def say_goodbye(name):
    print(f'Goodbye, {name}!')


def get_greeting(greeting):
    if greeting == 'hello':
        greeting_fun = say_hello
    elif greeting == 'goodbye':
        greeting_fun = say_goodbye

    return greeting_fun


def say_to_bob(greeting):
    greeting_fun = get_greeting(greeting)
    greeting_fun('Bob')


say_to_bob('hello')
say_to_bob('goodbye')

The get_greeting function returns a different greeting function depending on the greeting argument. say_to_bobcalls this function and gets a reference to the appropriate greeting function. It then calls this function with Bob.

We get the same output:

Hello, Bob!
Goodbye, Bob!

Note once again that when we return functions, we omit the parentheses.

Function Transformations by Hand

How can we use higher-order functions to make function transformations?

Like I mentioned earlier:

By definition, function transformations are no more than functions that extend the functionality of other functions, without modifying their original behaviors.

From this definition, let’s breakdown a function transformation into three properties:

• It’s a higher-order function

• It adds extra functionality to the passed function

• It retains the original functionality of the passed function

Let’s see an example wherein we add a few debug messages before and after calling a function:

def walkout():
    print('Bye Felicia')


def debug_transformer(fun):
    def wrapper():
        print(f'Function `{fun.__name__}` called')
        fun()
        print(f'Function `{fun.__name__}` finished')

    return wrapper


walkout = debug_transformer(walkout)
walkout()

Can you see what happened? Let’s look at the output:

Function `walkout` called
Bye Felicia
Function `walkout` finished

Note that we aren’t doing anything new here. This is the same concept of higher-order functions but in a different pattern.

We did the transformation of walkout in this line:

walkout = debug_transformer(walkout)

From here, the function walkout no longer points to the original function definition. It now points to the wrapper function which has refers to our original function:

def wrapper():
    print(f'Function `{fun.__name__}` called')
    fun() # Original reference to walkout()
    print(f'Function `{fun.__name__}` finished')

The @-syntax (Read: Pie-decorator-syntax)

Source: Pixabay

As you might have observed, writing function transformations is quite cumbersome. From the simple example above, we had to write walkout three times to transform it.

To simplify this pattern, the Python team introduced the @-syntax. Let’s see how we can use it to simplify our code above:

def debug_transformer(fun):
    def wrapper():
        print(f'Function `{fun.__name__}` called')
        fun()
        print(f'Function `{fun.__name__}` finished')

    return wrapper


@debug_transformer
def walkout():
    print('Bye Felicia')

walkout()

If you run it, you will get the same results as before:

Function `walkout` called
Bye Felicia
Function `walkout` finished

Much better! Behind the scenes, what the @-syntax does is the following modification:

# Before
walkout = debug_transformer(walkout)

# After
@debug_transformer
def walkout():
    print('Bye Felicia')

@debug_transformer is a simplified version of walkout=debug_transformer(walkout). This makes it easier to read code since the decorator and function definition are in the same place. Neat!

Vanishing Return Values

What happens if we apply our decorator to a function with a return value?

def debug_transformer(fun):
    def wrapper():
        print(f'Function `{fun.__name__}` called')
        fun()
        print(f'Function `{fun.__name__}` finished')

    return wrapper


@debug_transformer
def walkout():
    print('Bye Felicia')


@debug_transformer
def get_bob():
    return 'Bob'

bob = get_bob()
print(bob)

Which outputs:

Function `get_bob` called
Function `get_bob` finished
None

The decoration works, but we have no more reference to the return value of the original function. To make this work, change the wrapperfunction to return the results of the original function:

def debug_transformer(fun):
    def wrapper():
        print(f'Function `{fun.__name__}` called')
        res = fun() # get reference to original return value
        print(f'Function `{fun.__name__}` finished')

        return res

    return wrapper

We now get our expected output:

>>> bob = get_bob()
Function `get_bob` called
Function `get_bob` finished
>>> print(bob)
Bob

Decorating Functions with Arguments

Will our decorator still work if we decorate a function with an argument?

@debug_transformer
def walkout(name):
    print(f'Bye {name}')


walkout('Felicia')

This gives us the following error:

TypeError: wrapper() takes 0 positional arguments but 1 was given

The problem occurs because the wrapper function is not expecting any arguments. To fix this, we can change it to receive a single argument so our walkout function won’t complain. However, doing so limits the use of our decorator to functions that only receive one argument. We need a generalized solution.

We can change wrapper to receive an arbitrary number of arguments using *args and **kwargs. By doing so, we can support functions with any number of arguments:

def debug_transformer(fun):
    # Allow wrapper to receive arbitrary args
    def wrapper(*args, **kwargs):
        print(f'Function `{fun.__name__}` called')
        # And pass it to the original function
        res = fun(*args, **kwargs)
        print(f'Function `{fun.__name__}` finished')
        return res

    return wrapper

Try it out!

>>> walkout('Dionisia')
Function `walkout` called
Bye Dionisia
Function `walkout` finished

Great! It works. You can experiment and see that this solution for functions with any number of arguments.

A Couple More Examples

You can see that there is no magic involved in creating decorators. Knowing this, your creativity is the only limit in designing them!

Here are a couple of examples for inspiration:

Calling a function many times

In this example, the original function is called many times inside the wrapper. The final function call receives the result and returns it.

def call_three_times(fun):
    def wrapper(*args, **kwargs):
        fun(*args, **kwargs)
        fun(*args, **kwargs)
        res = fun(*args, **kwargs)

        return res

    return wrapper


@call_three_times
def say_hey():
    print('Hey!')


say_hey()

# Output
Hey!
Hey!
Hey!

Timing a function

Sometimes we want to note how long a function takes to run. We can do this effortlessly with decorators:

import time

def time_it(fun):
    def wrapper(*args, **kwargs):
        start = time.time()
        res = fun(*args, **kwargs)
        end = time.time()
        print(f'Function took {end-start}s')

        return res

    return wrapper


@time_it
def waste_time():
    for i in range(10000000):
        pass


waste_time()

# Output
Function took 0.18418407440185547s

Simple yet effective! You will see in the next section that decorators gain even more power when used with external libraries.

Decorating External Functions

How can we apply decorators to external functions if we can’t use the @-syntax to decorate them?

At the start of the article, we created decorators by calling function transforms directly on the functions. We can do the same thing to external functions! Say we want to apply our time_it decorator to the np.sort function. We can do the following:

import numpy as np

rng = np.random.RandomState(0)

# Create a lot of numbers
nums = rng.random(10000000)
# Decorate np.sort with our time_it transformer
timed_sort = time_it(np.sort)
# Perform the sort with our time_it functionality
timed_sort(nums)

Note here that we assigned the decorated np.sort to a new name instead of reassigning it to np.sort. This allows to keep the original and transformed versions of the function.

Conclusion

At the beginning of the tutorial, we learned about:

• higher order functions: functions that accept and return other functions

• function transformations: a technique of utilizing higher-order functions to add functionality

Then, we talked about how the @-syntax streamlines creating function transformations. We then made decorators that:

• Preserve the original return value of the function

• Accept an arbitrary number of arguments

We ended by:

• Seeing creative examples of decorators

• Learning how to decorate external functions

Simple, right?

Remember that there’s no magic in creating decorators! They are no more than function transformations written in a funny syntax.

Further reading

• PEP 318. The spec sheet for decorators (and why they call @-syntax the pie-decorator-syntax!)

• Real Python Primer on Decorators. More examples of creating decorators.

• Scopes and Namespaces and Python Closures. How does the decorator preserve its reference to the original function?

Hi! I'm Adrian, and I'm a software engineer. I work hard to provide helpful and highly intuitive content for free. If you like what you read, check out my blog or follow me on Twitter. Hope to see you again next time!