Deploy Django with Postgres, Nginx, and Gunicorn on Ubuntu 18.04
November 16, 2019
Metaclasses in Python
December 4, 2019
Show all

Python Decorators

A decorator takes in a function, adds some functionality and returns it. In this article, you will learn how you can create a decorator and why you should use it.

What are decorators in Python?

Python has an interesting feature called decorators to add functionality to an existing code.

This is also called metaprogramming as a part of the program tries to modify another part of the program at compile time.


Prerequisites for learning decorators

In order to understand about decorators, we must first know a few basic things in Python.

We must be comfortable with the fact that, everything in Python (Yes! Even classes), are objects. Names that we define are simply identifiers bound to these objects. Functions are no exceptions, they are objects too (with attributes). Various different names can be bound to the same function object.

Here is an example.

1234567def first(msg): print(msg) first(“Hello”)second = firstsecond(“Hello”)RunPowered by DataCamp

When you run the code, both functions first and second gives same output. Here, the names first and second refer to the same function object.

Now things start getting weirder.

Functions can be passed as arguments to another function.

If you have used functions like mapfilter and reduce in Python, then you already know about this.

Such function that take other functions as arguments are also called higher order functions. Here is an example of such a function.

123456789def inc(x): return x + 1def dec(x): return x – 1def operate(func, x): result = func(x) return resultRunPowered by DataCamp

We invoke the function as follows.

>>> operate(inc,3)4>>> operate(dec,3)2

Furthermore, a function can return another function.

123456789def is_called(): def is_returned(): print(“Hello”) return is_returnednew = is_called()#Outputs “Hello”new()RunPowered by DataCamp

Here, is_returned() is a nested function which is defined and returned, each time we call is_called().

Finally, we must know about closures in Python.


Getting back to Decorators

Functions and methods are called callable as they can be called.

In fact, any object which implements the special method __call__() is termed callable. So, in the most basic sense, a decorator is a callable that returns a callable.

Basically, a decorator takes in a function, adds some functionality and returns it.

12345678def make_pretty(func): def inner(): print(“I got decorated”) func() return innerdef ordinary(): print(“I am ordinary”)RunPowered by DataCamp

When you run the following codes in shell,

>>> ordinary()I am ordinary>>> # let's decorate this ordinary function>>> pretty = make_pretty(ordinary)>>> pretty()I got decoratedI am ordinary

In the example shown above, make_pretty() is a decorator. In the assignment step.

pretty = make_pretty(ordinary)

The function ordinary() got decorated and the returned function was given the name pretty.

We can see that the decorator function added some new functionality to the original function. This is similar to packing a gift. The decorator acts as a wrapper. The nature of the object that got decorated (actual gift inside) does not alter. But now, it looks pretty (since it got decorated).

Generally, we decorate a function and reassign it as,

ordinary = make_pretty(ordinary).

This is a common construct and for this reason, Python has a syntax to simplify this.

We can use the @ symbol along with the name of the decorator function and place it above the definition of the function to be decorated. For example,

@make_prettydef ordinary():    print("I am ordinary")

is equivalent to

def ordinary():    print("I am ordinary")ordinary = make_pretty(ordinary)

This is just a syntactic sugar to implement decorators.


Decorating Functions with Parameters

The above decorator was simple and it only worked with functions that did not have any parameters. What if we had functions that took in parameters like below?

def divide(a, b):    return a/b

This function has two parameters, a and b. We know, it will give error if we pass in b as 0.

>>> divide(2,5)0.4>>> divide(2,0)Traceback (most recent call last):...ZeroDivisionError: division by zero

Now let’s make a decorator to check for this case that will cause the error.

12345678910111213def smart_divide(func): def inner(a,b): print(“I am going to divide”,a,”and”,b) if b == 0: print(“Whoops! cannot divide”) return return func(a,b) return inner@smart_dividedef divide(a,b): return a/bRunPowered by DataCamp

This new implementation will return None if the error condition arises.

>>> divide(2,5)I am going to divide 2 and 50.4>>> divide(2,0)I am going to divide 2 and 0Whoops! cannot divide

In this manner we can decorate functions that take parameters.

A keen observer will notice that parameters of the nested inner() function inside the decorator is same as the parameters of functions it decorates. Taking this into account, now we can make general decorators that work with any number of parameter.

In Python, this magic is done as function(*args, **kwargs). In this way, args will be the tuple of positional arguments and kwargs will be the dictionary of keyword arguments. An example of such decorator will be.

def works_for_all(func):    def inner(*args, **kwargs):        print("I can decorate any function")        return func(*args, **kwargs)    return inner

Chaining Decorators in Python

Multiple decorators can be chained in Python.

This is to say, a function can be decorated multiple times with different (or same) decorators. We simply place the decorators above the desired function.

12345678910111213141516171819def star(func): def inner(*args, **kwargs): print(“*” * 30) func(*args, **kwargs) print(“*” * 30) return innerdef percent(func): def inner(*args, **kwargs): print(“%” * 30) func(*args, **kwargs) print(“%” * 30) return inner@star@percentdef printer(msg): print(msg)printer(“Hello”)RunPowered by DataCamp

This will give the output.

******************************
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Hello
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
******************************

The above syntax of,

@star@percentdef printer(msg):    print(msg)

is equivalent to

def printer(msg):    print(msg)printer = star(percent(printer))

The order in which we chain decorators matter. If we had reversed the order as,

@percent@stardef printer(msg):    print(msg)

The execution would take place as,

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
******************************
Hello
******************************
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


Understanding decorators is a milestone for any serious Python programmer. Here’s your step-by-step guide to how decorators can help you become a more efficient and productive Python developer.

Python’s decorators allow you to extend and modify the behavior of a callable (functions, methods, and classes) without permanently modifying the callable itself.

Any sufficiently generic functionality you can “tack on” to an existing class or function’s behavior makes a great use case for decoration. This includes:

  • logging,
  • enforcing access control and authentication,
  • instrumentation and timing functions,
  • rate-limiting,
  • caching; and more.

Why Should I Master Decorators in Python?

That’s a fair question. After all, what I just mentioned sounded quite abstract and it might be difficult to see how decorators can benefit you in your day-to-day work as a Python developer. Here’s an example:

Imagine you’ve got 30 functions with business logic in your report-generating program. One rainy Monday morning your boss walks up to your desk and says:

“Happy Monday! Remember those TPS reports? I need you to add input/output logging to each step in the report generator. XYZ Corp needs it for auditing purposes. Oh, and I told them we can ship this by Wednesday.”

Depending on whether or not you’ve got a solid grasp on Python’s decorators, this request will either send your blood pressure spiking—or leave you relatively calm.

Without decorators you might be spending the next three days scrambling to modify each of those 30 functions and clutter them up with manual logging calls. Fun times.

If you do know your decorators, you’ll calmly smile at your boss and say:

“Don’t worry Jim, I’ll get it done by 2pm today.”

Right after that you’ll type the code for a generic @audit_log decorator (that’s only about 10 lines long) and quickly paste it in front of each function definition. Then you’ll commit your code and grab another cup of coffee.

I’m dramatizing here. But only a little. Decorators can be that powerful 🙂

I’d go as far as to say that understanding decorators is a milestone for any serious Python programmer. They require a solid grasp of several advanced concepts in the language—including the properties of first-class functions.

But:

Understanding Decorators Is Worth It 💡

The payoff for understanding how decorators work in Python is huge.

Sure, decorators are relatively complicated to wrap your head around for the first time—but they’re a highly useful feature that you’ll often encounter in third-party frameworks and the Python standard library.

Explaining decorators is also a make or break moment for any good Python tutorial. I’ll do my best here to introduce you to them step by step.

Before you dive in, now would be an excellent moment to refresh your memory on the properties of first-class functions in Python. I wrote a tutorial on them here on dbader.org and I would encourage you to take a few minutes to review it. The most important “first-class functions” takeaways for understanding decorators are:

  • Functions are objects—they can be assigned to variables and passed to and returned from other functions; and
  • Functions can be defined inside other functions—and a child function can capture the parent function’s local state (lexical closures.)

Alright, ready to do this? Let’s start with some:

Python Decorator Basics

Now, what are decorators really? They “decorate” or “wrap” another function and let you execute code before and after the wrapped function runs.

Decorators allow you to define reusable building blocks that can change or extend the behavior of other functions. And they let you do that without permanently modifying the wrapped function itself. The function’s behavior changes only when it’s decorated.

Now what does the implementation of a simple decorator look like? In basic terms, a decorator is a callable that takes a callable as input and returns another callable.

The following function has that property and could be considered the simplest decorator one could possibly write:

def null_decorator(func):
    return func

As you can see, null_decorator is a callable (it’s a function), it takes another callable as its input, and it returns the same input callable without modifying it.

Let’s use it to decorate (or wrap) another function:

def greet():
    return 'Hello!'

greet = null_decorator(greet)

>>> greet()
'Hello!'

In this example I’ve defined a greet function and then immediately decorated it by running it through the null_decorator function. I know this doesn’t look very useful yet (I mean we specifically designed the null decorator to be useless, right?) but in a moment it’ll clarify how Python’s decorator syntax works.

Instead of explicitly calling null_decorator on greet and then reassigning the greet variable, you can use Python’s @ syntax for decorating a function in one step:

@null_decorator
def greet():
    return 'Hello!'

>>> greet()
'Hello!'

Putting an @null_decorator line in front of the function definition is the same as defining the function first and then running through the decorator. Using the @ syntax is just syntactic sugar, and a shortcut for this commonly used pattern.

Note that using the @ syntax decorates the function immediately at definition time. This makes it difficult to access the undecorated original without brittle hacks. Therefore you might choose to decorate some functions manually in order to retain the ability to call the undecorated function as well.

So far, so good. Let’s see how:

Decorators Can Modify Behavior

Now that you’re a little more familiar with the decorator syntax, let’s write another decorator that actually does something and modifies the behavior of the decorated function.

Here’s a slightly more complex decorator which converts the result of the decorated function to uppercase letters:

def uppercase(func):
    def wrapper():
        original_result = func()
        modified_result = original_result.upper()
        return modified_result
    return wrapper

Instead of simply returning the input function like the null decorator did, this uppercase decorator defines a new function on the fly (a closure) and uses it to wrap the input function in order to modify its behavior at call time.

The wrapper closure has access to the undecorated input function and it is free to execute additional code before and after calling the input function. (Technically, it doesn’t even need to call the input function at all.)

Note how up until now the decorated function has never been executed. Actually calling the input function at this point wouldn’t make any sense—you’ll want the decorator to be able to modify the behavior of its input function when it gets called eventually.

Time to see the uppercase decorator in action. What happens if you decorate the original greet function with it?

@uppercase
def greet():
    return 'Hello!'

>>> greet()
'HELLO!'

I hope this was the result you expected. Let’s take a closer look at what just happened here. Unlike null_decorator, our uppercase decorator returns a different function object when it decorates a function:

>>> greet
<function greet at 0x10e9f0950>

>>> null_decorator(greet)
<function greet at 0x10e9f0950>

>>> uppercase(greet)
<function uppercase.<locals>.wrapper at 0x10da02f28>

And as you saw earlier, it needs to do that in order to modify the behavior of the decorated function when it finally gets called. The uppercase decorator is a function itself. And the only way to influence the “future behavior” of an input function it decorates is to replace (or wrap) the input function with a closure.

That’s why uppercase defines and returns another function (the closure) that can then be called at a later time, run the original input function, and modify its result.

Decorators modify the behavior of a callable through a wrapper so you don’t have to permanently modify the original. The callable isn’t permanently modified—its behavior changes only when decorated.

This let’s you “tack on” reusable building blocks, like logging and other instrumentation, to existing functions and classes. It’s what makes decorators such a powerful feature in Python that’s frequently used in the standard library and in third-party packages.

⏰ A Quick Intermission

By the way, if you feel like you need a quick coffee break at this point—that’s totally normal. In my opinion closures and decorators are some of the most difficult concepts to understand in Python. Take your time and don’t worry about figuring this out immediately. Playing through the code examples in an interpreter session one by one often helps make things sink in.

I know you can do it 🙂

Applying Multiple Decorators to a Single Function

Perhaps not surprisingly, you can apply more than one decorator to a function. This accumulates their effects and it’s what makes decorators so helpful as reusable building blocks.

Here’s an example. The following two decorators wrap the output string of the decorated function in HTML tags. By looking at how the tags are nested you can see which order Python uses to apply multiple decorators:

def strong(func):
    def wrapper():
        return '<strong>' + func() + '</strong>'
    return wrapper

def emphasis(func):
    def wrapper():
        return '<em>' + func() + '</em>'
    return wrapper

Now let’s take these two decorators and apply them to our greet function at the same time. You can use the regular @ syntax for that and just “stack” multiple decorators on top of a single function:

@strong
@emphasis
def greet():
    return 'Hello!'

What output do you expect to see if you run the decorated function? Will the @emphasis decorator add its <em> tag first or does @strong have precedence? Here’s what happens when you call the decorated function:

>>> greet()
'<strong><em>Hello!</em></strong>'

This clearly shows in what order the decorators were applied: from bottom to top. First, the input function was wrapped by the @emphasis decorator, and then the resulting (decorated) function got wrapped again by the @strong decorator.

To help me remember this bottom to top order I like to call this behavior decorator stacking. You start building the stack at the bottom and then keep adding new blocks on top to work your way upwards.

If you break down the above example and avoid the @ syntax to apply the decorators, the chain of decorator function calls looks like this:

decorated_greet = strong(emphasis(greet))

Again you can see here that the emphasis decorator is applied first and then the resulting wrapped function is wrapped again by the strong decorator.

This also means that deep levels of decorator stacking will have an effect on performance eventually because they keep adding nested function calls. Usually this won’t be a problem in practice, but it’s something to keep in mind if you’re working on performance intensive code.

Decorating Functions That Accept Arguments

All examples so far only decorated a simple nullary greet function that didn’t take any arguments whatsoever. So the decorators you saw here up until now didn’t have to deal with forwarding arguments to the input function.

If you try to apply one of these decorators to a function that takes arguments it will not work correctly. How do you decorate a function that takes arbitrary arguments?

This is where Python’s *args and **kwargs feature for dealing with variable numbers of arguments comes in handy. The following proxy decorator takes advantage of that:

def proxy(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

There are two notable things going on with this decorator:

  • It uses the * and ** operators in the wrapper closure definition to collect all positional and keyword arguments and stores them in variables (args and kwargs).
  • The wrapper closure then forwards the collected arguments to the original input function using the * and ** “argument unpacking” operators.

(It’s a bit unfortunate that the meaning of the star and double-star operators is overloaded and changes depending on the context they’re used in. But I hope you get the idea.)

Let’s expand the technique laid out by the proxy decorator into a more useful practical example. Here’s a trace decorator that logs function arguments and results during execution time:

def trace(func):
    def wrapper(*args, **kwargs):
        print(f'TRACE: calling {func.__name__}() '
              f'with {args}, {kwargs}')

        original_result = func(*args, **kwargs)

        print(f'TRACE: {func.__name__}() '
              f'returned {original_result!r}')

        return original_result
    return wrapper

Decorating a function with trace and then calling it will print the arguments passed to the decorated function and its return value. This is still somewhat of a toy example—but in a pinch it makes a great debugging aid:

@trace
def say(name, line):
    return f'{name}: {line}'

>>> say('Jane', 'Hello, World')
'TRACE: calling say() with ("Jane", "Hello, World"), {}'
'TRACE: say() returned "Jane: Hello, World"'
'Jane: Hello, World'

Speaking of debugging—there are some things you should keep in mind when debugging decorators:

How to Write “Debuggable” Decorators

When you use a decorator, really what you’re doing is replacing one function with another. One downside of this process is that it “hides” some of the metadata attached to the original (undecorated) function.

For example, the original function name, its docstring, and parameter list are hidden by the wrapper closure:

def greet():
    """Return a friendly greeting."""
    return 'Hello!'

decorated_greet = uppercase(greet)

If you try to access any of that function metadata you’ll see the wrapper closure’s metadata instead:

>>> greet.__name__
'greet'
>>> greet.__doc__
'Return a friendly greeting.'

>>> decorated_greet.__name__
'wrapper'
>>> decorated_greet.__doc__
None

This makes debugging and working with the Python interpreter awkward and challenging. Thankfully there’s a quick fix for this: the functools.wraps decorator included in Python’s standard library.

You can use functools.wraps in your own decorators to copy over the lost metadata from the undecorated function to the decorator closure. Here’s an example:

import functools

def uppercase(func):
    @functools.wraps(func)
    def wrapper():
        return func().upper()
    return wrapper

Applying functools.wraps to the wrapper closure returned by the decorator carries over the docstring and other metadata of the input function:

@uppercase
def greet():
    """Return a friendly greeting."""
    return 'Hello!'

>>> greet.__name__
'greet'
>>> greet.__doc__
'Return a friendly greeting.'

As a best practice I’d recommend that you use functools.wraps in all of the decorators you write yourself. It doesn’t take much time and it will save you (and others) debugging headaches down the road.

Python Decorators – Key Takeaways

  • Decorators define reusable building blocks you can apply to a callable to modify its behavior without permanently modifying the callable itself.
  • The @ syntax is just a shorthand for calling the decorator on an input function. Multiple decorators on a single function are applied bottom to top (decorator stacking).
  • As a debugging best practice, use the functools.wraps helper in your own decorators to carry over metadata from the undecorated callable to the decorated one.

Was this tutorial helpful? Got any suggestions on how it could be improved that could help other learners? Leave a comment below and share your thoughts.

In this tutorial on decorators, we’ll look at what they are and how to create and use them. Decorators provide a simple syntax for calling higher-order functions.

By definition, a decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it.

This sounds confusing, but it’s really not, especially after you’ve seen a few examples of how decorators work. You can find all the examples from this article here.

Free Bonus: Click here to get access to a free “The Power of Python Decorators” guide that shows you 3 advanced decorator patterns and techniques you can use to write to cleaner and more Pythonic programs.

Decorators Cheat Sheet: Click here to get access to a free 3-page Python decorators cheat sheet that summarizes the techniques explained in this tutorial.

Decorators Q&A Transcript: Click here to get access to a 25-page chat log from our recent Python decorators Q&A session in the Real Python Community Slack where we discussed common decorator questions.

Updates:

  • 08/22/2018: Major update adding more examples and more advanced decorators
  • 01/12/2016: Updated examples to Python 3 (v3.5.1) syntax and added a new example
  • 11/01/2015: Added a brief explanation on the functools.wraps() decorator

Functions

Before you can understand decorators, you must first understand how functions work. For our purposes, a function returns a value based on the given arguments. Here is a very simple example:>>>

>>> def add_one(number):
...     return number + 1

>>> add_one(2)
3

In general, functions in Python may also have side effects rather than just turning an input into an output. The print() function is a basic example of this: it returns None while having the side effect of outputting something to the console. However, to understand decorators, it is enough to think about functions as something that turns given arguments into a value.

Note: In functional programming, you work (almost) only with pure functions without side effects. While not a purely functional language, Python supports many of the functional programming concepts, including functions as first-class objects.

First-Class Objects

In Python, functions are first-class objects. This means that functions can be passed around and used as arguments, just like any other object (string, int, float, list, and so on). Consider the following three functions:

def say_hello(name):
    return f"Hello {name}"

def be_awesome(name):
    return f"Yo {name}, together we are the awesomest!"

def greet_bob(greeter_func):
    return greeter_func("Bob")

Here, say_hello() and be_awesome() are regular functions that expect a name given as a string. The greet_bob() function however, expects a function as its argument. We can, for instance, pass it the say_hello() or the be_awesome() function:>>>

>>> greet_bob(say_hello)
'Hello Bob'

>>> greet_bob(be_awesome)
'Yo Bob, together we are the awesomest!'

Note that greet_bob(say_hello) refers to two functions, but in different ways: greet_bob() and say_hello. The say_hello function is named without parentheses. This means that only a reference to the function is passed. The function is not executed. The greet_bob() function, on the other hand, is written with parentheses, so it will be called as usual.

Inner Functions

It’s possible to define functions inside other functions. Such functions are called inner functions. Here’s an example of a function with two inner functions:

def parent():
    print("Printing from the parent() function")

    def first_child():
        print("Printing from the first_child() function")

    def second_child():
        print("Printing from the second_child() function")

    second_child()
    first_child()

What happens when you call the parent() function? Think about this for a minute. The output will be as follows:>>>

>>> parent()
Printing from the parent() function
Printing from the second_child() function
Printing from the first_child() function

Note that the order in which the inner functions are defined does not matter. Like with any other functions, the printing only happens when the inner functions are executed.

Furthermore, the inner functions are not defined until the parent function is called. They are locally scoped to parent(): they only exist inside the parent() function as local variables. Try calling first_child(). You should get an error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'first_child' is not defined

Whenever you call parent(), the inner functions first_child() and second_child() are also called. But because of their local scope, they aren’t available outside of the parent() function.

Returning Functions From Functions

Python also allows you to use functions as return values. The following example returns one of the inner functions from the outer parent() function:

def parent(num):
    def first_child():
        return "Hi, I am Emma"

    def second_child():
        return "Call me Liam"

    if num == 1:
        return first_child
    else:
        return second_child

Note that you are returning first_child without the parentheses. Recall that this means that you are returning a reference to the function first_child. In contrast first_child() with parentheses refers to the result of evaluating the function. This can be seen in the following example:>>>

>>> first = parent(1)
>>> second = parent(2)

>>> first
<function parent.<locals>.first_child at 0x7f599f1e2e18>

>>> second
<function parent.<locals>.second_child at 0x7f599dad5268>

The somewhat cryptic output simply means that the first variable refers to the local first_child() function inside of parent(), while second points to second_child().

You can now use first and second as if they are regular functions, even though the functions they point to can’t be accessed directly:>>>

>>> first()
'Hi, I am Emma'

>>> second()
'Call me Liam'

Finally, note that in the earlier example you executed the inner functions within the parent function, for instance first_child(). However, in this last example, you did not add parentheses to the inner functions—first_child—upon returning. That way, you got a reference to each function that you could call in the future. Make sense?

Simple Decorators

Now that you’ve seen that functions are just like any other object in Python, you’re ready to move on and see the magical beast that is the Python decorator. Let’s start with an example:

def my_decorator(func):
    def wrapper():
        print("Something is happening before the function is called.")
        func()
        print("Something is happening after the function is called.")
    return wrapper

def say_whee():
    print("Whee!")

say_whee = my_decorator(say_whee)

Can you guess what happens when you call say_whee()? Try it:>>>

>>> say_whee()
Something is happening before the function is called.
Whee!
Something is happening after the function is called.

To understand what’s going on here, look back at the previous examples. We are literally just applying everything you have learned so far.

The so-called decoration happens at the following line:

say_whee = my_decorator(say_whee)

In effect, the name say_whee now points to the wrapper() inner function. Remember that you return wrapper as a function when you call my_decorator(say_whee):>>>

>>> say_whee
<function my_decorator.<locals>.wrapper at 0x7f3c5dfd42f0>

However, wrapper() has a reference to the original say_whee() as func, and calls that function between the two calls to print().

Put simply: decorators wrap a function, modifying its behavior.

Before moving on, let’s have a look at a second example. Because wrapper() is a regular Python function, the way a decorator modifies a function can change dynamically. So as not to disturb your neighbors, the following example will only run the decorated code during the day:

from datetime import datetime

def not_during_the_night(func):
    def wrapper():
        if 7 <= datetime.now().hour < 22:
            func()
        else:
            pass  # Hush, the neighbors are asleep
    return wrapper

def say_whee():
    print("Whee!")

say_whee = not_during_the_night(say_whee)

If you try to call say_whee() after bedtime, nothing will happen:>>>

>>> say_whee()
>>>

Syntactic Sugar!

The way you decorated say_whee() above is a little clunky. First of all, you end up typing the name say_whee three times. In addition, the decoration gets a bit hidden away below the definition of the function.

Instead, Python allows you to use decorators in a simpler way with the @ symbol, sometimes called the “pie” syntax. The following example does the exact same thing as the first decorator example:

def my_decorator(func):
    def wrapper():
        print("Something is happening before the function is called.")
        func()
        print("Something is happening after the function is called.")
    return wrapper

@my_decorator
def say_whee():
    print("Whee!")

So, @my_decorator is just an easier way of saying say_whee = my_decorator(say_whee). It’s how you apply a decorator to a function.

Reusing Decorators

Recall that a decorator is just a regular Python function. All the usual tools for easy reusability are available. Let’s move the decorator to its own module that can be used in many other functions.

Create a file called decorators.py with the following content:

def do_twice(func):
    def wrapper_do_twice():
        func()
        func()
    return wrapper_do_twice

Note: You can name your inner function whatever you want, and a generic name like wrapper() is usually okay. You’ll see a lot of decorators in this article. To keep them apart, we’ll name the inner function with the same name as the decorator but with a wrapper_ prefix.

You can now use this new decorator in other files by doing a regular import:

from decorators import do_twice

@do_twice
def say_whee():
    print("Whee!")

When you run this example, you should see that the original say_whee() is executed twice:>>>

>>> say_whee()
Whee!
Whee!

Free Bonus: Click here to get access to a free “The Power of Python Decorators” guide that shows you 3 advanced decorator patterns and techniques you can use to write to cleaner and more Pythonic programs.

Decorating Functions With Arguments

Say that you have a function that accepts some arguments. Can you still decorate it? Let’s try:

from decorators import do_twice

@do_twice
def greet(name):
    print(f"Hello {name}")

Unfortunately, running this code raises an error:>>>

>>> greet("World")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: wrapper_do_twice() takes 0 positional arguments but 1 was given

The problem is that the inner function wrapper_do_twice() does not take any arguments, but name="World" was passed to it. You could fix this by letting wrapper_do_twice() accept one argument, but then it would not work for the say_whee() function you created earlier.

The solution is to use *args and **kwargs in the inner wrapper function. Then it will accept an arbitrary number of positional and keyword arguments. Rewrite decorators.py as follows:

def do_twice(func):
    def wrapper_do_twice(*args, **kwargs):
        func(*args, **kwargs)
        func(*args, **kwargs)
    return wrapper_do_twice

The wrapper_do_twice() inner function now accepts any number of arguments and passes them on to the function it decorates. Now both your say_whee() and greet() examples works:>>>

>>> say_whee()
Whee!
Whee!

>>> greet("World")
Hello World
Hello World

Returning Values From Decorated Functions

What happens to the return value of decorated functions? Well, that’s up to the decorator to decide. Let’s say you decorate a simple function as follows:

from decorators import do_twice

@do_twice
def return_greeting(name):
    print("Creating greeting")
    return f"Hi {name}"

Try to use it:>>>

>>> hi_adam = return_greeting("Adam")
Creating greeting
Creating greeting
>>> print(hi_adam)
None

Oops, your decorator ate the return value from the function.

Because the do_twice_wrapper() doesn’t explicitly return a value, the call return_greeting("Adam") ended up returning None.

To fix this, you need to make sure the wrapper function returns the return value of the decorated function. Change your decorators.py file:

def do_twice(func):
    def wrapper_do_twice(*args, **kwargs):
        func(*args, **kwargs)
        return func(*args, **kwargs)
    return wrapper_do_twice

The return value from the last execution of the function is returned:>>>

>>> return_greeting("Adam")
Creating greeting
Creating greeting
'Hi Adam'

Who Are You, Really?

A great convenience when working with Python, especially in the interactive shell, is its powerful introspection ability. Introspection is the ability of an object to know about its own attributes at runtime. For instance, a function knows its own name and documentation:>>>

>>> print
<built-in function print>

>>> print.__name__
'print'

>>> help(print)
Help on built-in function print in module builtins:

print(...)
    <full help message>

The introspection works for functions you define yourself as well:>>>

>>> say_whee
<function do_twice.<locals>.wrapper_do_twice at 0x7f43700e52f0>

>>> say_whee.__name__
'wrapper_do_twice'

>>> help(say_whee)
Help on function wrapper_do_twice in module decorators:

wrapper_do_twice()

However, after being decorated, say_whee() has gotten very confused about its identity. It now reports being the wrapper_do_twice() inner function inside the do_twice() decorator. Although technically true, this is not very useful information.

To fix this, decorators should use the @functools.wraps decorator, which will preserve information about the original function. Update decorators.py again:

import functools

def do_twice(func):
    @functools.wraps(func)
    def wrapper_do_twice(*args, **kwargs):
        func(*args, **kwargs)
        return func(*args, **kwargs)
    return wrapper_do_twice

You do not need to change anything about the decorated say_whee() function:>>>

>>> say_whee
<function say_whee at 0x7ff79a60f2f0>

>>> say_whee.__name__
'say_whee'

>>> help(say_whee)
Help on function say_whee in module whee:

say_whee()

Much better! Now say_whee() is still itself after decoration.

Technical Detail: The @functools.wraps decorator uses the function functools.update_wrapper() to update special attributes like __name__ and __doc__ that are used in the introspection.

A Few Real World Examples

Let’s look at a few more useful examples of decorators. You’ll notice that they’ll mainly follow the same pattern that you’ve learned so far:

import functools

def decorator(func):
    @functools.wraps(func)
    def wrapper_decorator(*args, **kwargs):
        # Do something before
        value = func(*args, **kwargs)
        # Do something after
        return value
    return wrapper_decorator

This formula is a good boilerplate template for building more complex decorators.

Note: In later examples, we will assume that these decorators are saved in your decorators.py file as well. Recall that you can download all the examples in this tutorial.

Timing Functions

Let’s start by creating a @timer decorator. It will measure the time a function takes to execute and print the duration to the console. Here’s the code:

import functools
import time

def timer(func):
    """Print the runtime of the decorated function"""
    @functools.wraps(func)
    def wrapper_timer(*args, **kwargs):
        start_time = time.perf_counter()    # 1
        value = func(*args, **kwargs)
        end_time = time.perf_counter()      # 2
        run_time = end_time - start_time    # 3
        print(f"Finished {func.__name__!r} in {run_time:.4f} secs")
        return value
    return wrapper_timer

@timer
def waste_some_time(num_times):
    for _ in range(num_times):
        sum([i**2 for i in range(10000)])

This decorator works by storing the time just before the function starts running (at the line marked # 1) and just after the function finishes (at # 2). The time the function takes is then the difference between the two (at # 3). We use the time.perf_counter() function, which does a good job of measuring time intervals. Here are some examples of timings:>>>

>>> waste_some_time(1)
Finished 'waste_some_time' in 0.0010 secs

>>> waste_some_time(999)
Finished 'waste_some_time' in 0.3260 secs

Run it yourself. Work through the code line by line. Make sure you understand how it works. Don’t worry if you don’t get it, though. Decorators are advanced beings. Try to sleep on it or make a drawing of the program flow.

Note: The @timer decorator is great if you just want to get an idea about the runtime of your functions. If you want to do more precise measurements of code, you should instead consider the timeit module in the standard library. It temporarily disables garbage collection and runs multiple trials to strip out noise from quick function calls.

Debugging Code

The following @debug decorator will print the arguments a function is called with as well as its return value every time the function is called:

import functools

def debug(func):
    """Print the function signature and return value"""
    @functools.wraps(func)
    def wrapper_debug(*args, **kwargs):
        args_repr = [repr(a) for a in args]                      # 1
        kwargs_repr = [f"{k}={v!r}" for k, v in kwargs.items()]  # 2
        signature = ", ".join(args_repr + kwargs_repr)           # 3
        print(f"Calling {func.__name__}({signature})")
        value = func(*args, **kwargs)
        print(f"{func.__name__!r} returned {value!r}")           # 4
        return value
    return wrapper_debug

The signature is created by joining the string representations of all the arguments. The numbers in the following list correspond to the numbered comments in the code:

  1. Create a list of the positional arguments. Use repr() to get a nice string representing each argument.
  2. Create a list of the keyword arguments. The f-string formats each argument as key=value where the !r specifier means that repr() is used to represent the value.
  3. The lists of positional and keyword arguments is joined together to one signature string with each argument separated by a comma.
  4. The return value is printed after the function is executed.

Let’s see how the decorator works in practice by applying it to a simple function with one position and one keyword argument:

@debug
def make_greeting(name, age=None):
    if age is None:
        return f"Howdy {name}!"
    else:
        return f"Whoa {name}! {age} already, you are growing up!"

Note how the @debug decorator prints the signature and return value of the make_greeting() function:>>>

>>> make_greeting("Benjamin")
Calling make_greeting('Benjamin')
'make_greeting' returned 'Howdy Benjamin!'
'Howdy Benjamin!'

>>> make_greeting("Richard", age=112)
Calling make_greeting('Richard', age=112)
'make_greeting' returned 'Whoa Richard! 112 already, you are growing up!'
'Whoa Richard! 112 already, you are growing up!'

>>> make_greeting(name="Dorrisile", age=116)
Calling make_greeting(name='Dorrisile', age=116)
'make_greeting' returned 'Whoa Dorrisile! 116 already, you are growing up!'
'Whoa Dorrisile! 116 already, you are growing up!'

This example might not seem immediately useful since the @debug decorator just repeats what you just wrote. It’s more powerful when applied to small convenience functions that you don’t call directly yourself.

The following example calculates an approximation to the mathematical constant e:

import math
from decorators import debug

# Apply a decorator to a standard library function
math.factorial = debug(math.factorial)

def approximate_e(terms=18):
    return sum(1 / math.factorial(n) for n in range(terms))

This example also shows how you can apply a decorator to a function that has already been defined. The approximation of e is based on the following series expansion:

Series for calculating mathematical constant e

When calling the approximate_e() function, you can see the @debug decorator at work:>>>

>>> approximate_e(5)
Calling factorial(0)
'factorial' returned 1
Calling factorial(1)
'factorial' returned 1
Calling factorial(2)
'factorial' returned 2
Calling factorial(3)
'factorial' returned 6
Calling factorial(4)
'factorial' returned 24
2.708333333333333

In this example, you get a decent approximation to the true value e = 2.718281828, adding only 5 terms.

Slowing Down Code

This next example might not seem very useful. Why would you want to slow down your Python code? Probably the most common use case is that you want to rate-limit a function that continuously checks whether a resource—like a web page—has changed. The @slow_down decorator will sleep one second before it calls the decorated function:

import functools
import time

def slow_down(func):
    """Sleep 1 second before calling the function"""
    @functools.wraps(func)
    def wrapper_slow_down(*args, **kwargs):
        time.sleep(1)
        return func(*args, **kwargs)
    return wrapper_slow_down

@slow_down
def countdown(from_number):
    if from_number < 1:
        print("Liftoff!")
    else:
        print(from_number)
        countdown(from_number - 1)

To see the effect of the @slow_down decorator, you really need to run the example yourself:>>>

>>> countdown(3)
3
2
1
Liftoff!

Note: The countdown() function is a recursive function. In other words, it’s a function calling itself. To learn more about recursive functions in Python, see our guide on Thinking Recursively in Python.

The @slow_down decorator always sleeps for one second. Later, you’ll see how to control the rate by passing an argument to the decorator.

Registering Plugins

Decorators don’t have to wrap the function they’re decorating. They can also simply register that a function exists and return it unwrapped. This can be used, for instance, to create a light-weight plug-in architecture:

import random
PLUGINS = dict()

def register(func):
    """Register a function as a plug-in"""
    PLUGINS[func.__name__] = func
    return func

@register
def say_hello(name):
    return f"Hello {name}"

@register
def be_awesome(name):
    return f"Yo {name}, together we are the awesomest!"

def randomly_greet(name):
    greeter, greeter_func = random.choice(list(PLUGINS.items()))
    print(f"Using {greeter!r}")
    return greeter_func(name)

The @register decorator simply stores a reference to the decorated function in the global PLUGINS dict. Note that you do not have to write an inner function or use @functools.wraps in this example because you are returning the original function unmodified.

The randomly_greet() function randomly chooses one of the registered functions to use. Note that the PLUGINS dictionary already contains references to each function object that is registered as a plugin:>>>

>>> PLUGINS
{'say_hello': <function say_hello at 0x7f768eae6730>,
 'be_awesome': <function be_awesome at 0x7f768eae67b8>}

>>> randomly_greet("Alice")
Using 'say_hello'
'Hello Alice'

The main benefit of this simple plugin architecture is that you do not need to maintain a list of which plugins exist. That list is created when the plugins register themselves. This makes it trivial to add a new plugin: just define the function and decorate it with @register.

If you are familiar with globals() in Python, you might see some similarities to how the plugin architecture works. globals() gives access to all global variables in the current scope, including your plugins:>>>

>>> globals()
{..., # Lots of variables not shown here.
 'say_hello': <function say_hello at 0x7f768eae6730>,
 'be_awesome': <function be_awesome at 0x7f768eae67b8>,
 'randomly_greet': <function randomly_greet at 0x7f768eae6840>}

Using the @register decorator, you can create your own curated list of interesting variables, effectively hand-picking some functions from globals().

Is the User Logged In?

The final example before moving on to some fancier decorators is commonly used when working with a web framework. In this example, we are using Flask to set up a /secret web page that should only be visible to users that are logged in or otherwise authenticated:

from flask import Flask, g, request, redirect, url_for
import functools
app = Flask(__name__)

def login_required(func):
    """Make sure user is logged in before proceeding"""
    @functools.wraps(func)
    def wrapper_login_required(*args, **kwargs):
        if g.user is None:
            return redirect(url_for("login", next=request.url))
        return func(*args, **kwargs)
    return wrapper_login_required

@app.route("/secret")
@login_required
def secret():
    ...

While this gives an idea about how to add authentication to your web framework, you should usually not write these types of decorators yourself. For Flask, you can use the Flask-Login extension instead, which adds more security and functionality.

Fancy Decorators

So far, you’ve seen how to create simple decorators. You already have a pretty good understanding of what decorators are and how they work. Feel free to take a break from this article to practice everything you’ve learned.

In the second part of this tutorial, we’ll explore more advanced features, including how to use the following:

Decorating Classes

There are two different ways you can use decorators on classes. The first one is very close to what you have already done with functions: you can decorate the methods of a class. This was one of the motivations for introducing decorators back in the day.

Some commonly used decorators that are even built-ins in Python are @classmethod@staticmethod, and @property. The @classmethod and @staticmethod decorators are used to define methods inside a class namespace that are not connected to a particular instance of that class. The @property decorator is used to customize getters and setters for class attributes. Expand the box below for an example using these decorators.

Example using built-in class decoratorsShow/Hide

Let’s define a class where we decorate some of its methods using the @debug and @timer decorators from earlier:

from decorators import debug, timer

class TimeWaster:
    @debug
    def __init__(self, max_num):
        self.max_num = max_num

    @timer
    def waste_time(self, num_times):
        for _ in range(num_times):
            sum([i**2 for i in range(self.max_num)])

Using this class, you can see the effect of the decorators:>>>

>>> tw = TimeWaster(1000)
Calling __init__(<time_waster.TimeWaster object at 0x7efccce03908>, 1000)
'__init__' returned None

>>> tw.waste_time(999)
Finished 'waste_time' in 0.3376 secs

The other way to use decorators on classes is to decorate the whole class. This is, for example, done in the new dataclasses module in Python 3.7:

from dataclasses import dataclass

@dataclass
class PlayingCard:
    rank: str
    suit: str

The meaning of the syntax is similar to the function decorators. In the example above, you could have done the decoration by writing PlayingCard = dataclass(PlayingCard).

common use of class decorators is to be a simpler alternative to some use-cases of metaclasses. In both cases, you are changing the definition of a class dynamically.

Writing a class decorator is very similar to writing a function decorator. The only difference is that the decorator will receive a class and not a function as an argument. In fact, all the decorators you saw above will work as class decorators. When you are using them on a class instead of a function, their effect might not be what you want. In the following example, the @timer decorator is applied to a class:

from decorators import timer

@timer
class TimeWaster:
    def __init__(self, max_num):
        self.max_num = max_num

    def waste_time(self, num_times):
        for _ in range(num_times):
            sum([i**2 for i in range(self.max_num)])

Decorating a class does not decorate its methods. Recall that @timer is just shorthand for TimeWaster = timer(TimeWaster).

Here, @timer only measures the time it takes to instantiate the class:>>>

>>> tw = TimeWaster(1000)
Finished 'TimeWaster' in 0.0000 secs

>>> tw.waste_time(999)
>>>

Later, you will see an example defining a proper class decorator, namely @singleton, which ensures that there is only one instance of a class.

Nesting Decorators

You can apply several decorators to a function by stacking them on top of each other:

from decorators import debug, do_twice

@debug
@do_twice
def greet(name):
    print(f"Hello {name}")

Think about this as the decorators being executed in the order they are listed. In other words, @debug calls @do_twice, which calls greet(), or debug(do_twice(greet())):>>>

>>> greet("Eva")
Calling greet('Eva')
Hello Eva
Hello Eva
'greet' returned None

Observe the difference if we change the order of @debug and @do_twice:

from decorators import debug, do_twice

@do_twice
@debug
def greet(name):
    print(f"Hello {name}")

In this case, @do_twice will be applied to @debug as well:>>>

>>> greet("Eva")
Calling greet('Eva')
Hello Eva
'greet' returned None
Calling greet('Eva')
Hello Eva
'greet' returned None

Decorators With Arguments

Sometimes, it’s useful to pass arguments to your decorators. For instance, @do_twice could be extended to a @repeat(num_times) decorator. The number of times to execute the decorated function could then be given as an argument.

This would allow you to do something like this:

@repeat(num_times=4)
def greet(name):
    print(f"Hello {name}")

>>>

>>> greet("World")
Hello World
Hello World
Hello World
Hello World

Think about how you could achieve this.

So far, the name written after the @ has referred to a function object that can be called with another function. To be consistent, you then need repeat(num_times=4) to return a function object that can act as a decorator. Luckily, you already know how to return functions! In general, you want something like the following:

def repeat(num_times):
    def decorator_repeat(func):
        ...  # Create and return a wrapper function
    return decorator_repeat

Typically, the decorator creates and returns an inner wrapper function, so writing the example out in full will give you an inner function within an inner function. While this might sound like the programming equivalent of the Inception movie, we’ll untangle it all in a moment:

def repeat(num_times):
    def decorator_repeat(func):
        @functools.wraps(func)
        def wrapper_repeat(*args, **kwargs):
            for _ in range(num_times):
                value = func(*args, **kwargs)
            return value
        return wrapper_repeat
    return decorator_repeat

It looks a little messy, but we have only put the same decorator pattern you have seen many times by now inside one additional def that handles the arguments to the decorator. Let’s start with the innermost function:

def wrapper_repeat(*args, **kwargs):
    for _ in range(num_times):
        value = func(*args, **kwargs)
    return value

This wrapper_repeat() function takes arbitrary arguments and returns the value of the decorated function, func(). This wrapper function also contains the loop that calls the decorated function num_times times. This is no different from the earlier wrapper functions you have seen, except that it is using the num_times parameter that must be supplied from the outside.

One step out, you’ll find the decorator function:

def decorator_repeat(func):
    @functools.wraps(func)
    def wrapper_repeat(*args, **kwargs):
        ...
    return wrapper_repeat

Again, decorator_repeat() looks exactly like the decorator functions you have written earlier, except that it’s named differently. That’s because we reserve the base name—repeat()—for the outermost function, which is the one the user will call.

As you have already seen, the outermost function returns a reference to the decorator function:

def repeat(num_times):
    def decorator_repeat(func):
        ...
    return decorator_repeat

There are a few subtle things happening in the repeat() function:

  • Defining decorator_repeat() as an inner function means that repeat() will refer to a function object—decorator_repeat. Earlier, we used repeat without parentheses to refer to the function object. The added parentheses are necessary when defining decorators that take arguments.
  • The num_times argument is seemingly not used in repeat() itself. But by passing num_times a closure is created where the value of num_times is stored until it will be used later by wrapper_repeat().

With everything set up, let’s see if the results are as expected:

@repeat(num_times=4)
def greet(name):
    print(f"Hello {name}")

>>>

>>> greet("World")
Hello World
Hello World
Hello World
Hello World

Just the result we were aiming for.

Both Please, But Never Mind the Bread

With a little bit of care, you can also define decorators that can be used both with and without arguments. Most likely, you don’t need this, but it is nice to have the flexibility.

As you saw in the previous section, when a decorator uses arguments, you need to add an extra outer function. The challenge is for your code to figure out if the decorator has been called with or without arguments.

Since the function to decorate is only passed in directly if the decorator is called without arguments, the function must be an optional argument. This means that the decorator arguments must all be specified by keyword. You can enforce this with the special * syntax, which means that all following parameters are keyword-only:

def name(_func=None, *, kw1=val1, kw2=val2, ...):  # 1
    def decorator_name(func):
        ...  # Create and return a wrapper function.

    if _func is None:
        return decorator_name                      # 2
    else:
        return decorator_name(_func)               # 3

Here, the _func argument acts as a marker, noting whether the decorator has been called with arguments or not:

  1. If name has been called without arguments, the decorated function will be passed in as _func. If it has been called with arguments, then _func will be None, and some of the keyword arguments may have been changed from their default values. The * in the argument list means that the remaining arguments can’t be called as positional arguments.
  2. In this case, the decorator was called with arguments. Return a decorator function that can read and return a function.
  3. In this case, the decorator was called without arguments. Apply the decorator to the function immediately.

Using this boilerplate on the @repeat decorator in the previous section, you can write the following:

def repeat(_func=None, *, num_times=2):
    def decorator_repeat(func):
        @functools.wraps(func)
        def wrapper_repeat(*args, **kwargs):
            for _ in range(num_times):
                value = func(*args, **kwargs)
            return value
        return wrapper_repeat

    if _func is None:
        return decorator_repeat
    else:
        return decorator_repeat(_func)

Compare this with the original @repeat. The only changes are the added _func parameter and the ifelse at the end.

Recipe 9.6 of the excellent Python Cookbook shows an alternative solution using functools.partial().

These examples show that @repeat can now be used with or without arguments:

@repeat
def say_whee():
    print("Whee!")

@repeat(num_times=3)
def greet(name):
    print(f"Hello {name}")

Recall that the default value of num_times is 2:>>>

>>> say_whee()
Whee!
Whee!

>>> greet("Penny")
Hello Penny
Hello Penny
Hello Penny

Stateful Decorators

Sometimes, it’s useful to have a decorator that can keep track of state. As a simple example, we will create a decorator that counts the number of times a function is called.

Note: In the beginning of this guide, we talked about pure functions returning a value based on given arguments. Stateful decorators are quite the opposite, where the return value will depend on the current state, as well as the given arguments.

In the next section, you will see how to use classes to keep state. But in simple cases, you can also get away with using function attributes:

import functools

def count_calls(func):
    @functools.wraps(func)
    def wrapper_count_calls(*args, **kwargs):
        wrapper_count_calls.num_calls += 1
        print(f"Call {wrapper_count_calls.num_calls} of {func.__name__!r}")
        return func(*args, **kwargs)
    wrapper_count_calls.num_calls = 0
    return wrapper_count_calls

@count_calls
def say_whee():
    print("Whee!")

The state—the number of calls to the function—is stored in the function attribute .num_calls on the wrapper function. Here is the effect of using it:>>>

>>> say_whee()
Call 1 of 'say_whee'
Whee!

>>> say_whee()
Call 2 of 'say_whee'
Whee!

>>> say_whee.num_calls
2

Classes as Decorators

The typical way to maintain state is by using classes. In this section, you’ll see how to rewrite the @count_calls example from the previous section using a class as a decorator.

Recall that the decorator syntax @my_decorator is just an easier way of saying func = my_decorator(func). Therefore, if my_decorator is a class, it needs to take func as an argument in its .__init__() method. Furthermore, the class needs to be callable so that it can stand in for the decorated function.

For a class to be callable, you implement the special .__call__() method:

class Counter:
    def __init__(self, start=0):
        self.count = start

    def __call__(self):
        self.count += 1
        print(f"Current count is {self.count}")

The .__call__() method is executed each time you try to call an instance of the class:>>>

>>> counter = Counter()
>>> counter()
Current count is 1

>>> counter()
Current count is 2

>>> counter.count
2

Therefore, a typical implementation of a decorator class needs to implement .__init__() and .__call__():

import functools

class CountCalls:
    def __init__(self, func):
        functools.update_wrapper(self, func)
        self.func = func
        self.num_calls = 0

    def __call__(self, *args, **kwargs):
        self.num_calls += 1
        print(f"Call {self.num_calls} of {self.func.__name__!r}")
        return self.func(*args, **kwargs)

@CountCalls
def say_whee():
    print("Whee!")

The .__init__() method must store a reference to the function and can do any other necessary initialization. The .__call__() method will be called instead of the decorated function. It does essentially the same thing as the wrapper() function in our earlier examples. Note that you need to use the functools.update_wrapper() function instead of @functools.wraps.

This @CountCalls decorator works the same as the one in the previous section:>>>

>>> say_whee()
Call 1 of 'say_whee'
Whee!

>>> say_whee()
Call 2 of 'say_whee'
Whee!

>>> say_whee.num_calls
2

More Real World Examples

We’ve come a far way now, having figured out how to create all kinds of decorators. Let’s wrap it up, putting our newfound knowledge into creating a few more examples that might actually be useful in the real world.

Slowing Down Code, Revisited

As noted earlier, our previous implementation of @slow_down always sleeps for one second. Now you know how to add parameters to decorators, so let’s rewrite @slow_down using an optional rate argument that controls how long it sleeps:

import functools
import time

def slow_down(_func=None, *, rate=1):
    """Sleep given amount of seconds before calling the function"""
    def decorator_slow_down(func):
        @functools.wraps(func)
        def wrapper_slow_down(*args, **kwargs):
            time.sleep(rate)
            return func(*args, **kwargs)
        return wrapper_slow_down

    if _func is None:
        return decorator_slow_down
    else:
        return decorator_slow_down(_func)

We’re using the boilerplate introduced in the Both Please, But Never Mind the Bread section to make @slow_down callable both with and without arguments. The same recursive countdown() function as earlier now sleeps two seconds between each count:

@slow_down(rate=2)
def countdown(from_number):
    if from_number < 1:
        print("Liftoff!")
    else:
        print(from_number)
        countdown(from_number - 1)

As before, you must run the example yourself to see the effect of the decorator:>>>

>>> countdown(3)
3
2
1
Liftoff!

Creating Singletons

A singleton is a class with only one instance. There are several singletons in Python that you use frequently, including NoneTrue, and False. It is the fact that None is a singleton that allows you to compare for None using the is keyword, like you saw in the Both Please section:

if _func is None:
    return decorator_name
else:
    return decorator_name(_func)

Using is returns True only for objects that are the exact same instance. The following @singleton decorator turns a class into a singleton by storing the first instance of the class as an attribute. Later attempts at creating an instance simply return the stored instance:

import functools

def singleton(cls):
    """Make a class a Singleton class (only one instance)"""
    @functools.wraps(cls)
    def wrapper_singleton(*args, **kwargs):
        if not wrapper_singleton.instance:
            wrapper_singleton.instance = cls(*args, **kwargs)
        return wrapper_singleton.instance
    wrapper_singleton.instance = None
    return wrapper_singleton

@singleton
class TheOne:
    pass

As you see, this class decorator follows the same template as our function decorators. The only difference is that we are using cls instead of func as the parameter name to indicate that it is meant to be a class decorator.

Let’s see if it works:>>>

>>> first_one = TheOne()
>>> another_one = TheOne()

>>> id(first_one)
140094218762280

>>> id(another_one)
140094218762280

>>> first_one is another_one
True

It seems clear that first_one is indeed the exact same instance as another_one.

Note: Singleton classes are not really used as often in Python as in other languages. The effect of a singleton is usually better implemented as a global variable in a module.

Caching Return Values

Decorators can provide a nice mechanism for caching and memoization. As an example, let’s look at a recursive definition of the Fibonacci sequence:

from decorators import count_calls

@count_calls
def fibonacci(num):
    if num < 2:
        return num
    return fibonacci(num - 1) + fibonacci(num - 2)

While the implementation is simple, its runtime performance is terrible:>>>

>>> fibonacci(10)
<Lots of output from count_calls>
55

>>> fibonacci.num_calls
177

To calculate the tenth Fibonacci number, you should really only need to calculate the preceding Fibonacci numbers, but this implementation somehow needs a whopping 177 calculations. It gets worse quickly: 21891 calculations are needed for fibonacci(20) and almost 2.7 million calculations for the 30th number. This is because the code keeps recalculating Fibonacci numbers that are already known.

The usual solution is to implement Fibonacci numbers using a for loop and a lookup table. However, simple caching of the calculations will also do the trick:

import functools
from decorators import count_calls

def cache(func):
    """Keep a cache of previous function calls"""
    @functools.wraps(func)
    def wrapper_cache(*args, **kwargs):
        cache_key = args + tuple(kwargs.items())
        if cache_key not in wrapper_cache.cache:
            wrapper_cache.cache[cache_key] = func(*args, **kwargs)
        return wrapper_cache.cache[cache_key]
    wrapper_cache.cache = dict()
    return wrapper_cache

@cache
@count_calls
def fibonacci(num):
    if num < 2:
        return num
    return fibonacci(num - 1) + fibonacci(num - 2)

The cache works as a lookup table, so now fibonacci() only does the necessary calculations once:>>>

>>> fibonacci(10)
Call 1 of 'fibonacci'
...
Call 11 of 'fibonacci'
55

>>> fibonacci(8)
21

Note that in the final call to fibonacci(8), no new calculations were needed, since the eighth Fibonacci number had already been calculated for fibonacci(10).

In the standard library, a Least Recently Used (LRU) cache is available as @functools.lru_cache.

This decorator has more features than the one you saw above. You should use @functools.lru_cache instead of writing your own cache decorator:

import functools

@functools.lru_cache(maxsize=4)
def fibonacci(num):
    print(f"Calculating fibonacci({num})")
    if num < 2:
        return num
    return fibonacci(num - 1) + fibonacci(num - 2)

The maxsize parameter specifies how many recent calls are cached. The default value is 128, but you can specify maxsize=None to cache all function calls. However, be aware that this can cause memory problems if you are caching many large objects.

You can use the .cache_info() method to see how the cache performs, and you can tune it if needed. In our example, we used an artificially small maxsize to see the effect of elements being removed from the cache:>>>

>>> fibonacci(10)
Calculating fibonacci(10)
Calculating fibonacci(9)
Calculating fibonacci(8)
Calculating fibonacci(7)
Calculating fibonacci(6)
Calculating fibonacci(5)
Calculating fibonacci(4)
Calculating fibonacci(3)
Calculating fibonacci(2)
Calculating fibonacci(1)
Calculating fibonacci(0)
55

>>> fibonacci(8)
21

>>> fibonacci(5)
Calculating fibonacci(5)
Calculating fibonacci(4)
Calculating fibonacci(3)
Calculating fibonacci(2)
Calculating fibonacci(1)
Calculating fibonacci(0)
5

>>> fibonacci(8)
Calculating fibonacci(8)
Calculating fibonacci(7)
Calculating fibonacci(6)
21

>>> fibonacci(5)
5

>>> fibonacci.cache_info()
CacheInfo(hits=17, misses=20, maxsize=4, currsize=4)

Adding Information About Units

The following example is somewhat similar to the Registering Plugins example from earlier, in that it does not really change the behavior of the decorated function. Instead, it simply adds unit as a function attribute:

def set_unit(unit):
    """Register a unit on a function"""
    def decorator_set_unit(func):
        func.unit = unit
        return func
    return decorator_set_unit

The following example calculates the volume of a cylinder based on its radius and height in centimeters:

import math

@set_unit("cm^3")
def volume(radius, height):
    return math.pi * radius**2 * height

This .unit function attribute can later be accessed when needed:>>>

>>> volume(3, 5)
141.3716694115407

>>> volume.unit
'cm^3'

Note that you could have achieved something similar using function annotations:

import math

def volume(radius, height) -> "cm^3":
    return math.pi * radius**2 * height

However, since annotations are used for type hints, it would be hard to combine such units as annotations with static type checking.

Units become even more powerful and fun when connected with a library that can convert between units. One such library is pint. With pint installed (pip install Pint), you can for instance convert the volume to cubic inches or gallons:>>>

>>> import pint
>>> ureg = pint.UnitRegistry()
>>> vol = volume(3, 5) * ureg(volume.unit)

>>> vol
<Quantity(141.3716694115407, 'centimeter ** 3')>

>>> vol.to("cubic inches")
<Quantity(8.627028576414954, 'inch ** 3')>

>>> vol.to("gallons").m  # Magnitude
0.0373464440537444

You could also modify the decorator to return a pint Quantity directly. Such a Quantity is made by multiplying a value with the unit. In pint, units must be looked up in a UnitRegistry. The registry is stored as a function attribute to avoid cluttering the namespace:

def use_unit(unit):
    """Have a function return a Quantity with given unit"""
    use_unit.ureg = pint.UnitRegistry()
    def decorator_use_unit(func):
        @functools.wraps(func)
        def wrapper_use_unit(*args, **kwargs):
            value = func(*args, **kwargs)
            return value * use_unit.ureg(unit)
        return wrapper_use_unit
    return decorator_use_unit

@use_unit("meters per second")
def average_speed(distance, duration):
    return distance / duration

With the @use_unit decorator, converting units is practically effortless:>>>

>>> bolt = average_speed(100, 9.58)
>>> bolt
<Quantity(10.438413361169102, 'meter / second')>

>>> bolt.to("km per hour")
<Quantity(37.578288100208766, 'kilometer / hour')>

>>> bolt.to("mph").m  # Magnitude
23.350065679064745

Validating JSON

Let’s look at one last use case. Take a quick look at the following Flask route handler:

@app.route("/grade", methods=["POST"])
def update_grade():
    json_data = request.get_json()
    if "student_id" not in json_data:
        abort(400)
    # Update database
    return "success!"

Here we ensure that the key student_id is part of the request. Although this validation works, it really does not belong in the function itself. Plus, perhaps there are other routes that use the exact same validation. So, let’s keep it DRY and abstract out any unnecessary logic with a decorator. The following @validate_json decorator will do the job:

from flask import Flask, request, abort
import functools
app = Flask(__name__)

def validate_json(*expected_args):                  # 1
    def decorator_validate_json(func):
        @functools.wraps(func)
        def wrapper_validate_json(*args, **kwargs):
            json_object = request.get_json()
            for expected_arg in expected_args:      # 2
                if expected_arg not in json_object:
                    abort(400)
            return func(*args, **kwargs)
        return wrapper_validate_json
    return decorator_validate_json

In the above code, the decorator takes a variable length list as an argument so that we can pass in as many string arguments as necessary, each representing a key used to validate the JSON data:

  1. The list of keys that must be present in the JSON is given as arguments to the decorator.
  2. The wrapper function validates that each expected key is present in the JSON data.

The route handler can then focus on its real job—updating grades—as it can safely assume that JSON data are valid:

@app.route("/grade", methods=["POST"])
@validate_json("student_id")
def update_grade():
    json_data = request.get_json()
    # Update database.
    return "success!"

Conclusion

This has been quite a journey! You started this tutorial by looking a little closer at functions, particularly how they can be defined inside other functions and passed around just like any other Python object. Then you learned about decorators and how to write them such that:

  • They can be reused.
  • They can decorate functions with arguments and return values.
  • They can use @functools.wraps to look more like the decorated function.

In the second part of the tutorial, you saw more advanced decorators and learned how to:

  • Decorate classes
  • Nest decorators
  • Add arguments to decorators
  • Keep state within decorators
  • Use classes as decorators

You saw that, to define a decorator, you typically define a function returning a wrapper function. The wrapper function uses *args and **kwargs to pass on arguments to the decorated function. If you want your decorator to also take arguments, you need to nest the wrapper function inside another function. In this case, you usually end up with three return statements.

You can find the code from this tutorial online.

Further Reading

If you are still looking for more, our book Python Tricks has a section on decorators, as does the Python Cookbook by David Beazley and Brian K. Jones.

For a deep dive into the historical discussion on how decorators should be implemented in Python, see PEP 318 as well as the Python Decorator Wiki. More examples of decorators can be found in the Python Decorator Library. The decorator module can simplify creating your own decorators, and its documentation contains further decorator examples.

Also, we’ve put together a short & sweet Python decorators cheat sheet for you

 https://www.programiz.com/python-programming/decorator 
 https://dbader.org/blog/python-decorators 
 https://medium.com/@henslejoseph/understanding-decorators-in-python-9bc55b16b5c8 
 https://realpython.com/primer-on-python-decorators/ 
Amir Masoud Sefidian
Amir Masoud Sefidian
Data Scientist, Researcher, Software Developer

Leave a Reply

Your email address will not be published. Required fields are marked *