Catching every single exception with Python

When it comes to exceptions, we all know that sticky situations can be handled by simply looking before you leap, and for others, the good ol'try/except does the trick. But, what will come of an exception you've missed? If you're writing in any modern OOP language, then it's likely that your program will print out some traceback data to STDERR immediately before closing. That's it.

In a production environment, this is not acceptable. Failure simply isn't an option.

Now, what if I told you there was another way? What if you could catch an exception you never explicitly tried to catch, and do something graceful before your program's demise?

Allow me to introduce sys's excepthook!

sys.excepthook(type, value, tb)

The interpreter calls this function immediately before a program crashes as the result of an unhandled exception.

So, what we can do with this? We can create a function that accepts the same arguments and assign it to sys.excepthook so that a call to sys.excepthook is actually a call to our own function.

Example:

import sys

def my_exception_hook(type, value, tb):
    """
    Intended to be assigned to sys.exception as a hook.
    Gives programmer opportunity to do something useful with info from uncaught exceptions.

    Parameters
    type: Exception type
    value: Exception's value
    tb: Exception's traceback
    """
    error_msg = "An exception has been raised outside of a try/except!!!\n" \
                f"Type: {type}\n" \
                f"Value: {value}\n" \
                f"Traceback: {tb}"
    # We're just printing the error out for this example, 
    # but you should do something useful here!
    # Log it! Email your boss! Tell your cat!
    print(error_msg)

sys.excepthook = my_exception_hook

# Let's do something silly now
x = 10 + "10"

I saved this in a file called "testfile.py". Let's run that and see what we get.

josh@debian:~/tmp$ python3 testfile.py
An exception has been raised outside of a try/except!!!
Type: <class 'TypeError'>
Value: unsupported operand type(s) for +: 'int' and 'str'
Traceback: <traceback object at 0x7fac25f231c8>

This is great. We have some details about our exception that we can log and/or alert the developers with. We know that not only was an exception raised without a try/except, but we also know it was a TypeError due to a + being used to perform addition/string concatenation (depending on the intentions of the developer). With that said, it would be nice to know more about the error, and we can learn more by using information extracted from the traceback argument, tb.

Extracting details from the traceback

To extract information from our traceback, we'll be importing Python's traceback module. Once we've imported that, we'll be using the extract_tb() function to extract information from our traceback. For the purposes of this article, we'll only be concerning ourselves with passing extract_tb() our traceback, but know that you can limit how much information is returned using the limit parameter.

extract_tb() returns an instance of traceback.StackSummary which carries all of the information about the traceback. Luckily, this class has a format() method that returns to us a list of string, each representing a frame from the stack.

Now, let's rewrite our function above to give us more information about our exception by extracting the details from the traceback object, our tb parameter.

import sys
import traceback

def my_exception_hook(type, value, tb):
    """
    Intended to be assigned to sys.exception as a hook.
    Gives programmer opportunity to do something useful with info from uncaught exceptions.

    Parameters
    type: Exception type
    value: Exception's value
    tb: Exception's traceback
    """

    # NOTE: because format() is returning a list of string,
    # I'm going to join them into a single string, separating each with a new line
    traceback_details = '\n'.join(traceback.extract_tb(tb).format())

    error_msg = "An exception has been raised outside of a try/except!!!\n" \
                f"Type: {type}\n" \
                f"Value: {value}\n" \
                f"Traceback: {traceback_details}"
    print(error_msg)

sys.excepthook = my_exception_hook

# Let's do something silly now
x = 10 + "10"

Let's run it and see what we get back!

josh@debian:~/tmp$ python3 testfile.py
An exception has been raised outside of a try/except!!!
Type: <class 'TypeError'>
Value: unsupported operand type(s) for +: 'int' and 'str'
Traceback:   File "testfile.py", line 19, in <module>
    x = 10 + "10"

Great! We can see the file this happened in, the line it happened on, and a glimpse of the faulty logic that caused the exception!

I'm not doing much with the exception's data here, but it's just an example. I'd like to encourage you to log this exception somewhere. Also, consider settings aside a special exit code for these types of exceptions.

In summation...

Make no mistake about this post - I am NOT suggesting you substitute your proper try/except blocks with this method. This method is what I would use as an emergency back up. If my code ends up here somehow, I still consider it a loss, but at least I may be able to learn about it this way, rather than losing the exception details to whatever (if anything) is watching STDERR.