Error Handling and Logging in Python

Writing software is an activity far from perfect. From ideation to production, errors can appear, and in some cases, failure can occur deliberately. This is why understanding error handling and logging in your primary programming language is a critical skill to master.

Errors can happen, and situations can arise, but how you respond—with preparation and information on the error—will get you out of the situation as quickly as possible.

In this article, we will learn about error handling and logging in Python. We will primarily explore exceptions and how to use Python’s logging package to write various types of logs.

If you are interested in more content covering topics like this, subscribe to my newsletter for regular updates on software programming, architecture, and tech-related insights.

Exceptions in Python

As in many other programming languages, Python has the capability to raise exceptions when errors occur. In programming, an exception is an event that occurs during the execution of a program, disrupting the normal flow of instructions.

In Python, exceptions are errors detected during execution. When an exception occurs, Python stops running the code and looks for a special block of code (a try/except block) to handle the error.

Here are some common exceptions that can occur in a Python program:

• ZeroDivisionError: Occurs when attempting to divide a number by zero.

• FileNotFoundError: Occurs when trying to open a file that doesn't exist.

• ValueError: Occurs when trying to convert a string into an integer when the string does not represent a number.

• IndexError: Occurs when trying to retrieve an element from a list with a non-existing index.

There are many more exceptions, and Python gives you the ability to create your own exceptions if you need custom behavior. This is a feature we will explore later in the article.

To handle Python exceptions, you need to catch them. Catching exceptions requires a simple syntax known as try/except. Let's explore this.

Try/Except

The try/except block is used to handle exceptions. Code that might raise an exception is placed in the try block, and if an exception occurs, the except block is executed. Here is the syntax of try/except in a code block:

try:
    # Code that might raise an exception
    pass
except ExceptionType as e:
    # Code to handle the exception
    pass

The code that could potentially fail is put inside the try block. If an issue arises, the program’s execution will enter the except block.

Here is a flowchart that illustrates how try/except works:

Let’s see how we can handle a division by zero with this approach:

# Handling division by zero
try:
    result = 10 / 0
except ZeroDivisionError:
    print("Error: Cannot divide by zero.")
# The code will continue its execution

There are also additional blocks in the try/except syntax, such as else and finally:

try:
    # Code that might raise an exception
    pass
except ExceptionType as e:
    # Code to handle the exception
    pass
else:
    # Code to run if no exception is raised
    pass
finally:
    # Code that always runs, regardless of whether an exception was raised or not
    pass

These blocks are optional but serve specific purposes:

• else Block (Optional): Contains code that runs if no exceptions are raised in the try block. It is useful for code that should only run when the try block is successful.

• finally Block (Optional): Contains code that always runs, regardless of whether an exception was raised or not. This is typically used for cleanup actions, such as closing files or releasing resources.

Here is an example where we handle the closing of a file in finally in case of an error:

try:
    # Open the file
    file = open('example.txt', 'r')

    # Read from the file
    content = file.read()

    # Print file content (this will only execute if no exceptions are raised)
    print(content)
except FileNotFoundError as e:
    # Handle the specific exception
    print(f"Error: {e}")
except Exception as e:
    # Handle any other exceptions
    print(f"An unexpected error occurred: {e}")
else:
    # Code that runs if no exception was raised in the try block
    print("File read successfully.")
finally:
    # Ensure the file is closed, regardless of whether an exception was raised
    try:
        file.close()
        print("File closed.")
    except:
        # Handle the case where file was never opened (e.g., if open() failed)
        print("File was not opened or already closed.")

Disclaimer: The example above demonstrates file handling using try/except/finally to ensure the file is properly closed even if an error occurs. However, this approach is not ideal for everyday file operations. In practice, it is recommended to use the with statement for file handling in Python. The with statement automatically manages file opening and closing, ensuring that the file is properly closed after its suite finishes, even if an exception occurs.

This is how the try/except works. Now, there might be some confusion with if/else. When should you use try/except, and when should you use if/else?

What’s the difference between try/except and if/else? Use if/else when you want to check conditions that you can predict and handle before they cause errors, and use try/except to catch and manage exceptions that occur during code execution, particularly for errors you can’t easily anticipate.

In the case below, if/else won’t work properly:

filename = 'non_existent_file.txt'

if filename:  # This only checks if filename is not empty, not if the file exists
    # The following line will raise an exception if the file doesn't exist
    content = open(filename, 'r').read()  # This will crash if the file does not exist
    if content:
        print("File content exists:")
        print(content)
    else:
        print("File is empty.")
else:
    print("Filename is invalid.")

Here is a better solution with try/except:

filename = 'non_existent_file.txt'

try:
    content = open(filename, 'r').read()
    if content:
        print("File content exists:")
        print(content)
    else:
        print("File is empty.")
except FileNotFoundError:
    print("Error: File not found.")

In the solution above, the code attempts to open and read a file, checking if its content exists and printing it if present. If the file does not exist, it catches the FileNotFoundError and prints an error message, preventing the program from crashing.

As mentioned earlier in the article, Python allows for custom exceptions. Let’s learn more about it.

Creating Custom Exceptions in Python

In Python, you can define your own exceptions to handle specific error conditions in a more granular way. Custom exceptions are particularly useful in complex applications, such as fintech, where you may need to enforce business rules or handle specific error cases uniquely.

For example, in a fintech application, you might have a scenario where a wallet’s balance is checked against certain criteria. You may want to raise an exception if a wallet’s balance is not sufficient or does not conform to specific rules. Here’s how you can create and use a custom exception for this purpose:

# Define a custom exception
class WalletBalanceError(Exception):
    def __init__(self, message):
        self.message = message
        super().__init__(self.message)

# Function that checks wallet balance
def check_wallet_balance(wallet_balance, required_balance):
    if wallet_balance < required_balance:
        # Raise the custom exception with a specific message
        raise WalletBalanceError(f"Insufficient balance: Wallet balance of {wallet_balance} is less than the required {required_balance}.")

# Example usage
try:
    # Example wallet balance and required balance
    wallet_balance = 50
    required_balance = 100

    # Check if the wallet balance is sufficient
    check_wallet_balance(wallet_balance, required_balance)
except WalletBalanceError as e:
    # Handle the custom exception
    print(f"Error: {e}")

In this example, we define a custom exception WalletBalanceError to handle cases where a wallet’s balance does not meet the required criteria. The check_wallet_balance function raises this exception if the wallet’s balance is insufficient, providing a clear and specific error message.

Custom exceptions in Python help make the code more readable and maintainable by clearly defining specific error conditions and handling them in a structured manner.

Now that we know how to handle errors in Python, it’s time to understand what to do when these errors occur. There are many strategies, but keeping a log of these errors can help identify issues later and correct them. In the next section of this article, we will explore logging.

Logging in Python

Logging helps developers track errors, events, or any runtime information in an application or program. Logging is an important and crucial aspect of software engineering as it has the ability to record everything that goes right or wrong in a post-development application. Logging is one of the most important pillars of monitoring.

Python provides a built-in module that can be used for logging

purposes. To use this module, the first thing to do is to import it.

import logging

Then, configure the logger using the basicConfig method. You need to pass parameters to it, such as the log level, the format of the message, and the output file to save the log.

import logging

# Set up the basic configuration for logging
logging.basicConfig(filename='app.log', level=logging.DEBUG,
                    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# Log messages of various severity levels
logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

In the example above, logs will be written to a file called app.log. The log message format includes the timestamp, logger name, log level, and the actual message.

Python logging has different log levels that indicate the severity of an event or message. These log levels allow you to categorize and filter messages based on their importance. Here’s a breakdown of the common log levels in Python:

Log Levels

1. DEBUG: Detailed information, typically of interest only when diagnosing problems. Used for debugging purposes during development.

2. INFO: Confirmation that things are working as expected. This is the level you would use for normal operations and informational messages.

3. WARNING: An indication that something unexpected happened, or indicative of some problem in the near future (e.g., "disk space low"). The software is still working as expected.

4. ERROR: Due to a more serious problem, the software has not been able to perform some function. An error indicates a significant issue that needs attention.

5. CRITICAL: A very serious error, indicating that the program itself may be unable to continue running. Critical errors often represent severe problems that require immediate action.

The logging module allows you to control which messages are recorded by setting the logging level. Only messages that are equal to or more severe than the set level will be logged. The default level is WARNING, meaning only WARNING, ERROR, and CRITICAL messages are logged unless you change the logging configuration.

In the code example above, we set the logging level to DEBUG, which means all log messages (DEBUG, INFO, WARNING, ERROR, and CRITICAL) will be recorded in the app.log file.

You can also create custom loggers, which give you more control over how messages are logged. Custom loggers allow you to set up multiple loggers with different configurations, such as different log levels, formats, or output destinations. This is particularly useful in larger applications where you need to separate logs for different modules or components.

Here’s how you can create and use a custom logger:

import logging

# Create a custom logger
logger = logging.getLogger('my_custom_logger')

# Set the log level for the custom logger
logger.setLevel(logging.DEBUG)

# Create a file handler to write logs to a file
file_handler = logging.FileHandler('custom.log')

# Create a console handler to output logs to the console
console_handler = logging.StreamHandler()

# Set log levels for the handlers
file_handler.setLevel(logging.ERROR)
console_handler.setLevel(logging.DEBUG)

# Create a formatter for log messages
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# Add the formatter to the handlers
file_handler.setFormatter(formatter)
console_handler.setFormatter(formatter)

# Add the handlers to the logger
logger.addHandler(file_handler)
logger.addHandler(console_handler)

# Log messages using the custom logger
logger.debug('This is a debug message')
logger.info('This is an info message')
logger.warning('This is a warning message')
logger.error('This is an error message')
logger.critical('This is a critical message')

In this example, we create a custom logger named my_custom_logger. This logger writes ERROR and more severe messages to a file called custom.log, while DEBUG and more severe messages are output to the console. By customizing the loggers, you can tailor the logging behavior to fit the specific needs of your application.

Real-world Example: Logging in a Web Application

In a web application, logging plays a critical role in monitoring and maintaining the system’s health. For example, in a Flask web application, you might use logging to track incoming requests, errors, and performance metrics.

Here’s a basic example of how you can use logging in a Flask application:

from flask import Flask, request
import logging

app = Flask(__name__)

# Set up the basic configuration for logging
logging.basicConfig(filename='webapp.log', level=logging.INFO,
                    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')

@app.route('/')
def index():
    app.logger.info('Index page accessed')
    return 'Welcome to the Flask Web Application!'

@app.route('/error')
def error():
    app.logger.error('Error page accessed')
    raise ValueError('This is a simulated error')

if __name__ == '__main__':
    app.run(debug=True)

In this Flask application, we configure logging to write logs to a file named webapp.log. Each time the index page is accessed, an informational log message is recorded. If the error page is accessed, an error log message is recorded, and a simulated error is raised.

By implementing logging in your web application, you can gain insights into user activity, system errors, and performance issues. This information is invaluable for debugging, troubleshooting, and optimizing the application.

Conclusion

Error handling and logging are essential aspects of software development, ensuring that applications run smoothly and that any issues are quickly identified and resolved.

In this article, we explored exceptions in Python, including how to handle them using try/except, and the importance of logging for tracking errors and events. We also discussed how to create custom exceptions and custom loggers to suit specific application needs.

By mastering error handling and logging, you’ll be better equipped to build robust and maintainable software that can gracefully handle unexpected situations and provide valuable insights into its operation.

If you enjoyed this article, consider subscribing to my newsletter so you don't miss out on future updates.

Your feedback is valuable! If you have any suggestions, critiques, or questions, please leave a comment below.