Python Asyncio: A Guide to Asynchronous Programming and Concurrency

Table of Contents

1. What is Concurrency?
2. Writing Concurrent Code in Python
3. Libraries for Asynchronous Programming
4. Getting Started with asyncio
   • Asynchronous Programming in Action
     • Synchronous Code
     • Asynchronous Code
     • Walkthrough of Asynchronous Code
5. Most Popular asyncio Functions
6. Other Relevant Information
   • Debug Mode
7. Conclusion

Python, known for its synchronous code execution, processes tasks line by line, causing delays when tasks take time to run. In a previous article, we discussed parallelization, the Global Interpreter Lock (GIL), and its impact on parallel execution.

To overcome this, we turn to concurrency.

What is Concurrency?

Concurrency involves running multiple tasks simultaneously by efficiently managing their wait times.

Writing Concurrent Code in Python

To achieve concurrency in Python, we turn to asynchronous programming, briefly discussed here.

Libraries for Asynchronous Programming

Three notable libraries for asynchronous programming are:

1. asyncio
2. anyio
3. aiohttp

This article focuses on asyncio, the foundation for anyio and aiohttp.

Getting Started with asyncio

asyncio enables writing concurrent code using the async/await syntax. It serves as the base for various Python asynchronous frameworks, offering high-performance solutions for network and web servers, database connections, distributed task queues, and more.

asyncio is particularly suitable for IO-bound and high-level structured network code. Its high-level APIs allow:

• Running Python coroutines concurrently
• Performing network IO and IPC
• Controlling subprocesses
• Distributing tasks via queues
• Synchronizing concurrent code

Low-level APIs are also available for creating and managing event loops, implementing efficient protocols, and bridging callback-based libraries with async/await syntax.

Asynchronous Programming in Action

Synchronous Code

#sync.py
import time

def print_message(message, delay):
    time.sleep(delay)
    print(message)

def main():
    print_message("Hello", 2)
    print_message("Sync", 1)
    print_message("World", 3)

if __name__ == "__main__":
    start_time = time.time()
    main()
    end_time = time.time()
    print(f"Total execution time: {end_time - start_time} seconds")

Asynchronous Code

#async.py
import asyncio
import time

# Define a simple asynchronous function
async def print_message(message, delay):
    # Simulate some asynchronous operation, like I/O or network request
    await asyncio.sleep(delay)
    print(message)

# Define the main asynchronous function
async def main():
    # Use asyncio.gather to run multiple asynchronous functions concurrently
    # Wait for each task to complete in order
    await asyncio.gather(
        print_message("Async", 2),
        print_message("Hello", 1),
        print_message("World", 3)
    )

# Run the event loop to execute the asynchronous code
if __name__ == "__main__":
    # Record the start time for measuring execution time
    start_time = time.time()

    # Run the main asynchronous function using asyncio.run()
    asyncio.run(main())

    # Calculate and print the total execution time
    end_time = time.time()
    print(f"Total execution time: {end_time - start_time} seconds")

Walkthrough of Asynchronous Code:

1. Import the required libraries: asyncio and time.
2. Define an asynchronous function print_message(message, delay). This function simulates asynchronous operations using asyncio.sleep(delay) and prints the given message.
3. Define the main asynchronous function main(). It uses asyncio.gather to concurrently run multiple instances of the print_message function with different messages and delays.
4. Check if the script is being run directly using if __name__ == "__main__":.
5. Record the start time using start_time = time.time().
6. Run the main asynchronous function using asyncio.run(main()).
7. Record the end time using end_time = time.time().
8. Print the total execution time: print(f"Total execution time: {end_time - start_time} seconds").

Most Popular asyncio Functions

1. run(): Create an event loop, run a coroutine, and close the loop.
2. Runner: A context manager simplifying multiple async function calls.
3. Task: Task object.
4. TaskGroup: A context manager holding a group of tasks, providing a way to wait for all to finish.
5. create_task(): Start an asyncio Task and return it.
6. current_task(): Return the current Task.
7. all_tasks(): Return all unfinished tasks for an event loop.
8. await sleep(): Sleep for a number of seconds.
9. await gather(): Schedule and wait for things concurrently.
10. await wait_for(): Run with a timeout.
11. await shield(): Shield from cancellation.
12. await wait(): Monitor for completion.
13. timeout(): Run with a timeout.
14. to_thread(): Asynchronously run a function in a separate OS thread.
15. run_coroutine_threadsafe(): Schedule a coroutine from another OS thread.
16. for in as_completed(): Monitor for completion with a for loop.

Other Relevant Information

• Debug Mode:
  • Enable by setting PYTHONASYNCIODEBUG=1, using Python Development Mode, passing debug=True to asyncio.run(), or calling loop.set_debug().
  • Configure the logger level to logging.DEBUG.
  • Display ResourceWarning warnings using -W default command line option.

With debug mode:

• asyncio checks for unawaited coroutines.
• Non-threadsafe asyncio APIs raise exceptions if called from the wrong thread.
• I/O selector execution time is logged if it takes too long.
• Callbacks longer than 100 milliseconds are logged.

Conclusion

Incorporate asyncio into your Python projects for efficient concurrent and asynchronous programming. Explore its rich set of functions and keep in mind the debugging options for smoother development.

I will be writing an article on anyio. The asynchronous programming library used by fastapi and starlette to handle requests and other tasks.

In the mean time, follow me, jump to comments, and leabe positive reactions.

Happy asynchronous coding!!!