Going async in python

Going async in python - the missing guide

TLDR;
Make your code run faster and scalable using async API calls in python. with a quick explanation and an example anyone can follow.

Many times we have to make API requests, and one of the most familiar repositories to do so is requests.

Unfortunately, the requests library is not equipped for performing asynchronous requests.
We can wrap async/await syntax around requests, but that will make the underlying requests no less synchronous.

Why should we make our code async?

There are several reasons why we might want our code to be async.
Async code is a programming paradigm that allows multiple tasks to be performed concurrently without blocking.
This can improve the performance, responsiveness, and scalability of an application.

Async code can be run on a single thread, allowing it to make better use of available resources and handle more concurrent requests without running out of resources.

Async code can also make the code easier to read and maintain by allowing complex operations to be broken down into smaller tasks.

So how do we turn our code to async?

In order to get async requests, we must address other tools that can provide it.

One of my go-to tools is asyncio and using aiohttp. (they have amazing docs )

Can we see some examples of sync turn into async code?

In the following section we will take a requests code and convert it to async code using aiohttp.

import requests

IP_GEOLOCATION_KEY="YOUR API KEY"

# requests example 
class GeolocationClient:
    def __init__(self, key=IP_GEOLOCATION_KEY):
        self.key = key

    def get_geolocation(self, ip: str):
        base = "https://api.ipgeolocation.io/ipgeo"
        params = dict(apiKey=self.key, ip=ip)
        data = requests.get(base, params=params)
        return data.json()

Now lets see how we can convert to async method.

To use an async library to fetch data in the GeolocationClient class, you would need to first import the async library and then use its methods to make the HTTP request and retrieve the response.

import aiohttp

IP_GEOLOCATION_KEY="YOUR API KEY"


class GeolocationClient:
    def __init__(self, key=IP_GEOLOCATION_KEY):
        self.key = key

    async def get_geolocation(self, ip: str):
        base = "https://api.ipgeolocation.io/ipgeo"
        params = dict(apiKey=self.key, ip=ip)
        async with aiohttp.ClientSession() as session:
            async with session.get(base, params=params) as response:
                data = await response.json()
                return data

Here, the aiohttp.ClientSession is used to create a session for making HTTP requests. Then, the session.get() method is used to make a GET request to the specified URL with the given query parameters.

The response.json() method is used to parse the response as JSON and return it.

Note that this code uses async/await syntax, which is only supported in Python 3.7 and above. If you are using an older version of Python, you would need to use a different syntax for async operations, so upgrade your version! :)

Here is a full example running the code

import aiohttp
import asyncio

IP_GEOLOCATION_KEY="YOUR API KEY"


class GeolocationClient:
    def __init__(self, key=IP_GEOLOCATION_KEY):
        self.key = key

    async def get_geolocation(self, ip: str):
        base = "https://api.ipgeolocation.io/ipgeo"
        params = dict(apiKey=self.key, ip=ip)
        async with aiohttp.ClientSession() as session:
            async with session.get(base, params=params) as response:
                data = await response.json()
                return data


async def main():
    client = GeolocationClient()
    ips = ["1.1.1.1", "8.8.8.8", "9.9.9.9"]

    # Start all the tasks concurrently
    tasks = [client.get_geolocation(ip) for ip in ips]
    results = await asyncio.gather(*tasks)

    # Print the results as they come in
    for result in results:
        print(result)


# Run the main function
asyncio.run(main())

In this code, the main function is an async function that creates a GeolocationClient and a list of IP addresses.
It then uses a list comprehension to create a list of tasks that call the get_geolocation method on each IP address.

The asyncio.gather function is used to start all of the tasks concurrently and wait for them to complete.

It returns a list of results, which are the JSON responses from the get_geolocation method. The results are then printed as they come in.

Finally, the asyncio.run function is used to run the main function.
This will start the asynchronous tasks and wait for them to complete before exiting the program.

Hope that helps, and as usual, I'd love to know what you think.

Keep calm and code on