DEV Community

Apify profile image Apify
Apify for Apify

Posted on • Originally published at blog.apify.com on

Python and APIs: how to use Python to connect and interact with APIs

#python #api #tutorial #request

An API (Application Programming Interface) is an interface that allows different systems and applications to communicate with each other. An API is like a messenger that takes your request and delivers it to the provider that youre requesting from, then returns the response to you.

For example, your web browser makes a request to the Apify web server to view this blog post, and the Apify web server, in turn, provides the blog post. An API works the same way: you request some data from an API server and receive a response.

Python and APIs: snake coiled around circuits to illustrate using Python to interact with APIs

Python is a great option to use to interact with APIs

You can save time and make tasks easier by using APIs, and if you already know Python, it's a great language to use for this. APIs are useful for integrating other software, such as payment processing systems, into your applications. APIs can be used to access data in real-time, such as the Weather API to access current weather conditions, the Google Maps API to integrate a Google map within your application, or the Stock Market API to access real-time stock quotes, data feeds and stock exchanges. There are many APIs available that can be used for lots of different purposes.

There are various HTTP methods and status codes for APIs. HTTP methods tell the API the type of action that needs to be performed on the data. Once the API processes the HTTP request that is made using HTTP methods, it returns a response with an HTTP status code. This status code tells us about the success or failure of the request.

API status codes and HTTP methods

Let's take a look at some of the most common status codes and HTTP methods.

Status codes

The status code is returned at every call made to the web server. Here are some of the most common status codes:

  • 200 - OK : The request was successful.

  • 201 - Created : The resource was created, such as a new user or a new file.

  • 204 - No Content : The request was successful, but there is no content to return.

  • 301 - Moved Permanently : The requested resource has been permanently moved to a new location.

  • 400 - Bad Request : The server cannot process the request because the request is either wrong or missing some information.

  • 401 - Unauthorized : The request requires authentication because you did not send the correct credentials to access an API.

  • 403 - Forbidden: You do not have the right permission to access the resource.

  • 404 - Not Found : The requested resource is not found on the server.

  • 405 - Method Not Supported : The method used in the request is not supported by the resource.

  • 500 - Internal Server Error : An unknown error has occurred on the server side.

  • 503 - Service Unavailable : The server is overloaded or undergoing maintenance.

HTTP methods

There are many HTTP methods, but the five most commonly used are listed below:

  • GET : This method is used to retrieve data (such as the content of a web page) from the server.

  • POST : This method is used to send data to the server or add new data to the server.

  • PUT : This method is used to update data on the server or change the existing information.

  • PATCH : This method is used to make partial changes to an existing resource on the server.

  • DELETE : This method is used to delete certain information from the server.

Getting started with the Python Requests library

In Python 3, you need only one library for making HTTP requests and working with APIs: the Requests library. Requests is not a part of the standard Python library, so youll need to install it to get started.

Python and APIs: snake coiled around keyboard

Install Python Requests with pip install requests to get started

The requests library allows you to send HTTP requests. The HTTP request returns a response object with all the response data (content, status code, headers, etc.). The two most important parts of response data are the status code and the response body. It's a versatile library that can even be used for web scraping with Python Requests.

The status code indicates the success or failure of the request. The response body contains the data returned by the API. The data can be in the form of a dictionary, string, or any other object.

You can install the requests library by running the below command in your terminal:

pip install requests
Enter fullscreen mode Exit fullscreen mode

After the installation finishes, import it using the code below.

import requests
Enter fullscreen mode Exit fullscreen mode

All set, so its time to make some API requests with Python. Well be using REQRES API.

Using GET request

One of the common HTTP methods is GET. The GET method allows you to retrieve data from the specified resource.

To make a get request, invoke the requests.get() method.

import requests
response = requests.get('https://reqres.in/api/users')
print(response)
Enter fullscreen mode Exit fullscreen mode

The code output is this:

<Response [200]>
Enter fullscreen mode Exit fullscreen mode

The code is generating status code 200, which shows that the URL is reachable. Its time to retrieve some data from the URL.

import requests
response = requests.get('https://reqres.in/api/users')
print(response.json())
Enter fullscreen mode Exit fullscreen mode

The code output is this:

{'id': 1, 'email': 'george.bluth@reqres.in', 'first_name': 'George', 'last_name': 'Bluth', 'avatar': 'https://reqres.in/img/faces/1-image.jpg'}, 

{'id': 2, 'email': 'janet.weaver@reqres.in', 'first_name': 'Janet', 'last_name': 'Weaver', 'avatar': 'https://reqres.in/img/faces/2-image.jpg'}, 

.
.
.
Enter fullscreen mode Exit fullscreen mode

The GET request is made to the API endpoint /users, and the list of all the users will be returned. Then the .json() function is called to produce the output in JSON (JavaScript Object Notation) format, as shown above. The JSON output looks like a Python dictionary thats easy to read.

To read the data of a single user, you can make an API call with id = 1.

import requests
response = requests.get('https://reqres.in/api/users/1')
print(response.json())
Enter fullscreen mode Exit fullscreen mode

Here's the code output:

{'data': {'id': 1, 'email': 'george.bluth@reqres.in', 'first_name': 'George', 'last_name': 'Bluth', 'avatar': 'https://reqres.in/img/faces/1-image.jpg'}, 'support': {'url': 'https://reqres.in/#support-heading', 'text': 'To keep ReqRes free, contributions towards server costs are appreciated!'}}
Enter fullscreen mode Exit fullscreen mode

If you try to make an API call with an id that doesnt exist, youll get a 404 response, which means the user with this particular id is not found. Run the below code, for example, and youll get a 404 response.

import requests
response = requests.get('https://reqres.in/api/users/22')
print(response)
Enter fullscreen mode Exit fullscreen mode

Using POST request

POST request will add new data to the API. First, create the API data in the form of JSON and then add it using the POST method.

import requests
new_user = {
    "name": "morpheus",
    "job": "leader"
}
response = requests.post("https://reqres.in/api/users", json=new_user)

print(response)
print(response.json())
Enter fullscreen mode Exit fullscreen mode

Heres the code output:

<Response [201]>
{'name': 'morpheus', 'job': 'leader', 'id': '781', 'createdAt': '2023-08-05T16:23:14.034Z'}
Enter fullscreen mode Exit fullscreen mode

The new user has been created, and the above API data is returned.

You can register new users by providing your email and password.

import requests

new_user = {
    "email": "eve.holt@reqres.in",
    "password": "pistol"
}
response = requests.post("https://reqres.in/api/register", json=new_user)

print(response)
print(response.json()
Enter fullscreen mode Exit fullscreen mode

The code output is this:

<Response [200]>
{'id': 4, 'token': 'QpwL5tke4Pnpja7X4'}
Enter fullscreen mode Exit fullscreen mode

The new user has been registered, and the id of a new user is 4.

If you try to register with only email id, then youll get a response of 400 which means a bad request.

import requests

new_user = {
    "email": "eve.holt@reqres.in"
}
response = requests.post("https://reqres.in/api/register", json=new_user)

print(response)
print(response.json())
Enter fullscreen mode Exit fullscreen mode

Heres the code output:

<Response [400]>
{'error': 'Missing password'}
Enter fullscreen mode Exit fullscreen mode

Using PUT request

PUT request is used to update the existing data, which means old data is replaced by new data. Below, were updating a new user whose id is 2.

import requests

update_user = {
    "name": "morpheus",
    "job": "zion resident"
}
response = requests.put("https://reqres.in/api/users/2", json=update_user)

print(response)
print(response.json())
Enter fullscreen mode Exit fullscreen mode

The code output is the following:

<Response [200]>
{'name': 'morpheus', 'job': 'zion resident', 'updatedAt': '2023-08-05T17:03:18.934Z'}
Enter fullscreen mode Exit fullscreen mode

The information has been updated as shown above, and you can also see the time and date of the update.

Using PATCH request

To make partial updates, you can make a PATCH request to the API.

import requests

update_user = {
    "name": "morpheus",
    "job": "zion resident"
}
response = requests.patch("https://reqres.in/api/users/2", json=update_user)

print(response)
print(response.json())
Enter fullscreen mode Exit fullscreen mode

The code output is this:

<Response [200]>
{'name': 'morpheus', 'job': 'zion resident', 'updatedAt': '2023-08-06T05:24:22.972Z'}
Enter fullscreen mode Exit fullscreen mode

Using DELETE request

To delete resources from the API, use the DELETE request.

import requests
response = requests.delete("https://reqres.in/api/users/2")
print(response)
Enter fullscreen mode Exit fullscreen mode

This is the code output:

<Response [204]>
Enter fullscreen mode Exit fullscreen mode

The user has been deleted, which is why the server is returning a 204 response. This means that the request was successful, but theres no content to return.

Query parameters

When make API calls with Python Requests, you get a large amount of data in return. In the above code examples, youve seen that when you make a GET request, you get tons of user details. However, in most cases, you only want certain information. This is where query parameters come in.

For example, when you make a Google search, the URL looks something like this: https://www.google.com/search?q=apify. The q=apify is a query parameter. To add a query parameter to the URL, you must use a question mark (?). For multiple query parameters, you can separate them with an ampersand (&).

Let's add some query parameters to the Python API call. Here, were adding two query parameters, which show that you want all the users of page 2 with a delay of 3 seconds. The final query will become this: https://reqres.in/api/users?page=2&delay=3.

import requests

query_para={
    'page': 2,
    'delay': 3
}

response = requests.get('https://reqres.in/api/users', params=query_para)
print(response.json())
Enter fullscreen mode Exit fullscreen mode

Heres the code output:

{'page': 2, 'per_page': 6, 'total': 12, 'total_pages': 2, 'data': [

{'id': 7, 'email': 'michael.lawson@reqres.in', 'first_name': 'Michael', 'last_name': 'Lawson', 'avatar': 'https://reqres.in/img/faces/7-image.jpg'}, 

{'id': 8, 'email': 'lindsay.ferguson@reqres.in', 'first_name': 'Lindsay', 'last_name': 'Ferguson', 'avatar': 'https://reqres.in/img/faces/8-image.jpg'}, 

{'id': 9, 'email': 'tobias.funke@reqres.in', 'first_name': 'Tobias', 'last_name': 'Funke', 'avatar': 'https://reqres.in/img/faces/9-image.jpg'}, 

{'id': 10, 'email': 'byron.fields@reqres.in', 'first_name': 'Byron', 'last_name': 'Fields', 'avatar': 'https://reqres.in/img/faces/10-image.jpg'}, 

{'id': 11, 'email': 'george.edwards@reqres.in', 'first_name': 'George', 'last_name': 'Edwards', 'avatar': 'https://reqres.in/img/faces/11-image.jpg'}, 

{'id': 12, 'email': 'rachel.howell@reqres.in', 'first_name': 'Rachel', 'last_name': 'Howell', 'avatar': 'https://reqres.in/img/faces/12-image.jpg'}], 

'support': {'url': 'https://reqres.in/#support-heading', 'text': 'To keep ReqRes free, contributions towards server costs are appreciated!'}
Enter fullscreen mode Exit fullscreen mode

All users on page 2 will be shown after a 3-second delay.

Python and APIs: illustration of high-tech snakes and ladders game

Troubleshooting with Python doesn't have to be like a high-tech game of snakes and ladders

Troubleshooting Python and APIs

Troubleshooting is important when working with APIs. Lots of things can go wrong (e.g. the API server may be down, or you may have sent the wrong parameters). Here are some of the most important advice that can help you troubleshoot errors:

  1. Identify the issue : determine if the issue is related to your code or an external factor. For example, if youre using the Google Maps API and you haven't filled out billing information, then youll get an error when you make a request using your the API key.

  2. Use print statements : use print statements at various stages of your code to debug the code and understand whats happening. You can print the responses of the request, URLs, response data, etc.

  3. Check your input : double-check your input, such as the API endpoint URL, credentials, headers, query parameters, and so on.

  4. Handle exceptions : you can use exception handling (try/except blocks) to catch exceptions.

  5. Rate limits and authentication : be aware of the rate limits imposed by the API and ensure that your API keys and tokens are correctly configured and you have the necessary permissions to access the API endpoints.

Let's deal with a real-world issue to understand this more clearly. Here, well use the GitHub API to retrieve repos using a GET request.

You might like to first read how to use Python with the GitHub API and then come back.

While retrieving the repos from the apify github, theres a chance that some issues may occur, such as HTTP errors, network exceptions, etc. That's why using the try/except blocks can help us to catch errors.

First, well use the requests.get() method to read all the repos of the given username. Then, well check the status of the response using the raise_for_status() method. This method will raise an exception for non-200 status codes, and then this exception will be handled in the except block of the HTTPError.

After that, well use one except block for the network error because sometimes you may lose your internet connection. And at last, theres an except block for any unknown errors that may occur.

import requests

If you change the repo name to "apifyy", a 404 error will occur. If you lose your network connection, a network error will occur, and you won't be able to fetch the repos.

Advanced tips

If you've read this far, and even more so if you've looked at our guides to using Python with the GitHub API, the Discord API, and the Google Maps API, you should have a solid understanding of Python and APIs.

If you want to take it up a level, here are some advanced Python API tips that you could learn and try: authentication, pagination, and rate limiting.

Authentication

Many APIs require some form of authentication in order to access their resources. Some APIs use basic authentication, while others require more complex techniques like OAuth.

Before making a request, you need to obtain an access token from the API provider. This is a unique identifier that grants you access to specific resources or actions.

headers = {

"Authorization": f"token {YOUR ACCESS TOKEN}

}

Pagination

Pagination is the process of dividing large amounts of data into multiple smaller pages. This is done to prevent the API from returning too much data at once, which can overload the API server. For example, if you go to the Apify repos, you will see something like this at the bottom:

Python and APIs: screenshot of pagination

Pagination is normally handled with the help of query parameters: page and size. The page parameter defines which page you are requesting, and the size parameter defines the size of each page, like how many repos you want on one page.

Rate limiting

Anyone can use public APIs, and sometimes people try to abuse them by making unnecessary requests or using bots. To prevent such attacks, the rate-limiting technique is used, which restricts you from making a certain number of requests per unit of time. Make sure you do not exceed the limit, as your requests will be blocked for some time.

For example, if you use the GitHub API and youre making unnecessary requests, youll get blocked after 60 requests and receive a 403 status code.

Python and APIs: screenshot of API rate limit exceeded error

Using APIs and Python

We've introduced you to the basics of using the Python Requests library to send API requests, and to some advanced API concepts and troubleshooting tips. You should now be aware of the most common HTTP methods and status codes and how they're used to retrieve or edit data through an API. From understanding status codes to manipulating data and even controlling the specifics of your request with query parameters, we've laid the foundation for more advanced use of APIs.

If you're interested in using Python for other purposes, check out web scraping with Python, or look at our other Python articles.

Top comments (0)

Read next

kuhanraja_ar_1df623b8e2e profile image

python session day 2 at payilagam

Kuhanraja A R -

josh_devquezada profile image

Pattern matching for everyone (Using in video games)

Josh Quezada -

chintanonweb profile image

Mastering JavaScript Memory: A Beginner’s Guide to Stack and Heap

chintanonweb -

keploy profile image

Understanding HTTP Status Codes: A Complete Guide

keploy -