HTTPX - A next-generation HTTP client for Python.
HTTPX is a fully featured HTTP client library for Python 3. It includes an integrated command line client, has support for both HTTP/1.1 and HTTP/2, and provides both sync and async APIs.
Install HTTPX using pip:
$ pip install httpx
Now, let's get started:
>>> import httpx
>>> r = httpx.get('https://www.example.org/')
>>> r
<Response [200 OK]>
>>> r.status_code
200
>>> r.headers['content-type']
'text/html; charset=UTF-8'
>>> r.text
'<!doctype html>\n<html>\n<head>\n<title>Example Domain</title>...'
Or, using the command-line client.
$ pip install 'httpx[cli]' # The command line client is an optional dependency.
Which now allows us to use HTTPX directly from the command-line...
Sending a request...
Features
HTTPX builds on the well-established usability of requests, and gives you:
- A broadly requests-compatible API.
- An integrated command-line client.
- HTTP/1.1 and HTTP/2 support.
- Standard synchronous…
HTTPX 🦋 is a next generation HTTP client for Python. It is Requests-compatible, supports HTTP/2 and HTTP/1.1, has async support, and an ever-expanding community of 35+ awesome contributors. ❤
HTTPX is part of a growing ecosystem of async-capable Python projects lead by the Encode organization (which backs the Django REST Framework). Some of the most notable projects include Uvicorn, Starlette, and Databases.
Hacktoberfest started yesterday, and on behalf of the HTTPX maintenance team I can say we're very excited to be part of it this year!
Before getting to the contributing guide, a short preview of what HTTPX can do…
Your first request to dev.to using HTTPX:
>>> import httpx
>>> r = httpx.get('https://dev.to')
>>> r.status_code
200
>>> r.headers['content-type']
'text/html; charset=utf-8'
>>> r.text[:50]
'<!DOCTYPE html>\n<html lang="en">\n <head>\n <met'
>>> r.http_version
'HTTP/1.1'
Does DEV support HTTP/2?
>>> import httpx
>>> # The top-level API only supports HTTP/1.1 for efficiency reasons,
>>> # but 'Client' uses HTTP/2 by default.
>>> with httpx.Client() as client:
... r = client.get('https://dev.to')
...
>>> r.http_version
'HTTP/2' # YAY! 🎉
HTTPX can even call into Python web apps (WSGI + ASGI). A Flask example…
>>> from flask import Flask
>>> import httpx
>>> app = Flask(__name__)
>>> @app.route("/")
... def hello():
... return "Hello, world"
...
>>> with httpx.Client(app=app, base_url='http://testserver') as client:
... r = client.get('/')
...
>>> r.status_code
200
>>> r.text
'Hello, world' # Tada! 🦋
>>>
And it supports async/await! Let's fire an IPython REPL to try this out:
In [1]: import httpx
In [2]: import asyncio
In [3]: # We're going to fetch tag pages concurrently...
In [4]: async def fetch(tag, client):
...: return await client.get(f'https://dev.to/t/{tag}')
...:
In [5]: async with httpx.AsyncClient() as client:
...: responses = await asyncio.gather(
...: fetch('hacktoberfest', client),
...: fetch('python', client),
...: fetch('opensource' client),
...: )
In [6]: responses
Out[6]: [<Response [200 OK]>, <Response [200 OK]>, <Response [200 OK]>]
In [7]: [r.url for r in responses]
Out[7]:
[URL('https://dev.to/t/hacktoberfest'),
URL('https://dev.to/t/python'),
URL('https://dev.to/t/opensource')]
(This example uses asyncio, but trio is also supported!)
HTTPX already has a lot of features built-in, but it's still in an alpha stage as some of the areas still need more work.
So enough of an intro, here's how YOU can help us build the future of Python HTTP.
How you can help
Try it out!
The most straight-forward way to help move HTTPX forward is by trying it out for yourself.
For example:
- Run the sample code in the introduction to this post to see HTTPX in action!
- Build a simple application that calls a public API using HTTPX.
- Migrate an application that uses Requests to make HTTP calls over to HTTPX.
- Use HTTPX as a test client for your web project powered by Flask, Starlette, FastAPI, etc. More on this in Calling into Python web apps.
- Build a web spider that fetches web pages in parallel using HTTPX.
- ... And any other idea you feel like trying!
However you decide to try HTTPX out, we'd love to hear about it! Send us a link to a repo, blog post, or anything else you'd like to share via the comments section. :-)
Contribute!
As of writing this post, there's a whopping list of 10 hacktoberfest-labelled issues, all up for grabs! Expect more to come!
Below are some highlights of what's available at the moment.
Documentation
Documentation is critical to any software project. It helps new users get started quickly, and existing users find the information they need on how to use a particular feature or piece of API.
We know that the HTTPX documentation needs refining in several places. Feel free to walk through it and submit PRs for anything you think can be improved — be it typos, copy, or missing information. :-)
.post(), .put(), and .patch() functions in API docs missing 'files' parameter
#409
Our API docs are missing the files parameter on .post(), .patch(), and .put() for Client and AsyncClient
Environment Variables docs should justify each usage case.
#404
Our environment variable documentation should include some justification or context for each case, by doing one of the following for each envvar:
- Make it clear that an environment variable is specific to HTTPX. (Probably only relevant to anything prefixed by
HTTPX_) - Make it clear that an environment variable is used as a convention by various tooling, with at least one relevant hyperlink.
Eg.
SSLKEYLOGFILE could give some more context about usage - "The keylog file is designed for debugging purposes only." doesn't make it clear how or when to use it, or which debug tooling it's usful with.
SSL_CERT_FILE and SSL_CERT_DIR can include a link to some docs about their convention. Perhaps give an example of other tooling that supports them?
HTTP_PROXY, HTTPS_PROXY, ALL_PROXY can include a link to some docs about their convention. Perhaps give an example of other tooling that supports them?
More explicit documentation that `httpx.Client` is equivelent to `requests.Session`.
#403
This isn't an issue (sorry if i posted in a wrong place), but i read the httpx doc and didn't find yet a solution. In requests i use s = requests.session(), how can i do the same thing with httpx? it will be possible to do something like s = httpx.session()? Thanks and sorry again.
Bug fixes
Fact: all software has bugs 🐛, and surely HTTPX is no different!
Some of the bugs have already been tracked…
Host header contains userinfo component
#391
BaseRequest.prepare() incorrectly sets the default Host header to userinfo@example.com:123 if given the following URL: http://userinfo@example.com:123.
We should be removing the userinfo component from the authority before assigning to Host.
…But most of them still need to be discovered, tracked down, and fixed!
This is also why it's super important that you give HTTPX a try. Your use case may not be one we have thought about yet, or it might be in conditions that haven't be accounted for yet.
If you find about something that doesn't look right, please open an issue and we'll look into it together!
New features
HTTPX already has a lot of features built-in, but we've got ideas on how to push things even further.
Below are some features you can tackle. If you try out HTTPX and feel like something's missing, please open an issue! We'll discuss it together and will gladly help you get started if you'd like to implement the feature.
Add support for the NO_PROXY environment variable
#360
The NO_PROXY environment variable is described here:
- http://www.gnu.org/software/wget/manual/html_node/Proxies.html
- https://github.com/psf/requests/blob/840540b6b1f07ef87faab73392c03fbef0dcc9fe/requests/utils.py#L395
In order to accomplish this task we'd need:
- Implementing NO_PROXY, a list of hosts that will bypass the proxy if found within the list.
- Only enabled if
trust_env=True - Documentation updates for the new environment variable.
- Test cases for all code branches.
[Feature Request]: raise httpx exception instead of buildin ConnectionResetError and ConnectionError
#349
This is a feature requests instead of bug report.
I'm using requests in my web server to request upstream api. So I can handler all requests.RequestException in my middleware and return a 502 upstream error.
But after I migrated to httpx, httpx doesn't have such exception when issuing a http(s) request, it will raise httpx.ConnectTimeout, buildin ConnectionError and ConnectResetError. And last 2 exception are not only raised by httpx.
Maybe httpx should also wrap ConnectionError and other buildin connection exception?
Closing thoughts
There are many ways to contribute to HTTPX, and all contributions are welcome! Once you've selected an issue, documentation improvement, feature request or bug report you'd like to work on, read the contributing guide to get started, and hack away!
Thank you so much for reading through this post, and we look forward to seeing the amazing contributions you'll make. 💖🦋
— The HTTPX maintenance team
Top comments (0)