Python: convenient way to parse URLs

There are various tools to process URLs in Python. Here, we'll see a quick and dirty way to do it.

Don't do it manually

It's not specific to Python, but I recommend using built-in or third-party package if you want to parse elements.

Otherwise, you will likely forget something or spend too much time on known problems.

Quick demo

Suppose you have some string URL to process, but you want to replace specific parts or add some parameters.

One easy way to achieve that could be the following:

import urllib

if __name__ == '__main__':
    parts = list(urllib.parse.urlsplit(url))
    parts[1] = 'otherdomain.com'
    new_url  = urllib.parse.urlunsplit(parts)

Source: Python doc - urllib

N.B.: if you want to split the params from the URL, you'll have to use urlparse() instead of urlsplit().

Pros and cons

urllib is convenient, as it parse URLs into components, and you get various helpers to achieve common tasks.

However, some Python devs would likely recommend using the requests module instead for HTTP-related work, as its API is more recent and standardized (response object, urllib3, etc).

It's up to you.