In this post, I am using Windows 10/Windows Terminal/PowerShell 7 with Python 3.8.1. Adjust examples according to your environment.

If you don't care about the logic and reasoning behind "why Poetry?", skip to Getting Started where we build a simple Flask app using Poetry as our package manager.

• History
• Problems
• Installation
• Getting Started
• Conclusion

§ History ↑

Python has existed for almost 30 years now and many revolutions have occurred in the software arena since its inception. Paradigms have shifted around OOP and functional, design patterns have come and gone, but one issue every language sees, and has varying success in tackling, is dependency management.

For the first half of its life thus far, Python did not have a dedicated way to handle packaging or dependencies. It was the wild west: relying on what's in the standard library or figuring it out yourself. It wasn't until Python 1.6 that the distutils package was added to the standard library in a first attempt to handle the problem in 2000. Here are some more milestones according to PyPA (Python Packaging Authority):

• 2003: PyPI was operational but wouldn't have "packages" hosted until 2005.
• 2004: setuptools is created and is the present packaging go-to.
• 2007: virtualenv package is created to segregate packages per-project.
• 2008: pip supersedes easy_install (from the setuptools package).
• 2011: PyPA is created to take over pip and virtualenv maintenance.
• 2013: PEP 425 and PEP 427 accepted which define the built-package (binary) format, wheel.
• 2016: PEP 518 accepted for the pyproject.toml format for static build dependency declarations.
• 2017: PEP 517 accepted for setup.py-independent (from setuptools) build system backends.

There is a lot of history when it comes to Python packaging and decisions, but I tried to highlight the milestones. The last two are the most relevant to this post as they enabled the Poetry project to take off as it is today and move away from the history of distutils/setuptools.

§ Problems ↑

When you start most Python projects, there isn't really guidance on how to start. Most guides used to give you a requirements.txt file to define your dependencies that you install using pip:

PS ~/> pip install --user --requirement requirements.txt

You would probably install those in your user site-packages (indicated by --user which could cause some chaos with changing versions, conflicting packages, etc.) and off you go. This was hard to scale and deploy, however, which is why virtualenv came along to ease that development process by keeping your dependencies contained per-project and "activating" the environment when you're using it:

PS ~/> python -m venv .venv
PS ~/> .venv/Scripts/Activate.ps1

Another problem with this approach is that the requirements file is rather simple. It's typically a newline-separated plaintext file with a package name and some version indicator that a person writes or generates using:

PS ~/> pip freeze > requirements.txt

But using this approach has its own problems. It locks all the versions with exact specifiers and their dependencies:

PS ~/> pip install flask==1.1.1
# ...
PS ~/> pip freeze
click==7.1.1
Flask==1.1.1
itsdangerous==1.1.0
Jinja2==2.11.1
MarkupSafe==1.1.1
Werkzeug==1.0.0

This creates a lot of manual work to audit your dependencies and bring them up-to-date when needed.

A route some people go with is using setuptools to manage their project as an installable package. This is particularly effective when your project has tests and sub-packages, but adds another layer to the stinky onion.

setuptools uses a setup.py script to control the installation of your package. This file has one requirement when called: to execute setuptools.setup().

import setuptools

setuptools.setup()  # barebones

Normally this function takes arguments to imperatively install the package, but a nice, declarative way to set these arguments is using a setup.cfg file.

# ...
[options]
install_requires =
    Flask==1.1.1
# ...

This file is also used by many packages for configuration, so it's a neat way to centralize your configuration, but it has the same problems as it uses the same format for dependencies. More information on this method of setup can be found here.

This leaves us with the following problems:

• How do we isolate our project? virtualenv is nice, but can be forgotten
• How do we manage our dependencies and their versions? Locking everything that gets installed is a management nightmare.

One last issue that deserves a little blurb: how do we install our own project/package and use it? Depending on PYTHONPATH is fraught with complications especially when you start talking about testing your code and many a solution has been created to mangle it to fit a use-case perhaps unnecessarily.

§ Installation ↑

Poetry's tagline is "Python packaging and dependency management made easy." It accomplishes this by using features introduced by PEP 517 and PEP 518 introducing independent build system backends and new dependency declaration (away from pip and setuptools; they're still there, but abstracted away from the user).

What this looks like in action is something that resembles JavaScript's yarn tool. There is the pyproject.toml similar to package.json and a poetry.lock like yarn.lock. There is also configuration of the tool itself kept in poetry.toml (again, like .yarnrc.yml).

Poetry is available on PyPI, but they recommend using their alternative installer to avoid polluting your Python packages.

# bash
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python

# PowerShell
(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python

# python
pip install --user poetry

§ Getting Started ↑

To the meat and potatoes- using the tool! Here, the tool creates our project scaffold so the package is installable in a virtual environment:

PS ~/> poetry new --name app --src poetry-intro
Created package app in poetry-intro

Now that we have a project, I go a step further to create the virtual environment local to the project for easier editor integration:

PS ~/> Set-Location -Path poetry-intro
PS ~/poetry-intro/> poetry config --local virtualenvs.in-project true

With the configuration set, we're ready to initialize the environment, install our package, and add some dependencies:

PS ~/poetry-intro/> poetry install
Creating virtualenv app in ~/poetry-intro/.venv
Updating dependencies
Resolving dependencies...

Writing lock file


Package operations: 11 installs, 0 updates, 0 removals

  - Installing pyparsing (2.4.6)
  - Installing six (1.14.0)
  - Installing atomicwrites (1.3.0)
  - Installing attrs (19.3.0)
  - Installing colorama (0.4.3)
  - Installing more-itertools (8.2.0)
  - Installing packaging (20.3)
  - Installing pluggy (0.13.1)
  - Installing py (1.8.1)
  - Installing wcwidth (0.1.9)
  - Installing pytest (5.4.1)
  - Installing app (0.1.0)
PS ~/poetry-intro/> poetry add flask
Using version ^1.1.1 for flask

Updating dependencies
Resolving dependencies...

Writing lock file


Package operations: 6 installs, 0 updates, 0 removals

  - Installing markupsafe (1.1.1)
  - Installing click (7.1.1)
  - Installing itsdangerous (1.1.0)
  - Installing jinja2 (2.11.1)
  - Installing werkzeug (1.0.1)
  - Installing flask (1.1.1)

If you're in an environment with private registries, add them according to the docs (I could not get them to work when specifying them in poetry.toml):

~/poetry-intro/pyproject.toml

[[tool.poetry.source]]
name = "private-repo"
url = "https://url"
default = true

[[tool.poetry.source]]
name = "private-repo2"
url = "https://url2"

The default tells Poetry to ignore the public PyPI registry.

Now we'll get to writing code:

~/poetry-intro/src/app/__init__.py

from flask import Flask

app = Flask(__name__)


@app.route('/')
def index():
    return 'Hello, World!'

To execute our new app, we take advantage of the run command which executes commands in the context of the virtual environment:

PS ~/poetry-intro/> $Env:FLASK_ENV = 'development'
PS ~/poetry-intro/> poetry run flask run
 * Environment: development
 * Debug mode: on
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 000-000-000

If you want to work in your environment for an extended period, you may want to use poetry shell to spawn a new shell according to $SHELL with the environment activated.

I usually have some linters and formatters to integrate into intellij/vscode for consistency, so let's add them here as developer dependencies:

PS ~/poetry-intro/> poetry add --dev yapf flake8
Using version ^0.29.0 for yapf
Using version ^3.7.9 for flake8

Updating dependencies
Resolving dependencies...

Writing lock file


Package operations: 6 installs, 0 updates, 0 removals

  - Installing entrypoints (0.3)
  - Installing mccabe (0.6.1)
  - Installing pycodestyle (2.5.0)
  - Installing pyflakes (2.1.1)
  - Installing flake8 (3.7.9)
  - Installing yapf (0.29.0)

You may notice a lack of test runner, but that was already included in the pyproject.toml dependencies when using poetry new. Speaking of which, no project is complete without some tests:

~/poetry-intro/tests/test_app.py

from app import app

app.testing = True
client = app.test_client()


def test_index():
    response = client.get('/')
    assert b"Hello, World!" in response.data

PS ~/poetry-intro/> poetry run pytest
======================== test session starts ========================
platform win32 -- Python 3.8.1, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: ~/poetry-intro
collected 1 item

tests\test_app.py .                                            [100%]

========================= 1 passed in 0.30s =========================

§ Conclusion ↑

This post is starting to get a bit long, but I feel like I've covered the bases to get started. There is a lot more to Poetry, such as building packages, deployment to registries, etc., but the docs are a great location to dig deeper.

In our sample project, we:

• showcased using Poetry for all interactions with the project which made forgetting our virtual environment an impossibility
• saw the lockfile in action which enables repeatable builds and avoids dependency nightmares
• avoided the overhead of setuptools/setup.py to install our package/module in our environment

I hope this project grows into further maturity and sees more adoption because it could be the catalyst to make pip better and/or become the standard like yarn has for many JavaScript devs.