package a poetry project in a docker container for production

Docker become the preferred mode of application distribution. For python applications it has become mandatory because it virtually guarantees that if an image works on one environment, then it will work on the others. This strategy also makes it possible to finely manage deployments. Docker allows the promotion of an image from environment to environment by playing on version number and allow devops practices as gitops or canary deployment.

As I explain in the article I move from pipenv to poetry in 2023 - Am I right ?, I move my projects from pipenv to poetry. I also had to upgrade the docker packaging. I will try to share what I learn. I will describe 5 practices I've used to build docker images from poetry projects.

• install a virtual environment in the docker image
• copy only what is strictly necessary by filtering files and folders with .dockerignore
• ignore dev libraries when installing dependencies
• add the virtual environment in the PATH
• use a multi-stage docker build to exclude poetry

install a virtual environment in the docker image

In docker image, we will install a virtual environment in the project as we would locally. The environment will be installed in /app/.venv folder. Doing that is required to be able to exclude poetry dependency thank to a docker multi-stage build.

To ensure that the virtual environment of a project is created next to the sources, in a predictable folder, poetry must be configured. We can do it with this command:

poetry config virtualenvs.in-project true --local

I have doubt first on this practice. I had the impression that creating a virtual environment in a container is counterproductive. However, it was necessary to be able to exclude poetry from the final image.

copy only what is strictly necessary by filtering files and folders with .dockerignore

In a production image, it is not a good idea to drag all files and folders that are used for development. The .dockerignore file is a manifest that specifies files and folders to exclude when adding a folder to an image with the ADD or COPY statement. The .dockerignore uses the same syntax as gitignore.

In general, we'll ignore the package that contains the tests, build artifacts, .venv folder. You can ignore all artifacts that are produced by utilities like mypy, coverage, ...

.dockerignore

/.venv
/tests
/build

Here is an example dockerfile. The code is deployed in the /app folder. The COPY . /app does not copy all items specified in manifest .dockerignore

Dockerfile

FROM python:3.10-slim

RUN pip install poetry  
RUN mkdir -p /app  
COPY . /app

WORKDIR /app

RUN poetry install
CMD ["poetry", "run", "python", "-m", "app.main"]

ignore dev libraries when installing dependencies

poetry allows us to install project dependencies while ignoring development dependencies. These dependencies take up space and are unnecessary when the application is running in production. In addition, they remain accessible at runtime and can degrade the security of your application.

FROM python:3.10-slim

RUN pip install poetry  
RUN mkdir -p /app  
COPY . /app

WORKDIR /app

RUN poetry install --without dev
CMD ["poetry", "run", "python", "-m", "app.main"]

add the virtual environment in the PATH

By default, the docker image uses system python. It's not practical to always prefix the command with poetry run. We will add the python of the virtual environment of poetry to the PATH to use it in priority.

This trick makes the container more robust because you won't need to think about poetry anymore to place a custom command to the container.

# ...
ENV PATH="/app/.venv/bin:$PATH"

RUN poetry install --without dev
CMD ["python", "-m", "app.main"]

I prefer this technique to the entrypoint overload because it makes using commands like sh or bash intuitive when we need to connect on the container.

use a multi-stage docker build to exclude poetry

This practice makes it possible to exclude all the libraries used during the build. The container environment will no longer contain poetry or the dependencies that this toolkit pulls.

This practice has the same effect as avoiding installing dev dependencies. It makes it possible to reduce the size of the container, to pass it from a container of 240Mb to 144Mb.

FROM python:3.10-slim as builder

RUN pip install poetry
RUN mkdir -p /app
COPY . /app

WORKDIR /app
RUN poetry install --without dev

FROM python:3.10-slim as base

COPY --from=builder /app /app

WORKDIR /app
ENV PATH="/app/.venv/bin:$PATH"
CMD ["python", "-m", "app.main"]

These 5 practices may help you build your workflow to distribute a docker image from a poetry project. If you use other workflows or tips on a daily basis, share them with us in the comments.

Finally, I have discarded several practices from this article that you can also apply.

You can make your own base image instead of using an already packaged base image. The python image is updated regularly. It has positive effects like being up to date but also negative like redownloading all the layers at each deployment. In a context where bandwidth is precious, it is interesting to better plan the update of the base image to reduce bandwidth consumption.

In the same vein, another practice is to install external dependencies and application code separately. It's always the same desire to save bandwidth by only downloading the layer that has been modified.

A last practice that I do not use at all and which may interest you is to use slim toolkit to keep only the useful elements in your final image.