Docker become the preferred mode of application distribution. For python applications it has become mandatory because it virtually guarantees that if an image works on one environment, then it will work on the others. This strategy also makes it possible to finely manage deployments. Docker allows the promotion of an image from environment to environment by playing on version number and allow devops practices as gitops or canary deployment.
As I explain in the article I move from pipenv to poetry in 2023 - Am I right ?, I move my projects from pipenv to poetry. I also had to upgrade the docker packaging. I will try to share what I learn. I will describe 5 practices I've used to build docker images from poetry projects.
- install a virtual environment in the docker image
- copy only what is strictly necessary by filtering files and folders with .dockerignore
- ignore dev libraries when installing dependencies
- add the virtual environment in the PATH
- use a multi-stage docker build to exclude poetry
install a virtual environment in the docker image
In docker image, we will install a virtual environment in the project as we would locally. The environment will be installed in /app/.venv folder. Doing that is required to be able to exclude poetry dependency thank to a docker multi-stage build.
To ensure that the virtual environment of a project is created next to the sources, in a predictable folder, poetry must be configured. We can do it with this command:
poetry config virtualenvs.in-project true --local
I have doubt first on this practice. I had the impression that creating a virtual environment in a container is counterproductive. However, it was necessary to be able to exclude poetry from the final image.
copy only what is strictly necessary by filtering files and folders with .dockerignore
In a production image, it is not a good idea to drag all files and folders that are used for development. The .dockerignore file is a manifest that specifies files and folders to exclude when adding a folder to an image with the ADD or COPY statement. The .dockerignore uses the same syntax as gitignore.
In general, we'll ignore the package that contains the tests, build artifacts, .venv folder. You can ignore all artifacts that are produced by utilities like mypy, coverage, ...
.dockerignore
/.venv
/tests
/build
Here is an example dockerfile. The code is deployed in the /app folder. The COPY . /app does not copy all items specified in manifest .dockerignore
Dockerfile
FROM python:3.10-slim
RUN pip install poetry
RUN mkdir -p /app
COPY . /app
WORKDIR /app
RUN poetry install
CMD ["poetry", "run", "python", "-m", "app.main"]
ignore dev libraries when installing dependencies
poetry allows us to install project dependencies while ignoring development dependencies. These dependencies take up space and are unnecessary when the application is running in production. In addition, they remain accessible at runtime and can degrade the security of your application.
FROM python:3.10-slim
RUN pip install poetry
RUN mkdir -p /app
COPY . /app
WORKDIR /app
RUN poetry install --without dev
CMD ["poetry", "run", "python", "-m", "app.main"]
add the virtual environment in the PATH
By default, the docker image uses system python. It's not practical to always prefix the command with poetry run. We will add the python of the virtual environment of poetry to the PATH to use it in priority.
This trick makes the container more robust because you won't need to think about poetry anymore to place a custom command to the container.
# ...
ENV PATH="/app/.venv/bin:$PATH"
RUN poetry install --without dev
CMD ["python", "-m", "app.main"]
I prefer this technique to the entrypoint overload because it makes using commands like sh or bash intuitive when we need to connect on the container.
use a multi-stage docker build to exclude poetry
This practice makes it possible to exclude all the libraries used during the build. The container environment will no longer contain poetry or the dependencies that this toolkit pulls.
This practice has the same effect as avoiding installing dev dependencies. It makes it possible to reduce the size of the container, to pass it from a container of 240Mb to 144Mb.
FROM python:3.10-slim as builder
RUN pip install poetry
RUN mkdir -p /app
COPY . /app
WORKDIR /app
RUN poetry install --without dev
FROM python:3.10-slim as base
COPY --from=builder /app /app
WORKDIR /app
ENV PATH="/app/.venv/bin:$PATH"
CMD ["python", "-m", "app.main"]
These 5 practices may help you build your workflow to distribute a docker image from a poetry project. If you use other workflows or tips on a daily basis, share them with us in the comments.
Finally, I have discarded several practices from this article that you can also apply.
You can make your own base image instead of using an already packaged base image. The python image is updated regularly. It has positive effects like being up to date but also negative like redownloading all the layers at each deployment. In a context where bandwidth is precious, it is interesting to better plan the update of the base image to reduce bandwidth consumption.
In the same vein, another practice is to install external dependencies and application code separately. It's always the same desire to save bandwidth by only downloading the layer that has been modified.
A last practice that I do not use at all and which may interest you is to use slim toolkit to keep only the useful elements in your final image.
Top comments (0)