Docker is a great tool even for local development. At the same time, managing Python requirements in containers can be tricky. The pretty standard behavior is to use
requirements.txt file that adds to the image during the building process. Usually, we are using something like that in the
... COPY ./requirements /requirements.txt RUN pip install -U pip && pip install --no-cache-dir -r /requirements.txt && rm -rf /requirements.txt ...
These instructions work fine for production deployment but not very useful for the development stage. Periodically you need to install a new library. With using a virtual environment, without Docker, we could use
pip install ... and then
pip freeze > requirements.txt for storing the installed libraries' information to the file. But with Docker, it is not a case because we need to install libraries on the build stage. At the same time, it is hard to change
requirements.txt manually without breaking dependencies.
There are available Python dependency managers, like
Poetry. But I prefer to use
pip-tools because it is more straightforward. We can add it to the container image by improving the following line:
RUN pip install -U pip pip-tools && pip install --no-cache-dir -r /requirements.txt && rm -rf /requirements.txt
The use case of the tool is simple. First of all, we prepare a file with the name
requirements.in that will contain a list of dependencies. Each line in this file is a dependency. We can also use the same rules for managing versions like the regular
pip install command. For example:
django==3.1.* celery pillow
We can now use the command
pip-compile requirements.in for generating
requirements.txt with all necessary dependencies. With
pip-tools installed in the container, we can use the command like that:
docker run -rm ... pip-compile requirements.in
Now, the file
requirements.txt will be generated, including all required dependencies of described libraries. So, it is possible to rebuild the container for installing new libraries.
You can check the
pip-tools documentation for available commands and options. For example, how to upgrade a particular library. It is a pretty simple yet powerful tool.