Resolving Module Version Chaos: Locking Down Dependencies in Python Projects with Poetry

Hey there! 👋 I've got a nifty trick to share about managing Python dependencies, especially when they're not version-locked. Let me walk you through how I tackled it using Poetry.

Problem 🤔

Ever faced a requirements.txt that looks like this?

tqdm
matplotlib

No version numbers can be a recipe for chaos during builds or at runtime due to inconsistencies. I needed to lock these dependencies to specific versions to keep things smooth and reliable, like this:

tqdm==4.64.0
matplotlib==3.5.3

Solution ✨

Why Poetry?

I chose Poetry because it's like the npm of the Python world—it respects semantic versioning and creates a lock file so every install is consistent. No more "works on my machine" issues!

Step-by-Step Guide

1) Install Poetry:

   curl -sSL https://install.python-poetry.org | python3 -

2) Grab a simple pyproject.toml template:

   wget https://gist.githubusercontent.com/ma7dev/7298ffc4409032edd4d18a57b4c38f3a/raw/1c32efcbde31aaf896c6d47b32dac19ed44d14a4/pyproject.toml

3) Install those unversioned dependencies:

   cat requirements.txt | xargs poetry add

4) Export the installed dependencies in a more structured format:

   poetry export -f requirements.txt --output long_requirements.txt --without-hashes

5) Clean up the exported file:

   # Strip unwanted python version constraints
   cat long_requirements.txt | cut -d ";" -f 1 > with_dep_requirements.txt
   # Filter out extraneous dependencies
   cat requirements.txt | while read line   do echo $(grep -n $line'==' with_dep_requirements.txt | cut -d ":" -f 2) >> final_requirements.txt done

Result 🚀

Here’s what you end up with, all dependencies neatly versioned (final_requirements.txt):

tqdm==4.64.0
matplotlib==3.5.3
... (rest of your dependencies)

This setup ensures that all packages are locked to specific versions, making your project stable and reproducible wherever it goes. 🌐

If you enjoyed reading this article, check my other articles on ma7.dev/blog.