Local Development Environments for Python using asdf & Poetry

In this blog post I introduced my general setup of how I use asdf for local development for languages. That was meant as a dip into the methodology in order to ramp us up for where I see the real benefit using this for Python & virtual environments.

We will go through almost the exact same setup until we add Poetry and i'll walk you through step by step.

Assumptions:

1. You have homebrew homebrew install
2. You are using MacOS or Linux.

Step 1: Pre-reqs:

1. asdf: Installing asdf this is how you can manage different versions of languages and use them across projects
2. Installing Poetry (for Python only) is a way to manage dependencies & virtual environments in Python. This was shown to me by Data Scientist at work and I would never go back using any other method.

Step 1: Setup for both install asdf & add to your shell (I use zsh, so adjust if you use bash or something else).

• brew install asdf
• echo -e "\n. $(brew --prefix asdf)/libexec/asdf.sh" >> ~/.zsh_profile

Step 2: Python Setup

In asdf the order of operations is adding a 1. language plugin, 2. adding a version(s) and 3.then setting the language locally or globally for your projects

Setup Langauge Plugin

• Setting up asdf for go run asdf plugin add python
• adding different go versions if you want the latest run asdf install python latest
• OR install a specific version running asdf install python 3.10.0
• For Python in my organization we recommend using Python version 3.n-1. So if the latest is 3.11 we would use 3.10 in production.
• In the case below I already have both installed

Setup Local Version

In this case I have a hello-python directory that i want to use Python 3.10 in order to do that you would run

• asdf local python 3.10.0 This is really important you want to keep your python versions separate for repos so you are using the correct version for your project this makes it easier to manage dependencies when we introduce Poetry.

An additional note you will only have to do this once in the directory unless you want to change the version number.

Step 2: Setting up Poetry

• We should have poetry installed & the path setup in your shell.
• Make sure you're in the directory of the project you want to setup the virtrual environment.
• Poetry is like pyenv or venv for Python. the difference is it also manages dependencies issues building a .lock & .toml file
• run poetry init and start the walkthrough in the CLI.
• You can add the naming of the project, the license, the python versions and even the packages. The packages can be added after the generation too.

• once you went through everything you can confirm the generation this will create a .toml file
  

• You can now install all the dependencies from the pyproject.toml by running
  poetry install

• this creates a virtual environment in the python version you have locally and all the dependencies.

• The beauty of Poetry is that if you have a version conflict with different packages it will attempt to resolve it by upgrading/downgrading the package based on the constraints you set for the project. An example is if you have numpy version 1.20 and pandas 0.16 they might be compatiable it will resolve the issue by upgrading pandas based on the constraints you set on the pyproject.toml

Step 3: Using the Virtual Env & Poetry

• to add packages you can run poetry add pandas
• to remove packages you can run poetry remove pandas
• to enter the virtual environment run poetry shell and then running python hello.py or run you files by running poetry run python hello.py

Tips:

• Dont name your project the same as a python project there can be naming conflicts. i.e. your project is named Pandas and you are adding Pandas.
• You can adjust the pyproject.toml and reinstall the packages in your virtural env