PyEnv && Pipenv

If there's something I've learnt with experience is that there's almost always a simple way to do things. There are millions of people dealing with development problems the same way you are, so chances are someone already stumble with a way to simplify things.

One of those simple things is putting together a Python development environments.

I'm wouldn't say I'm a Python developer. I only use it for a specific subset of cases, mostly requiring large amounts of data for analysis, Machine Learning, etc.

So intuitively, when setting up my development environment I try to make it as easy as possible, and coming from other programming languages and frameworks such as Ruby, Elixir, Node, etc where there's clear separation between version language and libraries almost by default, I'd expect Python to be no different.

Wrong! 😭

Python's setup is a mess. Python 2 is still around and it has no compatibility with Python 3. Python 3 at the same time has so many ways to be setup that it makes it so confusing.

If you pick the wrong way to setup and tie too many things to the system, then you end up with incompatibilities across projects and all other sorts of issues.

However, there's one type of setup that, in my opinion, seems to be the most similar to what I'm used to. And it's using Pyenv along with Pipenv to setup projects individually and isolated within their own Python and library versions.

Let's go ahead and dig deeper into this.

Pyenv

Pyenv is a Python version manager tool. It was based on the popular Ruby version managers rbenv and ruby-build.

Its functionality is quite simple. It lets you have different versions of Python in your environment, which are not dependent on each other or the system.

There are many other tools that sort of do something similar, but I find Pyenv to be the most unintrusive of all. It does not require Python itself as it's built with shell scripts, and it does not need to be loaded into your shell.

One additional thing to remember is that you don't want your system Python running the show. So the more you can stay away from it and use isolated environments the better.

Installation

There are many ways to install Pyenv.

By far the easiest if you're on MacOS or Linux is via Homebrew

❯ brew install pyenv

After this, you need to let your bash initiate pyenv on start. For this you can add the following init script to whatever shell you're using.

In my case for zsh I need to add it to my .zshrc but for you it could be .bash_profile or .bashrc.

if command -v pyenv >/dev/null 2>&1; then
  eval "$(pyenv init -)"
fi

This will allow Pyenv to manipulate your path and give itself priority inside pyenv environments.

And it's done!

For more installation instructions you can check their Github page.

Using Pyenv

Next thing to do is install a new version of Python. You can install as many as you need, even from different distributions.

To see what versions are available, use

❯ pyenv install --list

This will give you a full list of available versions.

Let's go ahead and install Python 3.9.1

❯ pyenv install 3.9.1

Just like that, you're rolling on Python 3.9.1, yay!

Last step is to tell your system the global version you want to use. You don't necessarily have to do this, but it makes things a bit easier down the line

❯ pyenv global 3.9.1

Congratulations, you have Python in your system 🙂

Pipenv

Pipenv works at a lower level than Pyenv, and they can work together very well.

Pipenv is a tool, again sort of inspired in Bundler, Npm, Yarn etc, that aims to keep your project environment nice and tidy.

It might be counter-intuitive and you may be tempted to just install everything at the global scale so it's available for every project you use, but don't! This is an easy way to depend on libraries that may not be available when using somewhere else.

Additionally Pipenv makes collaboration really easy. Stakeholders all use the environment setup in the Pipfile and this guarantees everyone is on the same page.

So what does Pipenv do exactly?

It uses a Pipfile and Pipfile.lock to sort of snapshot the dependencies you use in a project. It allows you to work in an enclosed environment, without the fear of your work slipping out to your other project environments.

Installation

Once you have Pyenv running and are on your preferred version of Python, run

❯ pip install -U pipenv

Pyenv sort of hijacks the pip command, to make sure you're working in the correct environment.

After this you're ready to start using pipenv.

The library is meant to be used within a project or project folder/context, so go ahead and navigate to your preferred project directory and initialise pipenv in there

❯ pipenv install

This will create a Pipfile containing your dependencies, and a Pipfile.lock containing the versions and dependencies of those dependencies.

Traditionally, another way to manage dependencies in Python projects is using a requirements.txt file. If you already have one of those on an existing project, pipenv will transform it to a Pipfile with the previous step and attempt to migrate your existing dependencies to it.

Using Pipenv

From now on, you're good to start using pipenv to manage your dependencies.

To add a new one you can let pipenv know what you want to install and it will update both the Pipfile and the Pipfile.lock for you

# Installing Jupyter notebooks
❯ pipenv install notebook

Just like that your new dependency has been added your your Pipfile and Pipfile.lock.

Now to "enter" the environment that pipenv has created for you, you need to activate a new shell using

❯ pipenv shell
Launching subshell in virtual environment…

This will load your current bash with the dependencies installed within your pipenv environment. Every command that you run within this shell will execute in the context of your Pipenv.lock.

To exit the shell just type exit.

Conclusion

There's so much to learn about Pyenv and Pipenv, that you're better off just following the documentation and trying it yourself, but I guarantee you it will save you countless hours of configuration and figuring out what's going on with your project dependencies and why some projects are failing after updating other unrelated ones.

If you want to know more about the different environment setups that Python has to offer and its differences, there's this really detailed StackOverflow post that explains it very well. Even though their recommendation is using virtualenv + pip instead, it's worth having a look at the different options summarised so you can make an informed decision of what you want.

For more topics, check my profile or visit rarias.dev.

Happy coding!