In today's enterprise world it is likely that when developing in python there will be a need to have multiple versions of python living on a system in parallel. There are a number of solutions to manage multiple versions on the same machine: *conda (miniconda, anaconda, etc.), different install directories, and pyenv to name a few.
Multiple Python Version Solutions
I have found that the conda style setups can be heavy weight and introduce their own package management interfaces. I am not a big fan of having to bounce between the two or providing the option to those that are not as technically savvy. In my experience, this has always ended up very messy.
The manual setup of different install directories is a solution that requires more manual intervention from setting the PATH correctly, installing new versions, and providing instruction on how to use each version. This easily leads to different methodologies being used between the more advanced users and the less advanced users.
Pyenv is light weight in that it is purely implemented in bash. It provides an interface for creating virtual python environments and installing different python versions along side one another. The project states that it "follows the UNIX tradition of single-purpose tools that do one thing well." Through my use, I have to agree that it succeeds there.
The one short coming that pyenv has is it is focused more on a single user environment, which for the individual developer is perfectly fine. In the enterprise realm, it is often desired to have a unified development environment when working on projects to limit the number of third party libraries in use, keeping internal libraries synced, and knowing that each developer's environment is the same to eliminate the "it worked on my system" retort.
Pyenv and Multiple Users
I introduced pyenv to my team as a solution for version management as well as python virtualenv management. The early adoption work went smoothly and there were no apparent issues, but the number of users and projects being worked was small. Often the users were working on the same project and using the same environments and it was not until we started to use pyenv on a wider scale that issues arose.
Incidences of file locks crept in as more virtual environments were setup and started to be used. Then there was the issue of activating an environment and suddenly your binaries changed from underneath you because another user activated a drastically different environment.
These issues were due to the "shims" directory being shared across all users. When an environment was activate, the shims directory would have the binaries updated to the latest activate environment binaries or in the case of the file lock issue, not updated at all. This is where the pyenv-multiuser plugin was originated.
Pyenv-multiuser Plugin
This plugin resolves the shared shims directory issue of pyenv. It does this by make some modifications to the pyenv base code regarding where it looks for and where it sets up the shims. The shims directory becomes local to the user. With the shims directory now under the user's control and not shared the file lock issue is resolved. The binaries under the shims directory issue is resolved. At least in part as the user could technically open a secondary shell and source another environment and cause issues for themselves.
Why a plugin?
One may be wondering, why make a plugin rather than alter the pyenv code? This is a good question. I opted to make a plugin because it would not change the way most users work with pyenv, which is in the single user space. This, unfortunately, is not completely avoidable.
As I mentioned earlier, the plugin makes changes to the pyenv code anyhow. There is another plugin pyenv-update which performs updates on pyenv and any installed plugins.
pyenv update
This is basically doing a pull from the git repositories. Since this plugin makes changes to the code, the local repository is dirty and thus cannot perform the pull. In order to account for this, an update command was implemented in the pyenv-multiuser plugin.
pyenv multiuser update
This command restores backed up versions of the files that were changed prior to executing the standard pyenv-update command. After the update is complete, the plugin creates new backups and re-applies the code changes.
Using the plugin approach, it should be easier to account for issues like these with other plugins as they may arise.
Looking Forward
Currently, I have deployed this within my company's development environment. I expect to see issues come up as more users and projects interact with it.
Top comments (0)