Pipenv: Named Categories

Today I am happy to announce expanded support for "named package categories" in the release of pipenv==2022.10.9 and here I will explain what this feature is about and some ways you might find it helpful. You may also wish to reference the package categories documentation.

If you are already familiar with pipenv then you are probably aware that until now the tool only supported two package groups: packages and dev-packages. These mapped to the default and develop groups in the lock file.

What if you wanted to have other groups defined? For example: sometimes prerequisites are required to build other packages that are part of your default package group and these have to be installed ahead of time.

Let's say the setup.py being built has import six but six is not installed, and so pipenv gets errors trying to build and install the dependent package. The guidance prior was to pipenv run pip install six ahead of running pipenv sync so that it would be available. Now though it should be possible to define an arbitrary group of prerequisites, for example:

# install prerequisite package six
# note: single category syntax
pipenv install six --catetgories prereq

# install the perquisite group followed by the packages group
# note: multiple categories syntax
pipenv sync --categories="prereq packages"

Support was added for across the range of pipenv commands for the categories argument.

# lock and uninstall examples
pipenv lock --categories="prereq dev-packages"
pipenv uninstall six --categories prereq

Note: The packages/default specifiers are used to constrain all other categories just as they have done for dev-packages/develop category. However this is the only way constraints are applied. The presence of other named groups do not constraint each other, which means it is possible to define conflicting package versions across groups. This may be desired in some use cases where users only are installing groups specific to their system platform.

Some organizations may wish to explore using named package categories to specify platform specific dependencies in groups such as linux, windows or macos. Using platform groupings may be a useful mechanism to work around some limitations of the pip resolver with regards to resolving certain platform specific packages targeting an alternate platform than the one being locked.

This change was also made possible by the concurrent efforts of Oz Tiram who first converted pipenv to the library plette for its Pipfile/Pipfile.lock management. Since we have access to modify plette it became possible to release changes required to support the feature of named package categories.

This particular feature for arbitrarily named package groups was requested multiple times in the pipenv issues reports. I hope the users of pipenv find this feature helpful in project workflows. Have you imagined other use cases not described in this article? Feel free to drop a message in the comments or send us a note.