DEV Community

Cover image for PDM: A smarter approach to manage Python packages

Posted on

PDM: A smarter approach to manage Python packages

Python Development Master makes use of new Python capabilities to manage packages in a project without the cost imposed by a virtual environment.

Image description

To keep their projects and dependencies distinct, modern Python developers employ virtual environments, or "venvs." However, the size of a project directory is one of the drawbacks of a virtual environment. Each venv may need up to 10MB of disk space - and that's just for the base installation, not counting the libraries. However, there is now a language feature, as well as a package management system, that can help you reduce the size of your project's footprint.

Python 3.8 has a capability that may automatically detect the presence of a subdirectory in a project that holds only packages related to that project. Python supports a pypackages directory to store version-specific versions of packages that can be imported before packages from the main Python install, or even a venv. This is codified in PEP 582.

Most Python developers used PEP 582-style package storage exclusively "by hand" until recently. However, PDM — short for Python Development Master — a newly built package management tool, allows you to add packages to a project utilizing the PEP 582 storage criteria. As a result, the project is smaller, more portable, and less inconvenient to work with.

PDM Configuration

PDM requires Python 3.7 or greater to install. PDM should be installed in the user directory that your Python installation accesses, not in the Python installation itself. This is explained in the PDM documentation. Pip install —user pdm, on the other hand, provides a dependable, automated technique to achieve the same outcome.

It's worth noting that permitting PEP 582 behaviors is a critical aspect of putting up PDM. This may be done either automatically or manually by altering the PYTHONPATH environment variable.

You should be able to run the command pdm on the command line once you've installed PDM successfully. If you have different Python installations on Windows, you may (and should) use py -m pdm to start PDM from the appropriate Python version.

It should be transparent to run a project that requires PDM-installed modules. Simply use the Python interpreter that came with PDM to run Python files.

Using PDM to manage project dependencies

To set up a project to use PDM, go to the root of the project directory in the console and type pdm init. You’ll be asked a few questions about the project, after which your project will have a pyproject.toml file and a pypackages directory.

If you're using source control (e.g., Git), be sure to include pyproject.toml in the repository, but exclude the pypackages directory and the .lock files generated by PDM.

Use pdm add dependency> to add dependencies to the project. You can mention many dependencies, and you should declare them using the PEP 508 standard. To put it another way, they should resemble the items in a conventional requirements.txt file. Use pdm remove dependency> to remove a dependency.

When you're adding or deleting dependencies, keep the following in mind:

  • PDM recomputes the dependency graph for the project every time you add or delete a dependence, which might take some time.

  • Any dependencies that are exclusively dependent on the one you remove are not instantly erased when you remove it, although they may be cleaned up semi-automatically. (I'll go into more detail about this later.)

To list the dependencies in a project, use pdm list. Or you can use pdm list --graph to show dependencies in a tree, so that you can see at a glance which packages depend on which other packages.

You may use one of two commands to confirm that all dependencies are installed correctly:

  • pdm install, the more commonly used of the two, should be your first choice. It will create the lock file — the description of package versions to use with the project — and then install the dependencies from that.

  • pdm sync uses a slightly different strategy. If a lock file doesn’t already exist, it will throw an error; otherwise, it will install the dependencies listed in the existing lock file. This is useful if you want to install only from an existing lock file, without recomputing dependencies.

Managing optional and development dependencies with PDM

Any dependencies installed to a PDM-managed project are installed by default without their purpose in the project being specified. For example, if you use pdm add black to install the black code formatter, PDM will not recognize it as a development dependent vs a runtime need.

If you wish to organize dependencies for special purposes, use the -dG ("dependent group") option in combination with a group name. For example, to install black to a subgroup of dependencies designated dev, we may use pdm add -dG dev black.

Using pdm add -d dependency>, you may additionally indicate that a dependency should be installed solely for production usage. For example, if we utilized the module regex in production but not in development, we'd use pdm add -d regex to add it.

Updating and cleaning PDM dependencies

PDM-controlled dependencies can be updated all at once by typing pdm update, updated individually by using a package name (pdm update ), or updated as a group by using the group name (pdm update -G ).

By default, packages pinned to a specific version will be kept to that version. You can forcibly update pinned packages to their latest versions by passing the --update-eager flag.

If you want to remove packages that are no longer needed, run pdm sync --clean.

When utilizing PDM, there are a few things to keep in mind.

PEP 582 is a very new program. As a result, there are a few limitations to utilizing PDM. For starters, many IDEs do not recognize or use pypackages **directories in a project automatically. The PDM documentation covers how to set up different editors to recognize a project's directory.

Second, certain packages may not work well with PDM if they have archaisms in their internal build and install procedures. If this happens, you'll have to use pip install -t to manually install them in your pypackages directory, and they won't be managed through PDM.

Discussion (0)