What is “pipenv”?
There are many package manager tools in other programming languages such as:
- Ruby
- bundler
- PHP
- composer
- NodeJS
- npm
- yarn
- Rust
- cargo
If you know any of the above tools, it might be easy to understand what it is.
pipenv is just a package management tool for Python as same as those tools.
Why “pipenv”?
The official documentation Pipenv: Python Dev Workflow for Humans and this blog post Why Python devs should use Pipenv would tell us why we should use pipenv.
Two main problems of “before” pipenv workflow are following:
- Using
pipandvirtualenvseparately is not necessary any more! -
requirements.txtis problematic: see A Better Pip Workflow™- without version specified -> different versions every time for dev/prod environments!
- with version specified ->
--upgradeis hard!
If we look back on the history of Python package management tools, there are many tools before, and this is terrible.
- Package manager
- Easy Install
- pip
- Many many virtual environment tools
- virtualenv
- venv
- pyenv
- pyenv-virtualenv
- etc
Finally, pipenv comes to our rescue!
Setup pipenv
Check Python version (should be 3.x)
$ python --version
Python 3.6.0
Check pip version
$ pip --version
pip 18.1
Install pipenv
brew install pipenv
Check pipenv version
$ pipenv --version
pipenv, version 2018.11.26
Success: pipenv is installed!
pipenv workflow
Install dependencies (pipenv automatically detect requirements.txtand solve the dependencies)
$ cd [YOUR PROJECT]
$ pipenv install
Pipfile and Pipfile.lock should be created
- Pipfile: list of all installed packages
(Pipfile example)
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
jupyter = "*"
seaborn = "*"
numpy = "*"
pandas = "*"
plotly = "*"
scipy = "*"
six = "*"
sklearn = "*"
pyspark = "*"
flake8 = "*"
autopep8 = "*"
Flask = "*"
[requires]
python_version = "3.7"
- Pipfile.lock: maintain a proper dependencies and sub packages with version
(Pipfile.lock example)
{
"_meta": {
"hash": {
"sha256": "xxxxxxxxxxxxxxxxxxxxxxxx"
},
"pipfile-spec": 6,
"requires": {
"python_version": "3.7"
},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"appnope": {
"hashes": [
"sha256: xxxxxxxxxxxxxxxxxxxxxxxx",
"sha256: xxxxxxxxxxxxxxxxxxxxxxxx"
],
"markers": "sys_platform == 'darwin'",
"version": "==0.1.0"
},
"attrs": {
"hashes": [
"sha256: xxxxxxxxxxxxxxxxxxxxxxxx",
"sha256: xxxxxxxxxxxxxxxxxxxxxxxx"
],
"version": "==19.1.0"
},
"autopep8": {
"hashes": [
"sha256: xxxxxxxxxxxxxxxxxxxxxxxx"
],
"index": "pypi",
"version": "==1.4.4"
...
Now, run your python script (Installed packages are available when you use pipenv run)
$ pipenv run python [YOUR PYTHON MAIN SCRIPT]
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✔ Success!
Updated Pipfile.lock (980232)!
Installing dependencies from Pipfile.lock (980232)…
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 73/73 — 00:01:22
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
Thanks for reading!
If you enjoyed this article, feel free to hit that clap button 👏.
References
- Pipenv: Python Dev Workflow for Humans: Official documentation
- Why Python devs should use Pipenv: Why pipenv?
- The Hitchhikers guide to pipenv: pipenv guide
- Python Dependency Management with pipenv: pipenv guide
- Five Myths About Pipenv
Top comments (9)
My team has struggled a lot with Pipenv. We use Python in a ton of different environments and have come across several weaknesses that should be considered before jumping in.
It is slow. Talking 20+ minutes to sync (install from lock, no resolving necessary) a large web app.
When you add a new dependency, it also always updates your other dependencies, which means adding takes a long time and can break things you weren’t intending to touch.
They’re recommended way of doing things is to declare dependencies with * version, meaning any time you update it will grab whatever the latest is, even when not compatible. This becomes quite frustrating, especially with how long it takes to fix it. If you’re going to use Pipenv I strongly recommend declaring dependencies like
flask = "~=1"which means use a version compatible with 1 (semantic versioning rules)Private repositories are weakly supported. You can do it by declaring an additional source in your Pipfile but if you have credentials protecting your repo (which you should), you’ll need to store them in environment variables and access the like
${USERNAME}in your Pipfile (which doesn’t always work).Lately I’ve been working with Poetry which solves all of those problems, though had a few of its own. The good news is they’re actively working on building features to cover all of the flaws I’ve found so far.
Thanks for your comment, Dylan.
I've just found pipenv recently and tried it just today.
I'm really glad to know some potential issues regarding pipenv from you.
I'll try Poerty and write a post about it :)
You should! Great post by the way, you should totally do posts on some more advanced usages as you try it. Information tends to be scattered on the topic and dependency management is important!
Which issues you found with Poetry? There are so many dependency managers out there. I settled with Pipenv for now because it's better than nothing but the issues you listed are not news to me.
Getting dependency management right in Python seems quite an ordeal!
Maybe I should do a whole post on my experience with different dependency systems for Python 😅.
The TL;DR is this:
For Python-pure libraries it works great, best solution I've found, but If you have some C-code or something in your lib, it won't work yet (hopefully eventually)
For deploying an app it's a lot trickier, Poetry basically only gives you the option of building a wheel and then installing that in whatever environment you're deploying to. If that works for what you're doing, great! Otherwise you end up having to export a requirements.txt file (Alpha version of Poetry only) to deploy the old fashioned way.
There are future features planned for integrating some deployment options, but they don't exist yet. Poetry is still very young which means it comes with the usual drawbacks of young projects.
Got it! I'l keep an eye on it for when it matures :D Please write the post!
Thanks, your discussion helps me a lot to think of what package manager I should use.
By the way, any thought on anaconda? I've been struggling with using both anaconda and pip, and they messed me up :(
I have limited experience with anaconda. It ended up causing me more trouble than it was worth. I guess all the bundled packages they include are useful for data scientists but they were unnecessary for me.
Personally I use pyenv on Unix-like systems for installing/managing more than one version of Python. If you try out pyenv make sure to have this page open, I’ve needed to run a command from it on every install (particularly on macOS).
On Windows the normal Python installer comes with the handy “py” launcher which has always been good enough for me.
I can’t say I’ve ever missed any package that conda has and I don’t. But again, I’m not a data scientist.
Pyenv works well, I have 4 different versions of Python installed on my Mac, plus the system python plus I think one from brew, never had a conflict. It's Pipenv that's slow sometimes