Welcome to the wild world of Python dev. Where nothing is consistent and there is no such thing as as consensus.
Don't worry I'll be your guide.
Background
I quietly released pyquilted which generates nice looking resumes from Yaml in PDF. I learned a lot about setting up a python dev environment and packaging an app to distribute on Pypi (pip) and now I am sharing that knowledge.
You can find out more about quilted resumes here.
Unittests
Running test cases were for some reason harder than it should be. Follow these steps to make your life easier.
Create a directory called 'test' in your project root.
Name all your tests 'test_*'.
Put an __init__.py so that the directory get's recognized as a package.
running all tests on Python 2 and 3
python -m unittest discover
running one test
python -m unittest test.test_name
Venv vs VirtualEnv vs PipEnv
These are all ways to isolate your python pip environment so you do not pollute your main system python. Uninstalling a package and it's dependecies is a pain in pip. This avoids this by allowing you to easily delete and create new envs.
But which one do you go with?
Venv vs VirtualEnv vs PipEnv
Just go with venv. It comes standard with python and is officially recommended post 3.3+. It also has less gotchas than virtualenv and I don't want to be gotten.
to create a new venv folder
python -m venv venv
to load the environment
source venv/bin/activate
turn off the venv
deactivate
Pipvenv is nice, but it's not standard and is based on virtualenv.
Python 2.7 and Venv
Venv doesn't support python 2.7, bummer.
Simple solution
pip install py2venv
This installs virutalenv for you and creates a wrapper around virtualenv. Meaning you can use the venv command. Great!
Where to put your venv folder?
If you are just running one venv per project, put it in the same directory as the project root and name the folder 'venv'. Make sure in your .gitignore you have added 'venv'—on github and gitlab this is added for you.
one venv environment per project
project_root/venv
The issues comes when you want different venv environments for different python versions you are testing. Here's a few options
multiple venv per project
project_root/venv/{{python_version}}
alternative, venv in home folder
~/venv/{{project_name}}_{{python_version}}
setup.py
Setup.py is like a makefile for your app. It's based off of setuptools. It normally comes pre-installed. If not you can install it like so
pip install setuptools
The documentation can be found here. Some parts are either confusing or not working as expected. I noted the parts here.
You can see an example of my setup.py in the pyquilted source
Command line entry point
You want to run your app in the terminal using your app's sweet name. This was a lot tricker than it should be. After endless searching, this method below worked for me.
In your project root, create a file named main.py. Have one function named main that calls your run function. An example here.
Add the following into your setup.py
entry_points={
'console_scripts': ['command=pacakge.main:main']
}
Where 'command' is the command you want to use. 'package' is your package name.
For example my pyquilted entry point
'console_scripts': ['pyquilted=pyquilted.main:main']
Markdown readme
Pypi now supports markdown. Use a with open file handler to set your readme doc to a variable.
long_description=readme,
long_description_content_type='text/markdown'
Including non py files
There are two ways to add files to either your site_packages for data or your source package for test cases and such.
package data
Use the below format for adding data files to your site_packages.
package_data={
'package_name': ['directory/*.pattern']
},
include_package_data=True
MANIFEST.in
This includes files in your source pacakge that are not in your site_packages folder. The format is a little weird.
include file # include single file
recursive-include directory pattern # include a directory and files
requirements.txt
Skip it. It's not really necessary, but simple to make.
Enter Pyenv
Still crying over the disaster that was migrating from Python 2 to 3. Me too, join the club. We stil have to support Python 2 and test other versions of 3 as well. How to solve multiple python versions? We could learn Vagrant or Docker. Nah that's way overkill.
Solution Pyenv.
Install Pyenv by following instructions here pyenv.
list installable python versions
pyenv install --list
install version
pyenv install 2.7.15
list versions installed
pyenv versions
switch to python version
pyenv global 2.7.15
Debian, Ubuntu and others
On Debian, Ubuntu and other distros, you will get an error about ssl when trying to build older versions of python. The issue is that libssl 1.1 is not compatble with Python < 3.3. The solution is to download the dev headers for libssl 1.0 and link them manually when running pyenv install, per this thread.
CFLAGS="-I/path_to/libssl1.0-dev/include -I/path_to/libssl1.0-dev/include/x86_64-linux-gnu" \
LDFLAGS="-L/path_to/libssl1.0-dev/lib/x86_64-linux-gnu" pyenv install 2.7.15
Eggs everywhere
To stop python from creating eggs that you accidently upload to Pypi when you are testing.
Use the below command
pip install .
instead of this command
python setup.py install
Building and Wheels
Honestly wheel files are not necessary. The difference in install time between source and binary is like a few seconds. However it's really easy to build binary wheels so why not.
make sure you have wheel installed
pip install wheel
run this to build your packages
python setup.py sdist bdist_wheel
Finally upload to Pypi
Create an account on Pypi.org. Then install twine which is the official pypi app
pip install twine
Upload with this command
twine upload --skip-existing dist/*
The key is skip existing files, otherwise you get errors when uploading the dist folder with existing files.
Congrats!
Congrats you beat the final boss! Pat yourself on the back for finally deciphering python dev environment.
Top comments (0)