Setting up and Using a Python Virtual Environment

• Introduction
• What is a Python Virtual Environment
• When to use a Virtual Environment with Python
• Setting up a Python Virtual Environment
  • Prerequisites
  • Checking the Current Package List
  • Setting up a GitHub Repository
  • Clone the GitHub Repository
  • Creating a Virtual Environment
  • Activate and Deactivate a Virtual Environment
  • Install a Package in a Virtual Environment
  • Remove a Package in a Virtual Environment
  • Python Sample Script
  • Pip Freeze and the Requirements.txt file
  • Pushing the Project to GitHub
  • Deploying to a New Folder
• References

Introduction

In this article I will cover the steps to setup a python virtual environment for a project on Linux or macOS.

By the end of this article, the topics covered will be:

• Setting up a virtual environment with python.
• Creating a simple python project (making an HTTP request to a website) that uses a virtual environment.
• Push the project to a GitHub repository.
• Deploying the project from GitHub to a new folder and deploying a new virtual environment

What is a Python Virtual Environment

When developing an application / program with python, typically there are a set of dependencies in the form of packages that a developer would need to install. Some of these are included with python and others are installed with a tool such as pip.

A python virtual environment is used to:

• Isolate dependencies that are used by a project.
  • This way it keeps the packages that are installed at the system level down to a minimum which keeps it clean, tidy and .
• Specify a particular version of python to use.
• Make the project transportable between different systems.

Now, Why would you need to specify a version of python to use you may ask? The reason is that some packages may only work with certain versions of python.

For example, say a new version of python is released and then installed on a system. A module may not work on that version as it may not have been updated to work with it and as a result could cause unexpected issues.

Another reason is that a vendor may state that a particular version of python is required for their solution to run and the system it will be running on may have multiple versions of python installed for different products or projects.

When to use a Virtual Environment with Python

It is recommended to use a virtual environment for every project. Now, there are some cases where this would be a waste of time, such as print("Hello World!") but they are few and far between. Personally, if you get used to using them early when learning python, the habit will form and it will be like second nature to you.

Setting up a Python Virtual Environment

Prerequisites

Before you can use python virtual environments, you will need:

• A version of python installed, along with pip that are at version 3.5 or later.
• A terminal application, such as the one built into your O/S or a third-party one, such as iTerm2.
• A GitHub account that you have logged into at the terminal level.
• The GitHub and git command line tools installed, with both being logged into.

For reference, python 3.10.4 was used for this article.

Checking the Current Package List

The first thing to do is to get a baseline of what packages are currently installed at the operating system-level. What this means is what python packages are installed that can be used without the use of a virtual environment. To see what packages are installed, run the following command:

pip list

The output will be similar to the below (your list will likely be different)

Package            Version
------------------ ---------
boto3              1.23.7
botocore           1.26.7
certifi            2022.6.15
charset-normalizer 2.0.12
distlib            0.3.4
filelock           3.7.0
idna               3.3
jmespath           1.0.0
pip                22.1.1
platformdirs       2.5.2
python-dateutil    2.8.2
requests           2.28.0
s3transfer         0.5.2
setuptools         60.10.0
six                1.16.0
urllib3            1.26.9
wheel              0.37.1

This is the list of installed packages that are available at the system level.

Setting up a GitHub Repository

The next step is to create a GitHub repository to store the project in. Open up your terminal application and run the following command, changing {repo-name} to whatever works best for you:

gh repo create {repo-name} \
                --public \
                --gitignore python

This will create a new public repository with a gitignore file that GitHub has pre-populated with a number of default files and folders that are typically used with python that do not need to be stored in the repository. The virtual environment folder name that we will use later is listed in this file so you won't be pushing any packages that are installed later to your GitHub repository.

Clone the GitHub Repository

Now that you have a GitHub repository setup, you need to clone it to a folder on your computer. You can do this a number of ways but for this guide, I will list two, one that uses the gh command and another that uses the git command.

First, go to a location on your computer where you would like to store the project files. For this example I will use the Downloads folder (this is on macOS so if you are on Linux, it might be located elsewhere).

cd ~/Downloads

Next, you need to clone the repository from GitHub. You can use one of the two commands below:

Using the gh command (change {username} to your GitHub username and {repo-name} to your repositories name):

gh repo clone {username}/{repo-name}

Using the git command (change {username} to your GitHub username and {repo-name} to your repositories name):

git clone https://github.com/{username}/{repo-name}.git

Once the repository has been cloned, you can go into the directory it created, which will be the same as the repositories name:

cd {repo-name}

Creating a Virtual Environment

Now that you have the repository stored locally, you can now create a virtual environment for your python project. To create one, run the following command:

python3 -m venv venv

The command will create a new folder called venv. Now, the second venv in the above command is the name of the folder it will store everything relating to the virtual environment. You can change this but it is a typical name that is used in the industry to clearly show what it is so it is best to leave it as is.

This virtual environment will use the version of python that is the default on your operating system. If you wish to use a different version, you can do so by specifying the version number in the python command. For example:

python3.9 -m venv venv

When you use the project on another system, you will need to ensure it has the same version of python installed and you specify that version when you recreate the virtual environment.

Activate and Deactivate a Virtual Environment

Once the virtual environment has been created, it now needs to be activated. If you don't activate it, then when you install a package it will be installed at the system level.

To activate it, run the following command:

source ./venv/bin/activate

Now, depending upon how (or if) you have customised your terminal, you should see venv show up on your command prompt. On a default bash or zsh terminal, it should look like the below.

bash:

(venv) bash-5.1$ _

zsh:

(venv) user@My-MacBook-Pro project _

The virtual environment is now active. To check this, you can run the pip list command again. You will notice that the list is different from before. For a typical new virtual environment, you should see pip and setuptools listed.

If you need to exit the virtual environment, run deactivate at the command line. You will see that (venv) is no longer shown in your command prompt. You can go reactivate it by running the the source ./venv/bin/activate command again.

Also, if you close the terminal, this will also deactivate the virtual environment.

For now, please ensure the virtual environment is active.

Install a Package in a Virtual Environment

To install a package into a python virtual environment, it is a very long and complex process. Only kidding! It is no different from installing a package as you would normally. Run the below command which will install the requests and zipp packages using pip:

pip install requests zipp -y

The -y flag will install the packages without asking you if you want to proceed.

Once the installation is finished, run pip list again and you should see the two packages installed, along with some others that were installed as dependencies for the requests and zipp packages. It should look similar to the below:

(venv) bash-5.1$ pip list
Package            Version
------------------ ---------
certifi            2022.6.15
charset-normalizer 2.0.12
idna               3.3
pip                22.0.4
requests           2.28.0
setuptools         60.10.0
urllib3            1.26.9
zipp               3.8.0

The packages that are listed above will be isolated from the any other python virtual environment and the system. This means that if you didn't use the virtual environment and tried to use the requests package / module, it will not work unless you install it into that environment.

Remove a Package in a Virtual Environment

Removing a package is very much the same as installing a package but you need to use uninstall instead. Run the following command to uninstall the zipp package as it will not be used:

pip uninstall zipp -y

Once the uninstall is finished, run pip list again and you should see the zipp package in no longer listed.

(venv) bash-5.1$ pip list

Package            Version
------------------ ---------
certifi            2022.6.15
charset-normalizer 2.0.12
idna               3.3
pip                22.0.4
requests           2.28.0
setuptools         60.10.0
urllib3            1.26.9

Python Sample Script

Now that the requests package installed, let's use it. But what does it do?

The requests package is used to make HTTP / HTTPS requests to a website and bring back the contents of the page as raw HTML. It can do a number of other things but you will be using it for just making a simple HTTP / HTTPS request.

First, run the nano text editor and create a file called main.py:

nano main.py

Once you are in nano, copy and paste the below into nano (you can change the contents of the url variable to a different site if you want):

# --- Import the required modules:
from requests import request
from pprint import pprint

# --- Define the variables used
url = "https://www.mirrorservice.org/"
get_request = request(url=url, method="GET")

# --- Show the HTML of the page
pprint(get_request.text)

# --- Show the status code of the request (200 is OK)
print(f"\nHTTP Response Code Is: {get_request.status_code}")

To save the file, hold CTRL and press x. Press y when asked about saving the buffer. Press enter when asked for the file name.

This will now take you out of nano and back to the command prompt.

To see what the application does, run the following command:

python3 main.py

The result should be a bunch of HTML and a line at the bottom that states HTTP Response Code Is: 200.

Pip Freeze and the Requirements.txt file

So far, you have created a virtual environment, installed the requests and zipp packages, uninstalled the zipp package and created a script to use the requests package to get the HTML contents of a webpage and its HTTP response code.

The next step is to begin making the application transportable to another system. To start with, you need to get a list of all the packages that are installed in the virtual environment and put them in a text file.

The reason for doing this is that we don't copy the virtual environment. Instead you recreate it on the destination system. One reason is that adding a large number of packages can take a long time to push up to GitHub and can also cause soft capacity limit warnings when pushing to GitHub.

To get the list of installed packages and put them into a text file, you will use the pip freeze command. To create the file, run the below command:

pip freeze > requirements.txt

You will see a new file called requirements.txt appear. If you open it, it will look similar to the below:

certifi==2022.6.15
charset-normalizer==2.0.12
idna==3.3
requests==2.28.0
urllib3==1.26.9

The file contains the name and version of each package that is installed in the virtual environment. This will allow you to deploy the project on another system and use those exact same versions of the packages when you install them. More on that part later.

During a typical projects development, you may need to add additional packages after you have created the requirements.txt file. As a good practice, I would recommend deleting the requirements.txt file and creating it again when you install new packages so that there are no missing dependencies later on.

Pushing the Project to GitHub

With the project completed and the requirements.txt file created, the files can now be pushed to your GitHub repository. To do that, run the following commands:

git add --all
git commit -m "Version one of project completed."
git push

The files will now appear in your GitHub repository.

Deploying to a New Folder

For the final step, you will perform a mock deployment to another system by using a different folder on you computer. The steps to do this are basically what you have already done with some minor tweaks:

• Go to the location where you want to store the project files. Create a new folder if needed.
• Clone the GitHub repository, the same way as you did previously.
• Go into that folder.
• Create a new virtual environment, the same way as you did before.
• Activate the virtual environment, again, the same way as you did before.

The only difference this time is rather than using pip install {package-1} {package-2} {package-3}, you can use the requirements.txt file to bulk-install them with version of each package that was to develop the project with. To do this, run the below command:

pip install -r requirements.txt

If you then run pip list, you will see all the packages installed as they were previously. You can then run python3 main.py again, with the results being the same (although the page may have been updated by the site maintainer).

If you were deploying to another system, the only thing you would do differently is to first connect to the system, typically via SSH on Linux and then do the above steps. Make sure that you have the version of python installed on that system that was used to develop the project.

This concludes how to use virtual environments with python.

References

References used for parts of this article:

Python venv documentation