dev_neil_a

Posted on Nov 5

Creating and Using Python Virtual Environments

#python #beginners #webdev #coding

YouTube Video
Introduction
What is a Python Virtual Environment
When to use a Virtual Environment with Python
Setting up a Python Virtual Environment
References

YouTube Video

If you would prefer to watch a video of this article, there is a video available on YouTube below:

Introduction

Hello and welcome. In this article I'll be going over how to setup a Python virtual environment for a project on Linux or macOS.

By the end of this article, the topics that will be covered are:

What a virtual environment is
Setting up a virtual environment with Python
Creating a simple Python program in a virtual environment that makes an HTTP request to a web page and displays the raw contents of the page

What is a Python Virtual Environment

So what is a virtual environment in the Python world? Well, Typically, with a Python project,
there are a set of dependencies in the form of packages that a developer would need to install and use.

Some of these are included with Python and others are installed with a tool such as pip.

A virtual environment is mainly used to:

Isolate dependencies that are used in a project
Removes the possibility of conflicts with programs that were developed with an older version of a package
Make the project transportable between different systems
- A list of packages can be exported to a file to make setting it up on another system very easy
Specify a particular version of Python and / or a package to use

Now, Why would you need to specify a version of Python to use you may ask? The reason is that some packages may only work with certain versions of Python.

For example, say a new version of Python is released and is installed on a system. A package may not work on that version as it may not have been updated to work with it and as a result, could cause unexpected issues.

I've had that happen a few times so always check that the package is supported. The easiest way to check is to search for the package on PyPi (a Python package repository) and look at the the tags in the description of the package which version is the highest supported.

For example, if you search for the requests package on PyPi, in the description (shown below in red) you can see it requires Python 3.8 or above and the highest supported version being 3.12 (as of November 2024).

Another reason is that a vendor may state that a particular version of Python is required for their solution to run and the system it will be running on may have multiple versions of Python installed for different products or projects.

When to use a Virtual Environment with Python

It is recommended to use a virtual environment for every project. Now, there are some cases where this would be a waste of time, such as print("Hello World!") but they are few and far between.

Setting up a Python Virtual Environment

Prerequisites

Before you can use Python virtual environments, you'll need:

A version of Python 3 installed, along with the pip command
A terminal application, such as the one built into your O/S or a third-party one

For reference, in this article, the version of Python used was 3.12.7 and iTerm2 was the terminal application used on macOS.

Checking the Current Package List

The first thing to do is to see what packages are currently installed at the operating system / global level. What this means is what Python packages are installed that can be used without the use of a virtual environment.

To see what packages are installed, run:

pip list

The output will be similar to the below (your list will likely be different)

Package      Version
------------ ---------
certifi      2024.8.30
cffi         1.17.1
cryptography 43.0.3
packaging    24.1
pip          24.2
pycparser    2.22
wheel        0.44.0

This is the list of installed packages that are available at the global level.

Creating a Project

Before a virtual environment can be created, a folder for storing all the files for the project needs to be created. For this project, create a folder in the Downloads folder called example and change to that directory. To do this, run:

cd Downloads
mkdir example
cd example

With the project folder setup, a virtual environment can now be created.

Creating a Virtual Environment

To create a virtual environment, there are two ways to do it. The first is with the virtualenv package and the second is to use the built-in venv module in Python. For this project, the venv module method will be used.

Next, run the following command:

python3 -m venv venv

To go over what the command does:

python3 is the python 3 program to use
-m indicates a module is to be run
The first venv is the name of the module
The second venv is the name of the folder that will be used for the virtual environment

Run the ls -lha command to see the newly created venv folder.

By default, the virtual environment will use the version of Python that is the default on your operating system. If you wish to use a different version, specify that version instead. For example, use python3.11 instead of python3 to use Python 3.11.

If you use the project on another system, you will need to ensure it has the same version of Python installed and you specify that version when you recreate the virtual environment if it isn't the default on that system.

Activate and Deactivate a Virtual Environment

Once the virtual environment has been created, it now needs to be activated. If it isn't activated it will install any packages at the system / global level.

To activate it, run the following command in the folder for the project:

source ./venv/bin/activate

Now, depending upon how (or if) you have customised your terminal, you should see venv show up on your command prompt. On a default bash or zsh terminal, it should look like the below.

bash:

(venv) bash-5.1$

zsh:

(venv) user@My-MacBook-Pro project

The virtual environment is now active. To check this, you can run the pip list command again. You will notice that the list is different from before.

If you need to exit the virtual environment, run deactivate at the command line. You will see that (venv) is no longer shown in your command prompt. You can reactivate it by running the the source ./venv/bin/activate command again.

Also, if you close the terminal, this will also deactivate the virtual environment.

For now, please ensure the virtual environment is active.

Install a Package in a Virtual Environment

To install a package into a Python virtual environment, it is no different than installing it at the global level using pip.

For this project, install the requests and zipp packages using pip by running:

pip install requests zipp

Once the installation is finished, run pip list again and you should see the two packages installed, along with some others that were installed as dependencies for the requests and zipp packages. It should look similar to the below:

(venv) bash-5.2$ pip list
Package            Version
------------------ ---------
certifi            2024.8.30
charset-normalizer 3.4.0
idna               3.10
pip                24.2
requests           2.32.3
setuptools         74.1.2
urllib3            2.2.3
zipp               3.20.2

The packages that are listed above will be isolated from the any other Python virtual environment and the system. This means that if you didn't use the virtual environment and tried to use the requests package / library, it will not work unless you install it into that environment.

Remove a Package in a Virtual Environment

Removing a package is very much the same as installing a package but you need to use uninstall instead. Run the following command to uninstall the zipp package as it will not be used:

pip uninstall zipp

Once the uninstall is finished, run pip list again and you should see the zipp package in no longer listed.

Package            Version
------------------ ---------
certifi            2024.8.30
charset-normalizer 3.4.0
idna               3.10
pip                24.2
requests           2.32.3
setuptools         74.1.2
urllib3            2.2.3

Python Example Program

Now that the requests package is installed, let's use it. But what does it do?

The requests package, if you haven't used it before, is used to make HTTP / HTTPS requests to a website and bring back the contents of the page as raw HTML. It can do a number of other things but for this project, it will be used for making a simple HTTP / HTTPS request and reading the contents of the response.

First, run the nano text editor and create a file called main.py:

nano main.py

Once nano is opened, copy and paste the below (you can change the contents of the url variable to a different site if you want):

# --- Import the required modules / libraries:
from requests import request
from pprint import pprint


# --- Define the variables used
url = "https://www.mirrorservice.org/"
get_request = request(url=url, method="GET")


# --- Show the HTML of the page
pprint(get_request.text)


# --- Show the status code of the request (200 is OK)
print(f"\nHTTP Response Code Is: {get_request.status_code}")

Next, save the file by pressing CTRL+x and then press y when asked about saving the buffer. Press enter when asked for the file name.

This will now take you out of nano and back to the command prompt.

To see what the application does, run the following command:

python3 main.py

The result should be a bunch of HTML and a line at the bottom that states HTTP Response Code Is: 200.

If the program was run without the virtual environment being active, it would throw a Module not found error as requests is not installed at the global level.

Pip Freeze and the Requirements.txt File

The next step is to begin making the program transportable to another system.

To start with, a list of all the packages that are installed in the virtual environment needs to be catalogued into a file that can be used to reinstall the packages.

The reason for doing this is that we don't copy the virtual environment. Instead you recreate it on the destination system. One reason is that adding a large number of packages can take a long time to push up to GitHub and can also cause capacity limit warnings when pushing to GitHub or another source control providers.

To get the list of installed packages and put them into a text file, use the pip freeze command in the following way:

pip freeze > requirements.txt

Run ls -lha and a new file called requirements.txt is now present. To view the contents, run more requirements.txt. It will look similar to the below:

certifi==2024.8.30
charset-normalizer==3.4.0
idna==3.10
requests==2.32.3
urllib3==2.2.3

The file contains the name and the exact version of each package that is installed in the virtual environment. This will allow for project to be deployed to another system and use those exact same versions of the packages when you install them.

During a projects development, you may need to add additional packages after you have created the requirements.txt file. As a good practice, I'd recommended that the requirements.txt file be recreated when a new package is installed so that there are no missing dependencies later on.

Deploying to a New Folder

For the final step, let's perform a mock deployment to another system by using a different folder on you computer. The steps to do this are basically what you have already done with some minor tweaks:

Open a new terminal tab or window
Go to the location where you want to store the project files
Create a new folder. Name it what you want
Go into that folder
Copy the main.py and requirements.txt file into that folder
Create a new virtual environment the same way as you did before
Activate the new virtual environment the same way as you did before

The only difference this time is rather than using pip install {package-1} {package-2} {package-3} to install the packages, use the requirements.txt file instead to bulk-install them at the same version was to develop the project with. To do this, run the below command:

pip install -r requirements.txt

The -r switch implies that pip needs to read the contents of the requirements.txt file to get a list of the packages to install.

If you then run pip list, you will see all the packages installed as they were previously. You can then run python3 main.py again, with the results being the same (although the page may have been updated by the site maintainer).

If you were deploying to another system, the only thing you would do differently is to first connect to the system, typically via SSH on Linux and then do the above steps. Make sure that you have the version of Python installed on that system that was used to develop the project.

This concludes how to setup and use a virtual environments with Python.

References

References used for parts of this article:

DEV Community