Installing the Firefox web driver on Linux for selenium

Installing the Firefox web driver on Linux for selenium

I found this cool Python tutorial for beginners on web scraping using the selenium module. However, in the book, there is a part where the author says "If you encounter the error message: "'geckodriver' executable needs to be in PATH" you will have to manually install the webdriver to get selenium working".

I was one of the lucky ones that got to learn how to install the driver manually, hence this guide was born.

In this guide, I will explain the steps I took to manually install the Firefox webdriver on Linux(Debian) and configure the PATH to get selenium working correctly.

Specifically, I will explain the following steps:

1. Downloading the Firefox web driver
2. Understanding the PATH environmental variable
3. Configuring the webdriver to work with selenium
4. Testing selenium with Python.

Assumptions:

1. You have the Firefox web browser installed. (If you don’t you can download it for free from https://getfirefox.com/.)
2. You have Python installed.
3. You have selenium installed. (You can install selenium by running pip install --user selenium from the command line terminal.)

Downloading the Firefox webdriver

Go to https://github.com/mozilla/geckodriver/releases and scroll down to assets. There you will find the gecko driver for the different operating systems.
Click on "geckodriver-v0.27.0-linux64.tar.gz" to download the Linux 64bit driver. Choose the directory where you want to save the zipped file and start the download.

Understanding the PATH environmental variable

In order for selenium to execute the webdriver successfully, it needs to know where the executable file “geckodriver” is located. To accomplish this there is an environmental variable called PATH in which your program looks for the address of executable files.

PATH is an environmental variable in Linux that tells the shell in which directories to search for executable files in response to commands given through the command line or shell scripts. This is also the reason you can type a command like ls (list) without having to specify its directory /bin/ls.

PATH (upper case letters) is different from path (lower case letters) where the latter refers to the address of a file or directory. ie./home/user/file.txt

Next, I will explain two ways that you can set up the webdriver to work with the PATH variable. The first is a permanent solution where you place the executable within a directory already in the PATH variable. The second is a temporary solution where you add the directory of the webdriver executable to the PATH variable. With the latter, the PATH resets when a new session is started.

Unzip to the directory in PATH

The easiest method is to unzip the geckodriver.tar into the /usr/local/bin directory, which is already in the PATH by default. To achieve this enter the following command into your command line from within the directory where the geckodriver.tar file is located. Note the name of your file should match that of which you downloaded.

tar -C /usr/local/bin/ -xvf geckodriver-v0.27.0-linux64.tar

This places an unzipped copy within the /usr/local/bin folder that is already listed in the PATH variable. You will now be able to run the geckodriver command to test it out.

This change will remain in place even after the session is restarted.

Add the chosen geckodriver directory to PATH

To temporarily add the geckodriver, add the directory where the geckodriver executable is located to the PATH variable. Enter the following command where YourDirectory is the directory of the geckodriver executable:

export PATH=$PATH:/YourDirectory

This adds the directory where the executable is located to the PATH variable. You can check this by looking at the PATH variable values. Enter the following command:

echo $PATH

This will show all the directories that are currently listed within the PATH variable separated by semicolons. You will now also see your geckodriver directory listed there and will be able to run the geckodriver command to test it out.

Open a web page with Python

Now you will be able to use selenium from within Python. You can test it out by opening a web page from the interactive Python shell. Type python3 to open the python interactive shell then enter the following commands:

>>> from selenium import webdriver
>>> browser = webdriver.Firefox()
>>> browser.get('https://beginnerpythonprojects.com/')

Note webdriver.Firefox() opens up the Firefox web browser and the next line opens up the webpage https://beginnerpythonprojects.com/