Web automation is one of the best ways companies can test a product in development, especially the app's functionalities, such as clicking, scrolling, and other actions.
Essentially, web automation is about mimicking human action as it is crucial to ensure the software works across all device types of users.
In this article, we will learn how to use Selenium as an automation tool to test a website with Python and automate the entire process without using a mouse or keyboard on the browser. Selenium doesn’t work with Python alone but with many other programming languages.
Selenium allows us to browse or use a browser without a human involved and automate processes through code, such as typing into a user input and interacting with the website. For example, automating form submissions with Selenium is possible. Selenium does everything all by itself without a single click from a human.
Before Selenium can carry out any action through code, we need the packages and tools to be able to run browser automation and make it seamless.
This section, we install the Selenium package and the WebDriver executable file. First, check the version of the Chrome browser so that it is equivalent to the same version of the WebDriver exec file.
The following needs to be installed on our local machine:
The Chrome browser. Download it here.
Download the latest stable release of the ChromeDriver. On the ChromeDriver web page, check to confirm the version of Chrome matches the version of the ChromeDriver.
Now, run this command to install Selenium in your terminal:
pip install selenium
To begin, create a new folder containing the file and the downloaded ChromeDriver exec files. Selenium requires the driver to interface with the chosen browser, which in this tutorial, we are using the Chrome browser.
It is essential to place the ChromeDriver in the same directory as the Python file.
The directory for the project should look like this:
. ├── automation.py └── chromedriver
Another way to use the ChromeDriver in this project is to find the path location of the file and copy it, which should look like this:
/Users/<computer user name>/Desktop/chromedriver
C:\Users\<computer user name\Desktop\chromdriver
The WebDriver is the bridge that bridges the Selenium code to work with the Chrome browser. The Chrome browser provides the bridge. Without it, automation will not be possible.
Next, let’s copy and paste the following code into the
from selenium import webdriver from selenium.webdriver.chrome.service import Service from selenium.webdriver.common.by import By from selenium.webdriver.common.keys import Keys chrome_driver_path = './chromedriver' service = Service(chrome_drive_path) driver = webdriver.Chrome(service=service) url = "https://en.wikipedia.org/wiki/Main_Page" driver.get(url) article_count = driver.find_element(By.CSS_SELECTOR, "#articlecount a") print(article_count.text) drive.close()
The following occurs in the code snippet above as we will use Selenium to navigate Wikipedia:
- Import the function webdriver from the selenium module
selenium.webdrivermodule provides all the WebDriver implementations, and the Service object is for handling the browser driver
Byclass is used to locate elements within the document of a web page using its CSS classes, ids, and so on
- The Keys class simulates the actions from the keypress of the keyboard like the RETURN / Enter, F1, Alt, and so on
- Assign the chromedriver exec file to a variable
- Call the executable path of the driver to the service object
- Next is to instantiate the Chrome browser with the
webdriver.Chrome()with the service argument
- Define the url of the web page to check
- Using the
.get()method will navigate to the page given by the url
On the Wikipedia page, open the inspect element to access the elements we will use to get the details from the page for automation.
Now to the task of finding the article count. WebDriver offers us a way to do so by targeting the element using the
find_element method and using the
By.CSS_SELECTOR as the first parameter with the selector name,
Before we run the Python program, the script has the method,
driver.close(), that closes the browser window after running the test.
Run the program with the command:
The result from the terminal should display the article count, which shows the exact figure on the web page, 6,545,457.
PS: This value of the article count won't remain the same as its value would have increased.
From what we have done so far, it is evident that Selenium navigates a website without the use or click of a mouse.
Now, let’s simulate an action where the browser reacts to a click of a link on the page.
automation.py with these lines of code:
... contact_us = driver.find_element(By.LINK_TEXT, "Contact us") contact_us.click()
With this code snippet, we are finding the element to the link text, “Contact us” with the
By.LINK_TEXT. After locating the elements, attach the
contact_us to an action with
Again, run the command:
The command above gives the result of the contact us page.
There are so many use cases with Selenium. Searching a web page can be achieved with automation, which gives you the result for a search word.
Still, in the same file,
automation.py, update the file with this code:
search = driver.find_element(By.NAME, "search") search.send_keys("React", Keys.ENTER)
The code above does a similar thing to the previous by using the
find_element method with the input's name attribute, search.
send_keys simulate pressing the ENTER key, searching the keyword React.
The result of the page should look like this:
This article is an overview of how to use Selenium in Python to automate the web, especially for professional developers who want to know if there are errors in the code and how to fix them.
The possibilities of using Selenium with Python for web automation are endless. Some use cases include filling out an online form, applying for LinkedIn jobs, playing a clicker game, and so on. Do you know other use cases for using Selenium?
Kindly share your favorite use case in the comments below.