Console Messages in Test Automation Reports: Implementation and Advantages

The web console, also known as f12, hacking screen and many other names that made you believe that you were a web developer by changing HTML tags to place your name in a big bold title.

With that said, for this tutorial I want to highlight the selenium feature to capture those web console messages that can get us a good idea of what could be causing an error in our AUT(Application Under Test), in case that we are pretty sure that the selector of our web element is not broken, changed or not visible by our web driver.

Things we will need to make this work:

1. Python
2. Selenium web driver
3. Pytest
4. Allure report

First we will setup a couple of test to work on. I'll be using the pulper a web page design for testing practice. You can click on the link to set it up in case you want to use it, but any website can do the job for this tutorial.

import unittest
from pageobject.pagefactory import Factory

class TestHomePage(unittest.TestCase):

    def setUp(self):
        Factory.homepage.go_to_homepage()

    def test_pulp_homepage_title(self):
        assert "Pulp App Main Menu" == Factory.homepage.pulp_homepage_title.text

    def test_table_link(self):
        Factory.homepage.click_table_link()
        assert "Table of Books" == Factory.tablepage.table_page_title.text

    def test_list_link(self):
        Factory.homepage.click_link_list()
        assert "List of Books" == Factory.listpage.list_page_title.text

    def test_FAQs_link(self):
        Factory.homepage.click_link_faq()
        assert "FAQs" in Factory.faqpage.faq_page_title.text

Once we have this test going, lets add the allure package by importing it and adding allure title and step decorators.

Test class

import allure
import unittest
from pageobject.pagefactory import Factory

class TestHomePage(unittest.TestCase):

    def setUp(self):
        Factory.homepage.go_to_homepage()

    @allure.title("Test homepage title")
    def test_pulp_homepage_title(self):
        assert "Pulp App Main Menu" == Factory.homepage.pulp_homepage_title.text

Page Object class

    @allure.step("Click table link")
    def click_table_link(self):
        self.link_table.click()

    @allure.step("Click link list")
    def click_link_list(self):
        self.link_list.click()

    @web_driver
    @allure.step("Click link faq")
    def click_link_faq(self, driver):
        self.link_faq.click()
        logs = driver.get_log('browser')
        print(logs)

Now with everything set, our allure report should look something like this once we run the tests in our CLI.

$ pytest tests/ -v --alluredir=test-report/

Now I'm going to focus on our FAQs test. Notice that we have a function defined to click on the FAQs link, I've done this in order to capture the step on the allure report and to add the console logs to see what happens backstage.

Test class

    @allure.title("Test FAQs link")
    def test_FAQs_link(self):
        Factory.homepage.click_link_faq()
        assert "FAQs" in Factory.faqpage.faq_page_title.text

Page Object class

    @web_driver
    @allure.step("Click link faq")
    def click_link_faq(self, driver):
        self.link_faq.click()
        logs = driver.get_log('browser')
        print(logs)

In python we get the console logs through the get_log() function giving browser as an argument to get the browser JS logs that are displayed in the web console. There are more arguments that can be passed to our get_log() function but for this tutorial we will be focusing on this one.

After we run this test again in our allure report, we should get a section called "stdout" under the steps of this particular test.

And with this we finish the implementation of the console messages into our allure report.

With the previous implementation of web console message into our test report we get a couple of advantages to catch potential issues into our application.

• Robust more complete test report: We can give to our developer a better perspective of the test results, a way to give a much better idea of the bug and it's solution.

• Proactive testing: We get ahead of potential issues by getting Network messages, security warnings, JavaScript warnings and API messages.

With this we conclude our tutorial, I hope it is helpful to you and your team.