Simplifying Automation with Visual Validation

When automating scenarios, we have to be very careful to capture all of the various verification points that a tester would process when executing a test. I’ve written about this issue in Adding a Peripheral View to Test Automation, and stressed how important it is to make sure we script all necessary assertions into a given scenario. But I’ve found a better way to do this! Visual UI testing.

As an automation engineer, I feel immense pressure to think outside of the box and ensure no assertions are omitted. Visual validation relieves this stress and makes my automation script a lot faster and simpler.

Let me show you what I mean.

Here’s an application that I’ve built for some of my UI automation workshops. It is a simple grid showing books on test automation. By searching in the textfield, the user can filter the books to just the ones that match the query.

WITHOUT VISUAL VALIDATION

Without visual validation, I would write a scenario to make sure the book(s) that I’m searching for are the only ones visible after my search completes. In this particular scenario, I wrote the following to make sure that when I search for “Automation”, the two books that I expect are the ones that are shown.

  @Test
  public void testSearchByPartialTitle() {
    String title1 = "Test Automation in the Real World";
    String title2 = "Experiences of Test Automation";

    page.search("Automation");

    assertTrue("Book not found: " + title1, page.isBookVisible(title1));
    assertTrue("Book not found: " + title2, page.isBookVisible(title2));
    assertEquals("Number of visible books is incorrect", 2, page.getNumberOfVisibleBooks());
  }

To support this scenario, I needed to add two methods to the Page Object class in my framework: isBookVisible and getNumberOfBooks (plus an additional supporting method as well as element locators):

  public boolean isBookVisible(String title){
    List<WebElement> books = findVisibleBooks();

    for(WebElement book : books) {
      if(title.equalsIgnoreCase(book.findElement(titleAttribute).getText())){
        return true;
      }
    }

    return false;
  }

  public int getNumberOfVisibleBooks() {
    return findVisibleBooks().size();
  }

  private List<WebElement> findVisibleBooks() {
    return driver.findElements(visibleBooks);
  }

Even with all of this code, I am only validating that the title of the book is correct. There’s so much more on the screen that could be wrong like the authors, price, book cover…and that doesn’t even begin to cover the overall look and feel!

WITH VISUAL VALIDATION

To fully verify the book’s properties, our methods would need to be a lot more robust. But instead of adding more code, I can actually remove code and gain even more coverage with visual validation.

Here’s my revised test method. Notice that I’ve replaced the three lines of assertions from above with three lines of visual validation, and with this swap, I get much more coverage for my test. This swap also eliminates the need for the Page Object code shown above, so I no longer need to maintain that. That’s 19 lines of code plus the 2 lines of element locators that I can delete. Now, this scenario verifies the entire page, ensuring not only that my books (and all of their properties) appear after search, but that the look and feel is perfect as well.

  @Test
  public void testSearchByPartialTitle() {
    page.search("Automation");

    eyes.open(driver, "Automation Bookstore",  "testSearchByPartialTitle");
    eyes.checkWindow();
    eyes.close();
  }

I can even make this test method smaller by abstracting out the visual validation code, especially since this will be reused across a bunch of my tests. In a Base test class, I can move the three lines of visual validation code into a utility method. This way, any test class that inherits from this class will have access to this:

   protected void validateWindow() {
        eyes.open(driver, "Automation Bookstore",  Thread.currentThread().getStackTrace()[2].getMethodName());
        eyes.checkWindow();
        eyes.close();
    }

Then in the tests themselves, we are down to 5 lines of code, with the actual body only being 2 lines!

  @Test
  public void testSearchByPartialTitle(){
    page.search("Automation");
    validateWindow();
  }

This works beautifully by covering much more validation – making sure everything we want to be there is actually there, making sure nothing we don’t want to be there is not, and ensuring the overall look and feel of the page is what is expected. All with less code and less stress! What’s also great about this is it doesn’t have to be a full replacement to the functional validations. I can easily keep any assertions I need (e.g. backend validation) and still use the visual validation for the rest. I can essentially have it all!

_{Originally posted on angiejones.tech}