Data-Driven Testing in TestCafe (Part 1) - JSON and XML

Many teams use data-driven testing as part of their software QA process. Data-driven testing, sometimes known as table-driven testing, provides lots of benefits for testers. It allows for a clear separation of test data from the written test cases, repeatable testing with different data sets, and less code. This type of testing shines when you have lots of data to test using similar test cases.

Most test frameworks use different techniques and built-in mechanisms to help you implement data-driven testing in your test suites. Some widely-used examples are Robot Framework's data-driven style for writing test cases and integrating Apache POI to access data from Excel documents into Selenium test suites.

In this two-part series, we'll explore different ways to implement data-driven testing with TestCafe using commonly-used file formats. This article covers using JSON and XML files, and we'll talk about CSV and Excel files next week.

Refactoring test cases using data-driven testing

For the examples in this article, we'll use a test application we've used in previous articles on Dev Tester, called Airport Gap. The primary purpose of this application is to practice API testing techniques, but also help demonstrate basic end-to-end testing.

The setup for the tests shown in this article comes from a previous Dev Tester article, How to Get Started with TestCafe, which serves as an introduction to the TestCafe testing framework. If you're starting with TestCafe and want to learn about the basics, read that article first so you can have a better grasp of the examples we'll go into soon.

For this article, we're starting with a test file containing four test cases. Each test case validates that a specific user account can successfully log in to the Airport Gap application and see their account information.

import loginPageModel from "./page_models/login_page_model";

fixture("Airport Gap Login (Not data-driven)").page(
  "https://airportgap-staging.dev-tester.com/login"
);

test("User with email airportgap@dev-tester.com can log in to their account", async t => {
  await t
    .typeText(loginPageModel.emailInput, "airportgap@dev-tester.com")
    .typeText(loginPageModel.passwordInput, "airportgap123")
    .click(loginPageModel.submitButton);

  await t.expect(loginPageModel.accountHeader.exists).ok();
  await t
    .expect(loginPageModel.accountEmail.innerText)
    .contains("airportgap@dev-tester.com");
});

test("User with email testuser1@dev-tester.com can log in to their account", async t => {
  await t
    .typeText(loginPageModel.emailInput, "testuser1@dev-tester.com")
    .typeText(loginPageModel.passwordInput, "airportgaptest1")
    .click(loginPageModel.submitButton);

  await t.expect(loginPageModel.accountHeader.exists).ok();
  await t
    .expect(loginPageModel.accountEmail.innerText)
    .contains("testuser1@dev-tester.com");
});

test("User with email testuser2@dev-tester.com can log in to their account", async t => {
  await t
    .typeText(loginPageModel.emailInput, "testuser2@dev-tester.com")
    .typeText(loginPageModel.passwordInput, "airportgaptest2")
    .click(loginPageModel.submitButton);

  await t.expect(loginPageModel.accountHeader.exists).ok();
  await t
    .expect(loginPageModel.accountEmail.innerText)
    .contains("testuser2@dev-tester.com");
});

test("User with email testuser3@dev-tester.com can log in to their account", async t => {
  await t
    .typeText(loginPageModel.emailInput, "testuser3@dev-tester.com")
    .typeText(loginPageModel.passwordInput, "airportgaptest3")
    .click(loginPageModel.submitButton);

  await t.expect(loginPageModel.accountHeader.exists).ok();
  await t
    .expect(loginPageModel.accountEmail.innerText)
    .contains("testuser3@dev-tester.com");
});

Note that this example is not a particularly useful test scenario, but it's intentionally simplistic for demonstrating how data-driven testing can help organize your code. In real-world test suites, you would most likely perform additional actions or mix both correct and incorrect logins.

As you can see, there's a lot of repetitive code here, along with hard-coded information for the login details for each account. Let's begin refactoring these tests by fetching data from different file types to simplify this code using data-driven testing.

Data-driven testing using JSON files

The first file type we'll explore is a JSON file. JSON is a common file format for data and a favorite in the JavaScript world. It's an open standard, easy to build and read, and almost all modern programming languages have support to parse the format. No matter what application you're working on, there's a good chance you'll use JSON in some form.

First, we need to structure the data for use in the test code to begin refactoring our tests with multiple logins. For this example, we'll use a JSON file containing the email address and password for each test. The file is called users.json, located in a data subdirectory.

[
  {
    "email": "airportgap@dev-tester.com",
    "password": "airportgap123"
  },
  {
    "email": "testuser1@dev-tester.com",
    "password": "airportgaptest1"
  },
  {
    "email": "testuser2@dev-tester.com",
    "password": "airportgaptest2"
  },
  {
    "email": "testuser3@dev-tester.com",
    "password": "airportgaptest3"
  }
]

We could have more data here, but this information is all the test needs. Next, we have to bring this data into our test. Thankfully, it's an easy process in TestCafe because of Node.js, which TestCafe is built upon. We don't need any third-party libraries to read a JSON file in Node.js. All that's necessary to read the file is to use require. This way, the JSON file gets imported as a JavaScript object for easy accessibility inside the test.

After pulling in the JSON data from the file, we'll iterate through the object, creating a test case for each record. We also make sure to name our test cases uniquely using template strings to keep track of the results after executing the tests.

In the end, we'll have a data-driven test that looks like this:

import loginPageModel from "./page_models/login_page_model";

// Import the user data from the JSON file.
const userData = require("./data/users.json");

fixture("Airport Gap Login (JSON)").page(
  "https://airportgap-staging.dev-tester.com/login"
);

// Iterate through each record in the JSON file
// and create a test case with the information.
userData.forEach(user => {
  test(`User with email ${user.email} can log in to their account`, async t => {
    await t
      .typeText(loginPageModel.emailInput, user.email)
      .typeText(loginPageModel.passwordInput, user.password)
      .click(loginPageModel.submitButton);

    await t.expect(loginPageModel.accountHeader.exists).ok();
    await t
      .expect(loginPageModel.accountEmail.innerText)
      .contains(user.email);
  });
})

This data-driven test cuts down the amount of duplicate code significantly. There's only one written test case, but it runs four times, once for every record in the file. The hard-coded data that existed previously is also gone from the test since we're using the data from the imported JSON file.

It's an improvement from the repetition we had before, and if we want to test more accounts, adding a new record to the JSON file creates a new test case without having to touch the code.

Data-driven testing using XML files

The next file type on our list is an XML file. While many developers prefer to work with JSON's simplicity when it comes to data files, XML is still widely used across many development shops. It's a mainstay in mature programming languages, Java in particular.

The XML file used for this example follows the same pattern of the JSON file. In a file called users.xml inside the data sub-directory, we'll have a properly-validated XML containing nodes with the login information to use during testing.

<?xml version="1.0" encoding="UTF-8" ?>
<users>
  <row>
    <email>airportgap@dev-tester.com</email>
    <password>airportgap123</password>
  </row>
  <row>
    <email>testuser1@dev-tester.com</email>
    <password>airportgaptest1</password>
  </row>
  <row>
    <email>testuser2@dev-tester.com</email>
    <password>airportgaptest2</password>
  </row>
  <row>
    <email>testuser3@dev-tester.com</email>
    <password>airportgaptest3</password>
  </row>
</users>

Node.js doesn't have built-in support for reading and parsing XML files, so we'll have to use a third-party library. Node.js has plenty of XML parsers that you can use for multiple purposes, depending on the formatting and structure of the file. Since all we want is to parse an XML file for use inside our test, we're going with a library called xml2json.

The xml2json library takes care of parsing simple XML files, like the one defined above, and converts it into a familiar JavaScript object. Once we have the JavaScript object, we'll have easy access to the data. The test can then get refactored similarly to the previous JSON file example.

First, we need to install the xml2json library with the command npm install xml2json in the root directory where we have our TestCafe tests. After installing the library, we can import it to parse XML, get the JavaScript object, and iterate through each record to create a unique test case:

import loginPageModel from "./page_models/login_page_model";

const fs = require("fs");
const parser = require("xml2json");

const xmlFile = fs.readFileSync("./data/users.xml", { encoding: "utf8" });
const { users } = parser.toJson(xmlFile, { object: true });

fixture("Airport Gap Login (XML)").page(
  "https://airportgap-staging.dev-tester.com/login"
);

users.row.forEach(user => {
  test(`User with email ${user.email} can log in to their account`, async t => {
    await t
      .typeText(loginPageModel.emailInput, user.email)
      .typeText(loginPageModel.passwordInput, user.password)
      .click(loginPageModel.submitButton);

    await t.expect(loginPageModel.accountHeader.exists).ok();
    await t
      .expect(loginPageModel.accountEmail.innerText)
      .contains(user.email);
  });
});

There's a little more happening in this test than in the JSON example. First, we need to read the XML file before using the xml2json library to parse the data. For this, we can use the File System module built into Node.js. This module provides support for accessing the local filesystem.

We import and set up the File System module using the fs variable, and then use the module's readFileSync method to read the XML from the local filesystem. This method reads a file and returns its contents. Since we want the access the XML file contents as a string, we specify the encoding option. In our case, we're using UTF-8 encoding, which works for most scenarios.

The fs.readFileSync method gives us a string with the contents of the XML file. With this information in hand, we can use the xml2json library to parse the string and return a usable JavaScript object, using the library's toJson method.

In this example, we're saving the parsed object using the destructuring assignment syntax to store the information we want. The JavaScript object returned from the xml2json library contains an array containing a row key, due to the structure of the XML file. row is an array containing objects with our login information. From there, all that's left is to iterate through the object and create our test cases. This approach - the same as we did with the previous JSON example - is common for most file types in data-driven testing.

Coming up next: CSV and Excel files

This article gives you an idea of how to implement data-driven testing to a framework like TestCafe. You can also use these same techniques for similar frameworks. It can help keep your test suite organized if you need to perform a group of tests using different sets of data or looking for an easier way to add additional data without modifying the code.

Next week, we'll continue refactoring our tests using the data-driven testing approach. We'll show how to use CSV and Excel files, two other common file types used in testing, to drive your TestCafe tests.

What other file types do you use for data-driven testing? Are there any different scenarios you'd like to see in a future article? Let me know by leaving a comment below!

P.S. Was this article helpful to you? I'm currently writing a book that will cover much more about about the TestCafe testing framework.

With the End-to-End Testing with TestCafe book, you will learn how to use TestCafe to write robust end-to-end tests on a real web app and improve the quality of your code, boost your confidence in your work, and deliver faster with less bugs.

For more information, go to https://testingwithtestcafe.com. Sign up and download the first three chapters for free!