Last week I made a training session for my team on end-to-end (E2E) testing with Jest and Playwright. Playwright is not new anymore, and there are many Playwright tutorials and boilerplates on the Internet. However, they often lacked practical setup like code generation, debugging, reporting, and IDE configuration. Therefore, I decided to roll my own at https://github.com/devalex88/playwright-boilerplate.
To use the boilerplate, clone and modify it to suit your needs. The boilerplate is pre-configured to works best in the Visual Studio Code, with enhanced code authoring and debugging experience. The following Visual Studio Code extensions should be installed:
Execute test cases
To execute all test cases, use the command npm test. Since this project uses Jest as the testing framework, you can peruse its documentation for more execution scenarios. The default Jest configuration is specified in jest.config.js.
> playwright-demo@1.0.0 test C:\Users\haimnguyen\data\playwright-demo
> jest
PASS browser: chromium tests/todomvc.test.js (6.298 s)
TodoMVC
√ should let users manage to-do list (5091 ms)
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: 7 s
Ran all test suites.
To maximize the performance, test cases will be executed against browsers in headless mode. Later sections will explain how to debug the test cases in headfull mode, execute with Github Action workflow, and retrieve the reports.
Debug a specific test case
Open the project in Visual Studio Code. If the extensions mentioned above have been installed, two buttons — Run and Debug — will be shown on top of the test case declaration. To debug a test case, click on Debug and utilize Visual Studio Code’s debugger features (breakpoints, evaluation) to inspect the running code.
The Jest configuration for debugging is specified in jest-debug.config.js. The debugging configuration extends normal configuration with settings that improve the experience, such as headless: false and slowMo: 200, which helps you see what is going on.
Generate code for new test cases
It may sound amateur to write E2E test cases by recording user interactions. However, writing locators for all elements of long test cases is often a tedious job. Playwright provides Playwright CLI with an advanced recording feature. The CSS and XPath locators it generates are often text-based, making locators resilient against changes in the DOM tree such as classes, IDs, or positions.
To generate a test case, use the command npm run codegen http://todomvc.com/examples/react/#/.
const { chromium } = require('playwright');
(async () => {
const browser = await chromium.launch({
headless: false
});
const context = await browser.newContext();
// Open new page
const page = await context.newPage();
// Go to http://todomvc.com/examples/react/#/
await page.goto("http://todomvc.com/examples/react/#/");
// Click input[placeholder="What needs to be done?"]
await page.click('input[placeholder="What needs to be done?"]');
// Fill input[placeholder="What needs to be done?"]
await page.fill(
'input[placeholder="What needs to be done?"]',
"Design a prototype"
);
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Fill input[placeholder="What needs to be done?"]
await page.fill(
'input[placeholder="What needs to be done?"]',
"Organize photo shoot"
);
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Fill input[placeholder="What needs to be done?"]
await page.fill(
'input[placeholder="What needs to be done?"]',
"Bring an umbrella"
);
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Check //div[normalize-space(.)='Design a prototype']/input[normalize-space(@type)='checkbox']
await page.check(
"//div[normalize-space(.)='Design a prototype']/input[normalize-space(@type)='checkbox']"
);
// Check //div[normalize-space(.)='Organize photo shoot']/input[normalize-space(@type)='checkbox']
await page.check(
"//div[normalize-space(.)='Organize photo shoot']/input[normalize-space(@type)='checkbox']"
);
// Check //div[normalize-space(.)='Bring an umbrella']/input[normalize-space(@type)='checkbox']
await page.check(
"//div[normalize-space(.)='Bring an umbrella']/input[normalize-space(@type)='checkbox']"
);
// Click //div[normalize-space(.)='Bring an umbrella']/button
await page.click("//div[normalize-space(.)='Bring an umbrella']/button");
// Click input[placeholder="What needs to be done?"]
await page.click('input[placeholder="What needs to be done?"]');
// Click input[placeholder="What needs to be done?"]
await page.click('input[placeholder="What needs to be done?"]');
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Fill input[placeholder="What needs to be done?"]
await page.fill(
'input[placeholder="What needs to be done?"]',
"Polish brand idea"
);
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Click text="Completed"
await page.click('text="Completed"');
// assert.equal(page.url(), 'http://todomvc.com/examples/react/#/completed');
// Close page
await page.close();
// ---------------------
await context.close();
await browser.close();
})();
Create a test case tests/todomvc.test.js with content from lines 9–86. The prolog and epilog are not necessary since they have been taken care of by jest-playwright-preset.
const { chromium } = require("playwright");
describe("TodoMVC", () => {
it("should let users manage to-do list", async () => {
// Open new page
const page = await context.newPage();
// Go to http://todomvc.com/examples/react/#/
await page.goto("http://todomvc.com/examples/react/#/");
// Click input[placeholder="What needs to be done?"]
await page.click('input[placeholder="What needs to be done?"]');
// Fill input[placeholder="What needs to be done?"]
await page.fill(
'input[placeholder="What needs to be done?"]',
"Design a prototype"
);
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Fill input[placeholder="What needs to be done?"]
await page.fill(
'input[placeholder="What needs to be done?"]',
"Organize photo shoot"
);
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Fill input[placeholder="What needs to be done?"]
await page.fill(
'input[placeholder="What needs to be done?"]',
"Bring an umbrella"
);
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Check //div[normalize-space(.)='Design a prototype']/input[normalize-space(@type)='checkbox']
await page.check(
"//div[normalize-space(.)='Design a prototype']/input[normalize-space(@type)='checkbox']"
);
// Check //div[normalize-space(.)='Organize photo shoot']/input[normalize-space(@type)='checkbox']
await page.check(
"//div[normalize-space(.)='Organize photo shoot']/input[normalize-space(@type)='checkbox']"
);
// Check //div[normalize-space(.)='Bring an umbrella']/input[normalize-space(@type)='checkbox']
await page.check(
"//div[normalize-space(.)='Bring an umbrella']/input[normalize-space(@type)='checkbox']"
);
// Click //div[normalize-space(.)='Bring an umbrella']/button
await page.click("//div[normalize-space(.)='Bring an umbrella']/button");
// Click input[placeholder="What needs to be done?"]
await page.click('input[placeholder="What needs to be done?"]');
// Click input[placeholder="What needs to be done?"]
await page.click('input[placeholder="What needs to be done?"]');
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Fill input[placeholder="What needs to be done?"]
await page.fill(
'input[placeholder="What needs to be done?"]',
"Polish brand idea"
);
// Press Enter
await page.press('input[placeholder="What needs to be done?"]', "Enter");
// Click text="Completed"
await page.click('text="Completed"');
// assert.equal(page.url(), 'http://todomvc.com/examples/react/#/completed');
});
});
To assert states, you can make use of the Jest’s built-in expect API, or the convenient expect-playwright library made by the Playwright community. Another assertion library that would prove handy is jest-extend.
Continuous Integration (CI)
Of course, automated testing is useless without the ability to execute test cases automatically upon an event. There is quite a lot of CI software such as Github Action, Azure DevOps, CircleCI, etc. Playwright has done a great job downloading browsers after being installed, and it also has robust support for headless mode. Therefore, it is quite trivial to configure the pipeline with any Cloud CIs or Docker. In this boilerplate, I only included a sample configuration for Github Action at .github/workflows/test.yml.
name: Test
on:
push:
branches: [master]
pull_request:
branches: [master]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Node.js
uses: actions/setup-node@v1
with:
node-version: 12.x
- run: npm install && npm test
- name: Upload reports to Katalon TestOps
uses: katalon-studio/report-uploader@v0.0.7.11
env:
PASSWORD: ${{ secrets.KATALON_API_KEY }}
PROJECT_ID: ${{ secrets.KATALON_PROJECT_ID }}
TYPE: junit
REPORT_PATH: ${{ github.workspace }}/reports
After the execution completes, we will often have to review the results. If you have many test cases, you would find it tedious to review and investigate or debug the failures. That’s why I have included jest-junit in the boilerplate. Thanks to this Jest’s reporter, results will be exported to reports/junit.xml. The reports are in JUnit’s XML format, which is almost universally understood by Test Management and Continuous Integration software.
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="jest tests" tests="1" failures="0" errors="0" time="7.005">
<testsuite name="TodoMVC" errors="0" failures="0" skipped="0" timestamp="2020-10-15T07:56:27" time="6.298" tests="1">
<testcase classname="TodoMVC should let users manage to-do list" name="TodoMVC should let users manage to-do list" time="5.091">
</testcase>
</testsuite>
</testsuites>
In the sample Github Action workflow, the reports are also uploaded to Katalon TestOps (disclaimer: I'm working on this project. However, since Katalon TestOps is available for free, I think it is also a reasonable choice. You can use your favorite tools to consume the JUnit XML reports). The dashboards and analytics provided by Katalon TestOps will give you significant insights into the testing activities. You can also use its Jira integration functions to link you test cases with other Jira data to keep the test cases align with the business requirements.
That’s all the major scenarios this boilerplate supports. If you have any requests regarding functionality or customization, don’t hesitate to create an issue.
Thank you and happy coding!
Top comments (0)