Adding standalone or "one off" scripts to your Playwright suite

There might be a time where you may be asked to automate some tasks that fall outside of what is considered a traditional test.

What do I mean by a traditional test? An automated test typically accomplishes some key ideas:

1. Perform some actions in the UI
2. Assert on the behavior resulting from those actions
3. Detects change in behavior and reports back on why a change might have occurred
4. Runs as often as needed, sometimes multiple times a day

Now there are times where automating the UI is required, but the outcome of that automation isn't needed regularly. I consider these to be standalone or "one off" Playwright scripts. Some examples might be:

1. Scraping some data off of pages to be used later (analytics, manual error checking, cataloging)
2. Inputting and setting (many) values in a form on a one-time basis
3. Reproducing a bug that might require repeated interactions with the same component / api / UI workflow.
4. Client requests such as "we have this spreadsheet data of configurations to create 1,000 widgets in your app, but we don't want to go through the UI manually.
5. A small set of smoke tests designed for production systems only, that aren't applicable in your lower environments
6. Cleanup scripts if tests aren't designed to do cleanup on their own

You can see that these obviously don't fit the bill of a "test suite", with interactions, assumptions, and assertions. So how do you include these type of files without having them run in CI and causing a disaster?

And here are some other requirements that were imposed:

1. We wanted devs to be able to run these scripts locally without any barriers. So a very simple npx playwright test reproduceBug was the goal

2. As just mentioned, we cannot have these files run in CI

3. They still needed to maintain the .spec.ts extension, otherwise Playwright will just spit out "No tests found"

4. The scripts should live in the same code repo we use for our other normal tests. An entirely separate repo just puts up more barriers for engaging with Playwright.

Playwright configs by default look like this:

export default defineConfig({
  testDir: './tests',

... rest of config

This means you cannot place test files outside of this directory, which was brought up as a question on Github some time ago. Initially, I thought it would be nice to add another folder in the repo called "scripts", but Playwright does not allow multiple testDir values.

So the easiest solution was simply to add a subfolder called tests-ignored, so the structure just looks like this:

├── root
│   ├── tests
│   │   ├── tests-ignored
├── package.json
├── package-lock.json
└── .gitignore

So in CI, you run your full battery of tests like this:

npx playwright test --grep-invert ignored

And if you're running a test file locally, everything is as normal as can be:

npx playwright test widgetCreation.spec.ts

And if you're running these other "one off" scripts, it's the same exact pattern:

npx playwright test reproduceDeadlockBug.spec.ts

So there you have it. A nice simple solution that avoids creating extra projects in your Playwright config, and avoids having to know extra options to have to pass to your CLI. This setup helps us freely automate anything we need, without it having to fit any rigid test structures purposefully designed for our regression suite.