TL;DR

I’ve released a Jest extension that allows you to write acceptance tests  in natural languages similar to Gauge, in which you can practice acceptance tests driven development; ATDD.

https://github.com/tnzk/jest-gauge

What is Acceptance-Test Driven Development (ATDD)?

Acceptance-Test Driven Development is a software development methodology derived from Test-Driven Development, as known as TDD. In contrast to TDD which focuses on describing what you expect a class or a module to be, ATDD encourages you to describe acceptance criteria of the subject system as a whole, in a larger sense.

Roughly speaking, they provide a feedback cycle of software tests around unit testing with TDD and E2E testing for ATDD, respectively.

Behavior-Driven Development, BDD, is another methodology which also derives from TDD. BDD shares the same feedback cycle as in TDD, but focuses on defining “behaviors” of a class or a module. More importantly, BDD involves a wider variety of stakeholders to participate in. Unlike TDD which is designed and evolved as a practice for programmers to get their job done nicely, BDD puts emphasis on describing User Stories to define what kind of value should be delivered to customers, through “Discovery workshop” by the “three amigos,” said the product owner, developers and testers.

User Stories by theirselves cannot drive a feedback cycle of software tests since they’re just some plain sentences in a natural language. However, expressing an expected behavior directly as a piece of test code will prevent non-programers to participate in the session as stakeholders, which exactly BDD encourages in the first place.

To overcome this dilemma, Cucumber, a well known BDD framework, provides a DSL called Gharkin, which you can describe the insights found in “Discovery workshop”. Gharkin allows you to write something like “AS a Customer, TO avoid standing in the line for a bank teller, I WANT to withdraw cash via ATM”, similar to well-known User Story Template. This looks understandable to stakeholders who don’t code.

As Daniel North explained was inspired by Ubiquitous Language by Eric Evans, they are almost identical to the domain definition, in other words, the "acceptance criteria" to the product from the business owner's perspective. The motivation towards ATDD is the question of how we can drives software development by defining these acceptance criteria in terms of executable specification of the software.

Why Gauge?

Through the explanation above, you might find BDD and ATDD don't seem to differ that much. If so, can't we use Cucumber to drive ATDD, as long as we let the minor mismatches aside?

The dealbreaker is the fact that Gherkin is not a natural language on a closer look, but a loosely-defined formal language. Finished examples in Gherkin looks as if written in a natural language indeed, but when it comes to writing, stakeholders who don't code found it very tough to write in it (Programmers, of course including me, sometimes misbelieve even non-programmers can read and write a simple piece of RSpec or something, but in reality they can't. This should be a cognitive bias which many programmers share I think).

Gherkin's syntax in which you write the behavioral definition is, in fact, not a natural language but slightly formally constrained.

In Gauge, you can write literally in a natural language. While specification file itself is defined as subset of Markdown, sentences inside are just a line of a human language.

# Top page Specification 

You can explain freely the background or motivation of the specification, since paragraphs here will be ignored as just comments.

## Scenario: a user can open a site and see the top page.

- Open "https://duckduckgo.com/".
- It shows a picture of a cute Cucumber-looking bird to the user.

To run tests, Gauge will look up step implementations by the title for corresponding to specification lines, in exact matching basis. You can insert a template variable to give it to some robustness as you like, whereas in turn it makes the sentences look like in a formal language. Unlike Gherkin, you can tweak the strictness of the language by using or not using these variables or other similar mechanisms.

I'm interested in Gauge since it allows us to write specifications and acceptance criteria in a free-form natural language.

Why Jest-gauge?

Gauge seemed a little opinionated to me, and felt it required a heavy-lifting to integrate into an existing product. It was hesitant for me to give a surprise like "Hey guys, we need to switch the testing framework to Gauge!" to team members who just had been familiarized with unit/E2E testing in Jest.

I suppose I will need Gauge in the future to navigate full-fledged ATDD, but my primary motivation at this time is something narrower than "Is ATDD good or not?", more like "Will specs in a Gauge-like natural language be the catalyst that leads communication between developers and non-developer stakeholders?"

So, I've started building a tool for a team with existing unit/E2E tests in Jest to try ATDD easily.

https://github.com/tnzk/jest-gauge

Refer README for installation and usage.

Since it is a Jest extension, you can try it out by adding a configuration into jest.config.js. For specification above, it shows a test report like this:

$ npx jest --config=jest.config.gauge.js specs/

PASS  examples/welcome.spec

Top page Specification 

Scenario: a user can open a site and see the top page.
✓ Open "https://duckduckgo.com/"
✓ It shows a picture of a cute Cucumber-looking bird to the user.

Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        0.913 s

It supports a large part of Gauge's functionality, but of course there's many things not supported yet. You can find what is supported and what is not in TODO section in README.

Motivation

I'm running a coding bootcamp in Japan. We are building and maintaining a learning management system for internal use from scratch. We have stakeholders of the project working as a non-tech staffs and they have very raw and factual needs about what matters for operating the educational activity in the bootcamp, as massive and complex Excel spreadsheets! It had motivated me to wander how to keep them organized and ship as working software.

I tried convincing the stakeholders "We're a coding bootcamp, so we should take a risk and try new things on software development methodology. It must lead us to some insights about quality of the educational service". I knew this is a kind of big ask to be accepted in other domain of businesses. Even developers are surprised on this. So I must say a warm appreciation for accepting this to colleagues.

A bit tangentially, that was not the only stretching. We've built a video chat functionality from scratch with WebRTC into the LMS, where the lectures and technical support sessions take place. Screenshot below are us doing a workshop on it. It takes much of efforts to stabilize the connectivity since we don't rely on a famous product like Agora and Zoom APIs, and in course we had learnt many insights and grown team unity.

So, I decided to build Jest-gauge, after concluding Gauge itself is not for our team at this moment and catching up ATDD in 2021 and Cucumber/Gherkin which I experienced half decade ago. And then on, our CI runs acceptance tests written in our mother tongue everyday.

Yes, this is our proud production case :)

A joke aside, we accept risks because we are not a big-co and of the background above. There's many rough edge in it, so please use it at your own risk.

Conclusion

These experience above, and the books on ATDD like "Growing Object-Oriented Software, Guided by Tests", leads me to a belief that it might be too early for developers and other stakeholders to communicate through behavioral descriptions or in a ubiquitous language, in evidence-based manner, a decade ago.

Increasing importance of technology and things like no-code platforms may change this. We may be able to discuss and share what a product is and should be through a semi-formal language in like Gauge or Cucumber.

Members of a company who runs a coding bootcamp, maybe inevitably, looks like having relatively good understanding of computers and software, even they're not developers. This might be the reason we had adopted ATDD with relatively little friction. We'll practice the ATDD with Jest-gauge there at this moment, and later transfer that insight to other fields with less association with software.

But this is just a try. Maybe it works, or maybe not. If you try Jest-gauge, and got some insights, please share it with me. Of course, any contributions are warmly welcomed.