DEV Community

Yuri Costa
Yuri Costa

Posted on • Updated on

#7 Acceptance Testing - Tips from The Clean Coder

This is the seventh article from the series "Tips from The Clean Coder". Here we gathered and summarized the main tips from the seventh chapter.

The role of the professional developer is a communications role as well as a development role. Remember that garbage-in/garbage-out applies to programmers too, so professional programmers are careful to make sure that their communication with other members of the team, and the business, are accurate and healthy.

Communicating requirements

One of the most common communication issues between programmers and business is the requirements. The business people state what they believe they need, and then the programmers build what they believe the business described. At least that's how it's supposed to work. In reality, the communication of requirements is extremely different, and the process is fraught with error.

communication

Premature precision

Both business and programmers are tempted to fall into the trap of premature precision. Business people want to know exactly what they are going to get before they authorize a project. Developers want to know exactly what they are supposed to deliver before they estimate the project. Both sides want a precision that simply cannot be achieved, and are often willing to waste a fortune trying to attain it.

The uncertainty principle

The problem is that things appear different on paper than they do in a working system. When the business actually sees what they specified running in a system, they realize that it wasn't what they wanted at all. Once they see the requirement actually running, they have a better idea of what they really want - and it's usually not what they are seeing.

morty

Estimation anxiety

First, even with the perfect information, your estimates will have a huge variance.
Second, the uncertainty principle makes hash out of early precision. The requirements will change-making that precision moot.

Professional developers understand that estimates can, and should, be made based on low precision requirements, and recognize that those estimates are estimates; To reinforce this, professional developers always include error bars with their estimates so that be business understands the uncertainty.

Late ambiguity

the solution to premature precision is to defer precision as long as possible. Professional developers don't flesh out a requirement until they are just about to develop it. However, that can lead to another malady: late ambiguity.

Often stakeholders disagree. When they do, they may find it easier to wordsmith their way around the disagreement rather than solve it. They will find some way of phrasing the requirement that they can all agree with, without actually resolving the dispute. I once heard Tom DeMarco say, "An ambiguity in a requirements document represents an argument amongst the stakeholders."

ambiguity

It is the responsibility of professional developers (and stakeholders) to make sure that all ambiguity is removed from the requirements.

This is hard, and there's only one way I know how to do it.

Acceptance tests

The term acceptance test is overloaded and overused. We will define it as tests written by a collaboration of the stakeholders and the programmers in order to define when a requirement is done.

The definition of "Done"

Professional developers have a single definition of done: Done means done. Done means all code written, all tests pass, QA and the stakeholders have accepted. Done.

But how can you get this level of done-ness and still make quick progress from iteration to iteration? You create a set of automated tests that, when they pass, meet all of the above criteria! When the acceptance tests for your feature pass, you are done.

done

Communication

The purpose of acceptance tests is communication, clarity, and precision. By agreeing to them, the developers, stakeholders, and testers all understand what the plan for the system behavior is. Achieving this kind of clarity is the responsibility of all parties. Professional developers make it their responsibility to work with stakeholders and testers to ensure that all parties know what is about to be built.

communication key

Automation

Acceptance tests should ALWAYS be automated. There is a place for manual testing elsewhere in the software lifecycle, but these kinds of tests should never be manual. The reason is simple: cost.

The cost of automating acceptance tests is so small in comparison to the cost of executing manual test plans that it makes no economic sense to write scripts for humans to execute. Professional developers take responsibility for their part in ensuring that acceptance tests are automated.

automation

Extra work

Don't look to these tests as extra work. Look at them as massive time and money savers. These tests will prevent you from implementing the wrong system and will allow you to know when you are done.

Who writes acceptance tests, and when?

In an ideal world, the stakeholders and QA would collaborate to write these tests, and developers would review them for consistency. In the real world, stakeholders seldom have the time or inclination to dive into the required level of detail. So they often delegate the responsibility to business analysts, QA, or even developers.

tests

Typically business analysts write the "happy path" versions of the tests because those tests describe the features that have business value. QA typically writes the "unhappy path" tests, the boundary conditions, exceptions, and corner cases. This is because QA's job is to help think about what can go wrong.

The developer's role

Implementation work on a feature begins when the acceptance tests for that feature are ready. The developers execute the acceptance tests for the new feature and see how they fail. Then they work to connect the acceptance test to the system, and then start making the test pass by implementing the desired feature.

Test negotiation and passive aggression

Test authors are human and make mistakes. Sometimes the tests as written don't make a lot of sense once you start implementing them. They might be too complicated. They might be awkward. They might contain silly assumptions. Or they might just be wrong.

As a professional developer, it's your job to negotiate with the test author for a better test. What you should never do is take the passive-aggressive option and say to yourself, "Well, that's what the test says, so that's what I'm going to do."

Remember, as a professional it's your job to help your team to create the best software they can.

angry

Acceptance tests and Unit tests

Acceptance tests are not unit tests. Unit tests are written by programmers for programmers. They are formal design documents that describe the lowest level structure and behavior of the code.

Acceptance tests are written by the business for the business (even when you, the developer, end up writing them). They are formal requirements documents that specify how the system should behave from the business point of view. The audience is the business and the programmers.

Although it is true that unit and acceptance tests often test the same things, they are not redundant at all. They may test the same things but through different mechanisms and pathways. Unit tests dig into the guts of the system making calls to methods in particular classes. Acceptance tests invoke the system much farther out, at the API, or sometimes even UI level. So the execution pathways that these tests are very different.

But the real reason these tests aren't redundant is that their primary function is not testing. The fact that they are tests is incidental. Their primary purpose is to formally document the design, structure, and behavior of the system.

GUIs and other complications

It's hard to specify GUIs upfront. It can be done but is seldom done well. The reason is that aesthetics are subjective and therefore volatile. People want to fiddle with GUIs. They want to massage and manipulate them.

This makes it challenging to write acceptance tests for GUIs. The trick is to design the system so that you can treat the GUI as a though it were an API rather than a set of buttons, sliders, grids, and menus. This may sound strange, but it's really just good design.

css

Some acceptance tests specify the behavior of the GUI itself. However, these tests do no test business rules and therefore don't require the business rules to be connected to the GUI. Therefore, it's a good idea to decouple the GUI and the business rules and replace the business rules with stubs.

Keep the GUI tests to a minimum. They're fragile.

Continuous Integration

Make sure that all your unit and acceptance tests are run several times per day in a continuous integration system. This system should be triggered by your source code control system. Every time someone commits a module, the CI system should kick off a build, and then run all the tests in the system. The results of that run should be emailed to everyone on the team.

Conclusion

Communication about details is hard. It's too easy for each party to wave their hands and assume that the other party understands. All too often they agree that they understand and leave with completely different ideas.

understanding

The only way I know to effectively eliminate communication errors between the parties is to write acceptance tests. They are the perfect requirements document.

Next article: #8 Testing Strategies
Previous article: #6 Practicing

Top comments (0)