One way to do TDD incorrectly (such that it isn't TDD) is to write the unit tests in arrears. That misses out on the entire point of TDD.
I think unit tests are necessarily internal to a service/feature/component -- if they weren't they wouldn't be unit tests, they'd be integration tests.
They are to fill the gap that languages have which do not have contract support.
One of the quotes I like is "untested code is buggy code". Unit tests can exercise every code path, such that code can be known to fulfill basic correctness.
It may not work together -- something that integration tests are useful for -- but at least it has basic correctness. Integration tests do not have the ability to exercise every code path (as pointed out by Rainsberger in his Integration Tests are a Scam presentation).
That's where TDD can help provide a suite of unit tests as a residual value. And that suite of unit tests will also good code coverage because the unit tests are written a priori, one by one, in a tight cycle with the implementation.
Contract tests are great. Acceptance tests. Integration tests. Security tests. Performance tests. All of them have their place. None of them are substitutes for unit tests, none of them give assurance of basic correctness. And unlike those other tests, with TDD the developers write the unit tests not for the sake of unit testing, but rather for the sake of design and development.
All those other kinds of tests ought to be created by quality engineers. But TDD style unit tests: those are created by the developers.
Another problem with TDD is the language and environment used. C++ is not a great language to use for TDD, when every other step involves a lengthy build cycle. On the other hand, C#, Visual Studio, NUnit or xUnit.net, and NCrunch is an amazing combination -- one which makes TDD actually fun. (Seriously. FUN! Not kidding.)
If I understand you (I can be a bit slow sometimes!), this approach takes external (contract, and quality) requirements (and hopefully tests from independent QA folks), decomposes them into TDD unit tests to ensure the internals are designed, implemented and refactored correctly, piece by piece to meet each larger contract requirement? These internal unit tests are now disposable, having provided most of their value, leaving the external tests to provide confidence that future changes are not breaking contracts?
If so, then sure, I think this is a good way of working without accumulating excessive amounts of redundant internal unit tests, although I may agree with James C that it's /passing/ tests that provide no value in the longer term, and their removal is primarily to limit the test maintenance pain... YMMV :)
TDD unit tests are at the level of quarks, electrons, and atoms.
Integration tests, system tests, and acceptance tests are at the level of bricks, mortar, girders, electrical, and plumbing. The external contract and quality (the requirements and specifications) are at this level; these kinds of tests reflect those things.
Unit tests are run in debug mode. For a large application, the entire suite of unit tests should take a few seconds or less to run.
Integration tests, system tests, and acceptance tests are run against optimized release code. And can take hours to run... or longer.
Unit tests ensure basic correctness. (Unit tests are a solution for languages that do not provide contracts. In this sense contracts are the prerequisites, postconditions, and invariants of the methods and classes. That's entirely different from contracts at the scope of system requirements, external contracts, and quality standards.)
TDD is not about testing; TDD is about design. TDD unit tests are a kind of scaffolding. The value of the unit test is how it helps guide development. The residual value of unit tests is that they provide confidence in basic correctness. However, the primary value of TDD unit tests is in the process (the steps I cited above), which means that if the test is deleted it still has provided enormous value.
Much like scaffolding when building a building: once the building is built, the scaffolding can come down. Coplien considers a large suite of unit tests -- which have to be maintained -- as muda (waste, in the Lean sense). He considers eliminating that waste as a good and prudent thing to do.
For my projects, I do not consider the suite of unit tests (which have to be maintained) to be muda. However, I would much rather have proper language support for contracts which would obviate the need for unit tests to ensure basic correctness, and with C++20 there is hope.
The ironic thing is that "a suite of unit tests to ensure basic correctness" is a byproduct of TDD. The point of TDD is design, and contract support in the language would not be facilitated (nor hindered) by contract support -- but would eliminate the need for unit tests, which would probably make some developers who do TDD to stop doing TDD. Especially those developers who do TDD or do TDD incorrectly as a means to create a unit test suite, rather than as a means to facilitate better design.
"Unit tests are a solution for languages that do not provide contracts. In this sense contracts are the prerequisites, postconditions, and invariants of the methods and classes. That's entirely different from contracts at the scope of system requirements, external contracts, and quality standards."
I contest however that these internal correctness contracts /must/ be derived from the external contracts defining expected behaviour or they are useless in proving that the right thing is being constructed, indeed Mr Garcia in the linked paper says as much on slide 20 "Correctness -> Degree to which a software component matches its specification." - scaffolding that helps build a bungalow when a block of flats was required is not very helpful :)
If you've got the luxury of having quality engineers that can write unit tests, integration tests and acceptance tests (and they have the time to do so) you can throw away TDD unit tests easily.
I think projects that work this way are in the minority.
One way to do TDD incorrectly (such that it isn't TDD) is to write the unit tests in arrears. That misses out on the entire point of TDD.
I think unit tests are necessarily internal to a service/feature/component -- if they weren't they wouldn't be unit tests, they'd be integration tests.
They are to fill the gap that languages have which do not have contract support.
One of the quotes I like is "untested code is buggy code". Unit tests can exercise every code path, such that code can be known to fulfill basic correctness.
It may not work together -- something that integration tests are useful for -- but at least it has basic correctness. Integration tests do not have the ability to exercise every code path (as pointed out by Rainsberger in his Integration Tests are a Scam presentation).
That's where TDD can help provide a suite of unit tests as a residual value. And that suite of unit tests will also good code coverage because the unit tests are written a priori, one by one, in a tight cycle with the implementation.
Contract tests are great. Acceptance tests. Integration tests. Security tests. Performance tests. All of them have their place. None of them are substitutes for unit tests, none of them give assurance of basic correctness. And unlike those other tests, with TDD the developers write the unit tests not for the sake of unit testing, but rather for the sake of design and development.
All those other kinds of tests ought to be created by quality engineers. But TDD style unit tests: those are created by the developers.
Another problem with TDD is the language and environment used. C++ is not a great language to use for TDD, when every other step involves a lengthy build cycle. On the other hand, C#, Visual Studio, NUnit or xUnit.net, and NCrunch is an amazing combination -- one which makes TDD actually fun. (Seriously. FUN! Not kidding.)
Thanks for the detailed reply :)
If I understand you (I can be a bit slow sometimes!), this approach takes external (contract, and quality) requirements (and hopefully tests from independent QA folks), decomposes them into TDD unit tests to ensure the internals are designed, implemented and refactored correctly, piece by piece to meet each larger contract requirement? These internal unit tests are now disposable, having provided most of their value, leaving the external tests to provide confidence that future changes are not breaking contracts?
If so, then sure, I think this is a good way of working without accumulating excessive amounts of redundant internal unit tests, although I may agree with James C that it's /passing/ tests that provide no value in the longer term, and their removal is primarily to limit the test maintenance pain... YMMV :)
TDD unit tests are at the level of quarks, electrons, and atoms.
Integration tests, system tests, and acceptance tests are at the level of bricks, mortar, girders, electrical, and plumbing. The external contract and quality (the requirements and specifications) are at this level; these kinds of tests reflect those things.
Unit tests are run in debug mode. For a large application, the entire suite of unit tests should take a few seconds or less to run.
Integration tests, system tests, and acceptance tests are run against optimized release code. And can take hours to run... or longer.
Unit tests ensure basic correctness. (Unit tests are a solution for languages that do not provide contracts. In this sense contracts are the prerequisites, postconditions, and invariants of the methods and classes. That's entirely different from contracts at the scope of system requirements, external contracts, and quality standards.)
TDD is not about testing; TDD is about design. TDD unit tests are a kind of scaffolding. The value of the unit test is how it helps guide development. The residual value of unit tests is that they provide confidence in basic correctness. However, the primary value of TDD unit tests is in the process (the steps I cited above), which means that if the test is deleted it still has provided enormous value.
Much like scaffolding when building a building: once the building is built, the scaffolding can come down. Coplien considers a large suite of unit tests -- which have to be maintained -- as muda (waste, in the Lean sense). He considers eliminating that waste as a good and prudent thing to do.
For my projects, I do not consider the suite of unit tests (which have to be maintained) to be muda. However, I would much rather have proper language support for contracts which would obviate the need for unit tests to ensure basic correctness, and with C++20 there is hope.
The ironic thing is that "a suite of unit tests to ensure basic correctness" is a byproduct of TDD. The point of TDD is design, and contract support in the language would not be facilitated (nor hindered) by contract support -- but would eliminate the need for unit tests, which would probably make some developers who do TDD to stop doing TDD. Especially those developers who do TDD or do TDD incorrectly as a means to create a unit test suite, rather than as a means to facilitate better design.
"Unit tests are a solution for languages that do not provide contracts. In this sense contracts are the prerequisites, postconditions, and invariants of the methods and classes. That's entirely different from contracts at the scope of system requirements, external contracts, and quality standards."
OK, I get this - and I found this presentation on contracts within the language for C++20: cppeurope.com/wp-content/uploads/2...
I contest however that these internal correctness contracts /must/ be derived from the external contracts defining expected behaviour or they are useless in proving that the right thing is being constructed, indeed Mr Garcia in the linked paper says as much on slide 20 "Correctness -> Degree to which a software component matches its specification." - scaffolding that helps build a bungalow when a block of flats was required is not very helpful :)
Unit tests cannot fulfill the role of acceptance tests.
To further elaborate...
Unit tests they are useless in proving that the right thing is being constructed. They only demonstrate basic correctness.
Unit tests won't show whether a bungalow is correct, or a block of flats is correct.
They'll show if the nail is correct. The nail does not care if it is used in a bungalow, or a block of flats.
If you've got the luxury of having quality engineers that can write unit tests, integration tests and acceptance tests (and they have the time to do so) you can throw away TDD unit tests easily.
I think projects that work this way are in the minority.
Quality engineers should not be writing unit tests. If the developers do TDD, the developers should be writing unit tests.
Quality engineers should be writing integration tests and acceptance tests.
Sorry I misread your comment.