This post was originally published in my blog smartpuffin.com.
I have been unit testing my code for many years.
While building a GIS-system, we r...
For further actions, you may consider blocking this person and/or reporting abuse
Hi Elena.
First of all great article. I would like to add mine observation about unit tests.
In discussion with the business about writing unit tests necessity, I used to use the argument that unit test help us discover a bug earlier and at the end of the day they are cheaper than manual testing. Costs always play role in discussions with the business. 😉
About modularization and dependencies, as you said, unit tests improve the architecture of our code. It is easier to follow SOLID principles when we're writing unit tests. In my opinion, we can treat unit tests as a guard of SOLID principles in our code. If something is painful to test it smells like not SOLID enough.
About code which doesn't require unit tests, I totally agree with you that we should be aware of the amount which is needed to cover a particular piece of code with a test and in the case when costs are higher than benefits, we should resign of writing it. BUT it could be dangerous in some cases. We can end with code which is hard to test due to its dependencies, which is fragile and not maintainable. We can end a vicious circle. On the one hand, we want and need to write a unit test and on the other hand, writing it is very, very time-consuming. I think that sometimes we should pay the costs. The longer we're waiting, the cost is rising. If we are talking about some legacy code, which doesn't change at all, I think we can deal with this. When we need to maintain the legacy code, we should pay and write unit tests.
To sum up, great article. I've read it with real pleasure. Cheers.
Hey Rafal, thank you!
The importance of testing depends on business priorities. After all, tests consume time, and sometimes the business decides to stay on the side of development speed, rather than quality.
Sometimes, when this happens, the code becomes untestable and also harder to change, but at this time refactoring and tests cost even more. They are a long-term investment. This is a good time to make the statement about the future costs you were talking about, and start to introduce testing from the most critical parts.
Exactly, this was my thought ;)
I disagree with the need for mocking and I think people spend way too much time setting up mocks. The rest of your modules should alos work, since you've tested them as well, thus it's totally fine to rely on them working to write new tests.
The goal is, as you said, to test business needs. There isn't a strong objective to have modules tested independently.
This isn't to say that mocks aren't useful tools, just that they are being too widely used in situations when they just aren't necessary.
I think I understand your point of view - you are talking about integration testing.
But I have an objection. When your code is complex enough and has several nested dependencies, it becomes very hard to test all combinations of all code paths with integration testing.
Imagine this:
{code}
class A {
void methodA(B b) {
if(condition) {
b.methodB1();
} else {
b.methodB2();
}
}
class B {
void methodB1() {
if(condition) {
doSomething();
} else {
doSomethingElse();
}
}
void methodB2() {
if(condition) {
doSomething2();
} else {
doSomething3();
}
}
{code}
If we don't mock while testing class A, we end up with a dependency on a particular implementation of B. How many tests for A we need to write now?
If in the code we use different B's, say, different subclasses, do we need to test these combinations as well? This is a lot of tests for A.
Now, if there is a problem in B's implementation, all tests for B and A will break, and it will be harder to understand where's the problem.
I think there should be a healthy combination of pure unit-tests and integration tests. In case of doubt, and especially if the logic is complex, I would prefer to have unit-tests as well.
If you're testing A you still limit your testing to A, on the assumption that B is working. That is, just becasue B branches on a condition doesn't mean you test it in A's tests.
What you're essentially doing is just using a current implementation of B as the mock object. You know how it works and rely on that for the testing of A. This is sufficient to demonstrate that A is working correctly.
If B breaks, then yes, tests in A will also break, but so will tests in B. Unless you have circular dependencies you still have a tree of changes to work through in order.
Keep in mind the primary value of unit tests is to ensure things are working. Debugging is a secondary value. Getting more coverage by reusing code is of more immediate value than getting clean mocks. I also start with the breadth of coverage, and if, and only if, debugging is proving problematic, do I start writing focused mock objects.
I agree with edA-qa on this point. Creating multitudes of mocks just to keep unit tests 'pure' seems like a great deal of effort to create and maintain additional code, and the benefit of doing so seems unclear to me. As edA-qa said, the real code can usually do double-duty as a test object.
There are cases for using mocks, but my opinion is that they should be introduced when depending on the real application logic in a test causes a problem. That can be for performance reasons; possibly to reduce the amount of setup/configuration needed just to run unit tests; perhaps because the real code may not be available when tests run; often mocks are used because time affects the results of the real code - in fact I imagine that things like the current system time and calls to random are some of the most often mocked pieces of logic.
Note: For this comment, by 'mock' I mean any kind of stand-in used for a test that replaces the actual application code. That includes 'mocks', 'stubs', 'fakes'...
Thank you, Elena, for this really interesting article!
I have a couple of questions. You wrote that the tests' directory structure repeats the production code's directory structure. Do your tests completely repeat the production code's structure? Do you change your tests, when you refactor? In other words, are your tests contra-variant with the structure of your production code?
You also wrote that we have to explain the importance of unit testing to the business. What scenario do you mean? When you take over some legacy code and you have to turn it into something maintainable or when you start working on some legacy project?
Thanks again for your article, I think you emphasize on some very important practices that so many us lack.
Hi Sandor, happy you liked it:)
Yes, for us the tests repeated the code structure in terms of classes - one class called, i.e., ToolX had a complementary class called ToolXTests.
Regarding refactoring: if while refactoring I change the logic or the API of a module, then I have to change the tests and make sure they pass.
About the business. Sometimes I hear people starting a new project, or already in the middle of a project, and struggling: do we need tests? When do I start with tests? How to explain unit-tests to the management? Business needs are a good way to prioritize and begin.
For a legacy project which you have to support, it is also considered a good practice to start with tests. This way you're supposed to understand better what the code does, to make it more maintainable (through modularization and decoupling), and to provide yourself a safety net for code changes. I never used this approach with legacy code myself yet, but I do see the pros.
Thanks for your answer, Elena.
A few months ago, one of our architects commented on my pull request in which I refactored some classes and I was copying the new structure to the tests. He told me that I was basically coupling production and test code and he also shared with me this article about test contra-variance. You might find it interesting.
I think it's always a good practice to start with tests. Unless you are really, I mean really sure that it's just a bit of throw-away code, most probably a prototype.
As I try to follow TDD as much I can, I don't even feel the urge to explain unit testing to the business. It's just the way I write code. Unit tests are part of the code, their cost is part of the estimation. I don't explain to them if statements or polymorphism either, but more the relevant decisions about the business logic we have to make.
I have a bit of a different point of view than that article. Let me try to explain.
For me tests are the integral part of the code, just like comments or naming. So I don't really see a problem with changing tests while refactoring.
The most important thing about the code, for me, is how easy it is to read and understand. I do my best to keep the code as plain as possible because I am probably not smart enough to understand and remember complex code. If the tests are not reflecting the code structure at all, like they propose in the article, it becomes very hard for me to understand how tests and code are connected, where they are, and which parts of code they are for.
If I refactor by moving a piece of code to a separate public class or method, I do want a test for it! They propose to only have integration tests for the generic API. But I want to have a unit-test for this new piece too, for several reasons:
Regarding the TDD, I find it an interesting concept, but I never managed to actually start with tests. I don't start with API, I always need some time to crystallize it based on implementation.
Thank you for a very interesting discussion!:)
I must agree, the most important thing about the code is to make it easy to understand. Then if we understand it, we can correct it, if needed.
Thank you too, it's always interesting to learn about other ways to see the things!
Hi Elena, great article!I really enjoyed reading it :)
Which unit testing/mocking frameworks do you use?
The project was mostly in C#. We had NUnit, MSTest, and Rhino Mocks.
For the part that was in Java, I think we had JUnit and Mockito, but here I'm not so sure - it was more than 2 years ago:)
Were you satisfied with these frameworks?
Have you heard about Typemock by any chance?
I liked the C# ones - they were rather easy to use and rich in features at the same time. I.e., you can "expect" how many times a certain function is called, or make your mock return different values on different calls (although the syntax for that wasn't very easy).
I don't think I tried Typemocks, so can't say anything about it, unfortunately.
Thanks!
I'm asking because my team and I are working with Typemock, and your article made me realize how powerful it is.
Oh how nice:)) happy I helped!
Great article. The article covers adding tests to an existing code base however it leads back this confusion of tests and test driven development where you write tests first to drive the development of the system architecture the code coverage is an added benefit thrown in.
I highly recommend Ian Coopers TDD where did it all go wrong youtube.com/watch?v=EZ05e7EMOLM and of course the classic Kent Beck TDD book and the 3 laws of tdd.