In my head there are two types of automated tests worth (in 99% of cases) talking about and I've usually called them:
- Unit Tests
- Integration Tests
Recently I've realised that using those names comes with so much baggage that I need a better name for them. When I'm talking about unit tests I mean any test that starts at some public entry point of the code, goes all the way through and ends at some boundary with an external system. This is either a mocked dependency or is the response of the function/method called initially. This response or call to an external system then has some expected properties asserted to be true. The unit test says nothing about how many functions/methods are called, how many classes might be involved - nothing. The only restriction is that you don't go into some external system such as a database, a Web API, a filesystem etc.
Is this not what everyone else thinks a unit test is?
An integration test, then, is a test to ensure your system can connect and integrate with the external systems we mocked out in the unit tests. We're not testing the external systems, only that we integrate successfully with them. If they're under your control you test them using unit tests inside that system.
Maybe I have the wrong idea and what I describe aren't unit tests and integration tests. Maybe there's a different name I should be using? I ask because I recently discussed unit tests with several people and got responses similar to:
Oh, I don't bother with unit tests, they're a waste of time. I don't like testing one method/function/class at a time
or
Unit testing? No, I prefer to test in the way the user will interact with the system
Both of these objections seem completely crazy to me. Why on Earth would I write a test that does one method at a time? And why would I test something in a way the user wouldn't use it?
One person I was talking to was actually complaining about how, in C#, interfaces couldn't have static functions defined on them (put aside how that would actually work in practise; the only thing I can think is making the class generic for that interface type or something) and this was a problem due to a "rule" their company had that said every class should be mocked out for a unit test except the class under test! This meant that any "Helper" class or "Calculation" class had to be mocked out, which meant it had to adhere to an interface, and therefore had to have instance methods even if it was more suited to being a static class. It also meant that internal classes (and private classes) were not able to be used, because you wouldn't be able to test them and would therefore have holes in your testing.
This is I do not understand.
Why do companies make rules like this? The tests shouldn't drive the architecture. If the class you're testing does everything in one big method or calls 50 other internal classes it shouldn't change your tests! Test that class. Whatever it's up to, test it; and test it in the way a user would use your system. If a user shouldn't use some class because it is for code only inside your library then make it internal and don't test it directly. Please, don't make everything public just for the tests.
So that's my rant. If people think I mean tests testing every method in the code individually when I say unit test then I need a better name for it. I'm open to suggestions. Spec test? BDD test? Behaviour test? All these seem to mean different things to different people.
Is this the meaning you ascribe to "unit test"? Do you have a better name for what I describe? Does your company have an rules similar to the the rule I described above? Do you agree with the rule above? Let me know - I really want to know if I'm just missing something obvious or if we do need a better name for these things free of the baggage.
Top comments (7)
A unit test is normally defined as something that tests a single "unit", or component of code. A unit can be a class or function, but any dependencies must be replaced with mocks for it to be a true unit test. For instance, a true unit test should never write to the database.
An integration test is one that tests two or more components together. It can still mock out dependencies if appropriate, but this should be rare. An integration test can be as high level as creating a dummy request object and passing it through the main application class, or as low level as just checking two classes work together as expected.
Finally, acceptance tests test the application from an end-user's point of view. As such they do things users might - log in, upload pictures etc, and they often either drive a real browser using something like Selenium, or use some high level interface to describe their I reactions. Even for these, it's often appropriate to mock out external dependencies- if you don't, you may get falsely broken tests due to a third party service going down, or find the API accidentally taking action you don't want, such as sending an email to customers.
And I think this is what confuses people. They see things like single unit and think single class. There's a reason they're not called class tests. Sometimes a single unit is more than just one class. That's how we end up with these rules where people are mocking out "calculator" classes instead of having them be static classes and covered by whatever test.
I think what I'm really wanting is for us to concentrate more on the ideas and concepts rather than dictionary definitions. That's probably why I'm after a new name for the tests I'm trying to describe. The more I think about it, the more I realise Spec Test is probably the closest name I can come up with to describe how I think about testing. Maybe I'll use that from now on.
What sort of tests do you value? What do you call them?
I'm inclined to agree with the Testing Pyramid, which is used by Google, among others. I tend to get the most value out of having every class or function having most, if not all, paths tested with unit tests, then integration tests for things like a route (so, for instance, there will be an integration test for submitting a particular fotm), then maybe a few high level acceptance tests.
Because you control all the input to a class in a unit test, you can test situations that are difficult to simulate in integrated or acceptance tests, or test against a wide range of inputs that wouldn't be practical either higher level tests. For instance, if a class relies on a third party service, you can mock that service and have it throw an exception. Unit tests are also fast.
However, you can't rely solely on unit tests, as just because two classes work in isolation, that doesn't mean they interact correctly.
Have you looked at some of the SpecBDD tools around? I found PHPSpec to be a revelation in terms of testing because it makes it more natural to test a class from the outside in. It mocks out dependencies by default and let's you concentrate on designing your classes by specifying what they should do, rather than focusing on implementation.
'Unit' is what you define it as.
One of the side projects I'm working on involves a couple of potential back end modules for a given front end module. I could, in theory, test each back end completely independently of the front end API provided by the main module. However, I'm 'unit' testing the back end modules through the front end module instead. Which back end is in use needs to be completely opaque to consumers of the front end API and not change anything about the API behavior (except possibly timings, but nothing should be depending on that for this API), so the 'unit' really is the combination of front end and back end because that's the whole 'unit' from the perspective of every consumer of this API.
However, that testing does step through every possible code path that could be run by a consumer of this API, because I do need to check all of that. I also do have a test for the front end by itself so that I can be certain it's calling into the back ends properly, which in turn means that if some of the tests fail, I can be certain if it's the front end or the back end without having to test the back ends in isolation.
This is how I think about it too, and I'm convinced this is the most value for least cost of all the testing approaches I've come across. Very well put.
I have been leaning towards not having descriptors for tests at all. If you think about it even an "integration" test is really just a really big unit test. And all tests should be written from the end-user's point of view, just some end users are developers and others are customers.
Just define what the unit and user is for a given test suite.
Yes, this seems like a very sensible approach. I think it's just a shame there isn't a good short word to describe these tests that everyone agrees on and has the same understanding of. I suppose we should concentrate on the concepts and not the names.