As employees or freelancers we sell our time and work to companies and customers. But one thing that distinguishes our profession from others is the fact that we sell intellectual work. Even high level intellectual work sometimes.
So, taking care of the wellness of our mind and intelligence seems to be a reasonable activity we should often practice.
Many people in the development world think, or even say, that testing is hard and that it hurts, not to mention Test Driven Development (TDD).
But I want to demonstrate that TDD is the way to follow if you want to have a healthier life as developer.
But first, let me tell you a couple of things about how our brain works.
Is there anyone in the room that doesn’t know how to use a door? Sure? Have you ever seen a door with an instruction manual? I have: those with pull or push reminders.
Have you ever stood in front of a door without knowing how to open it? I have.
In fact, there are lots of that kind of doors out there.
The point is that a door should be easy to use. But this not always
happens. How to use a door is something that should be obvious, isn’t it?
And, what about switches?
I’m talking about those switch panels in large rooms that don’t correlate with the locations of the lights they control. They are sometimes located where lights are out of sight. You need to try several times to find the secret combinations to turn on the light you want.
The relation between a switch and the bulb that it controls should be obvious.
When we talk about obvious, we talk about knowledge that we don’t need to access in our heads. The knowledge is there, in the world. We only need to use it while doing other things. We want to open doors and switch lights without having to think about.
So, when we have to think about things that should be obvious, we are wasting part of our mental resources, allocating space in our working memory that we’d rather use for another purposes.
Knowledge is in the world when all clues we need to use an object are present in the object itself. So, we don’t need to care, reason or retrieve how to operate that thing. When we need to do that (to reason, to remember instructions, etc), we need to put the knowledge in our heads in order to
achieve our goals.
So, with more knowledge in the world when we perform a task, we need less knowledge in our head, leaving free room we can use, for example, to think better about the task at hand.
The less I have to think in the tools, the more I can think in the task I’m performing with those tools.
But, how much room is available?
Well… it sounds disappointing: not too much.
We have a practically infinite storage capacity in our memory. Think of it as a really huge and intelligent hard disk, that keeps memories and data for years. It is not a passive storage. In fact, it rebuilds our memory in order to store and retrieve things. This is important because when we store or retrieve things, we need to use our working memory in order to hold the data we are using. Very similar to a computer, isn’t it?
However our working memory is a pretty different thing than our long-term memory. Some call it “short-term memory”, and others “working memory”… I think you could see it as a processor, with some registers that can store a limited quantity of information units called chunks while working. Chunks can have variable size, but they are meaningful.
Do you remember a phone number? I bet you group the digits to have only two or three numbers to retain.
That’s because our processor can manage a limited number of chunks. Something around seven (plus or minus two), This varies with age and individuals, but it is a very good approximation. So, we try to save as much “registers” as possible, grouping information in chunks, and leaving some of them free.
What happens if we fill up all the registers? Well, task precision and performance decreases, errors increase. In general, we perform worse if we try to maintain many things in our working memory at the same time.
Of course, this is an over simplification. However, I think you could get the big picture. We can reduce the overload if we put knowledge in the world, instead of maintaining it in our head, and we will perform better at any task by doing so.
You can put some knowledge out of the working memory by practice. That’s why when we introduce a new technique, or a new language feature, or a new tool, etc, we need to go slower and we have some more errors in our work. We need time to automate things in our mind while putting knowledge in the world.
In our work we need to manage tasks that cannot be automated, we usually work in tasks that are new every time.
And now, it’s time to return to the main goal of this article. Let’s talk about a developer’s life.
Let’s analyze for a moment what happens when we program without tests…
In fact, we always do tests, but we tend to do manual testing. This is what we call debugging. We use a trial and error process: Does this work? No…? Try again. Yes…? Go ahead.
The process is best reflected in this picture: We try to write code and verify that this code works as we want at the same time we write it, until we think it’s done. After that, we try to verify that the code works as a whole, and, then, we remember that we forgot some details… We deploy, and we discover new details that don’t work, so we need to fix them.
At the end of the day we’ll find ourselves with big headaches and under the impression of having missed something.
This happens because we try to manage all the information in our head at the same time (remember, it is limited), we overcharge ourselves. The best strategy is to write down goals, subtasks, helping us with these external aids.
For example: a simple endpoint for an API could need a lot of things:
- A controller action
- A route to this controller action
- A use case or command object to perform the action
- Probably an entity (or more) and the repository
- The repository DIC definition
- Maybe a service
- The service DIC definition
- A response object
Our memory becomes overcharged. This explains why we feel tired and stressed, and with the feeling that we may have forgotten something, and unsecure about what we are doing or if we have left out something important.
So, let’s take a look at how we would execute the same process, this time with testing at the end.
Well. This looks familiar, but now there are tests at the end of the process. The kind of tests we automate.
The end result is better, because we are more confident about our code thanks to the tests. But we have the same headache at the end of the day.
Yes, we have been doing the same amount of work, with the same memory overload and with the addition of having to write a bunch of tests, while our brain is screaming: “Hey, dude, work is done! What the heck are you doing?“
In that conditions, maybe our tests are not the best tests in the world…
In fact, we are already tired when we start the testing phase.
This explains why many people think that testing is hard and that it hurts.
So, tests improve our confidence in the code, at the cost of lots of extra work. Our life is not better with tests, even if we sleep better at night…
What’s wrong?
To really improve your life, you should try a different approach. You should do TDD.
This is the TDD cycle: one thing at a time (and to postpone decisions):
- A simple test that fails (don’t write code until you have a test)
- Code to make that simple test to pass (don’t write anything more or less than what is needed)
- Review the code to improve things, like naming, architecture, apply patterns, reduce duplication, but don’t implement anything new keeping the tests in green state.
Here you can see the process from the point of view of our working memory model. When we write the first failing test we are focused on that test. So we don’t want to pay attention to anything out of this. Writing the tests also means that we are putting the knowledge we will need in the world. Our memory is almost free.
Next, we focus on writing the code needed to make the test pass. The knowledge we need is in the test, that happens to be the goal of our task, so it is in the world, not in our head.
We only have to think in a way to make the test pass, so, if it is the first test, we only need to write the more obvious implementation that is possible. Even if that implementation is one as simple as to directly return the value expected by the test. Don’t need to worry about anything else.
And then, once the test has passed, we can take a look at the code and see if we can make some improvements via refactoring. We don’t have to add features. We must keep the test passing while tiding things, removing undesired duplication, etc.
We’ll repeat the cycle till we have the feature fully implemented. We don’t need to write extra tests, we don’t have the risk of forgetting something. Our head doesn’t hurt. We have used our brain to think, avoiding memory overload.
It’s not magic, it’s TDD. Of course, to achieve this needs some training. TDD is an intellectual tool, and tool use needs to be automated.
Therefore, you should be doing exercises, like katas, both for yourself, both with some colleagues’ help, in a community of practice… whatever fits you or your team. Practice, practice and practice. Once you are able to proceed step by step, you will discover that not only things become easier, but you will be happier and less stressed in the long term.
A final advice:
Store the most of knowledge you need in the world: use a backlog, post-its, write-down a list of tasks, draw schemas, models, concept maps…
Free your head to leave room to work on one thing at a time.
TDD is more than tests. It is to put the knowledge you need in the code (in the world) and free your mind. It is to postpone decisions to the time when you are ready to make them.
Really, try TDD, your life as developer will improve.
Top comments (8)
I've tried TDD many times and found it to be worse than TLD (Test Last Development). With TDD, don't you find that design suffers and it hinders your ability to refactor code beyond the internals of a function?
Thanks for your comment :-)
In fact, my experience is the opposite. By doing TDD I feel myself less tied to a particular implementation, so I can start with a rough and naive approach and improve the design iteratively being well protected with the tests at every moment. It is far easier for me to explore ideas, extract methods or classes and, in general, develop a better design and code less prone to defects.
When I write code without TDD I tend to "fall in love" with my current approach and when I return to this code after some time I detect more design flaws and defects.
How do you do design with TDD? That's one of the things that doesn't make sense to me.
For example, imagine I need to write a new use case for an application.
I first start TDD'ing the use case as if it would be the unique class I'm gonna need.
Usually I start testing expecting exceptions because they usually are the simplest, but this may vary. I look for the simplest behavior. Sometimes this is very straight, some times it is a bit difficult to find it.
With every new test I may discover that I need collaborators for my use case in order to get data or services. Maybe it is a repository that I will need to double for testing, maybe it is some kind of service that already exists in the code, so I usually double it also. Sometimes I prefer use it as is.
Test by test I write the needed code to resolve my problem, and when I'm in the refactor phase I extract parts to private methods in the use case, that can lead me to extract some behavior to a new collaborator. This new services and collaborators are covered by the initial use case tests, so I don't need to separately test or add them (the collaborate with the Use Case to perform the behavior). I will add tests if I want to be more confident about them.
Of course, depending on the outcome of the UseCase (Is it a query returning something or is it a command?) I will need to test for a response or for a side effect, maybe I need to use a mock to verify some behavior.
Hope that helps.
By the way, this post by Uncle Bob is very interesting to understand this approach:
blog.cleancoder.com/uncle-bob/2017...
That's completely new to me in terms of TDD, but then again it doesn't sound like TDD and more decoupling. What about unit tests?
I'm not sure if I'm explaining well the methodology, but maybe you can see it in action in this example (text is in Spanish, but I think you can follow the code samples to get the overall idea)
leanpub.com/testingytddparaphp/rea...
I like this recent tweet from Ron Jeffries:
From his point of view, we must always keep design in mind at all times. This is regardless of using TDD or not using TDD. TDD might help with the design since the "refactoring phase" is really the thinking about design phase.
I've recently learned refactoring means not changing public API of a system: only the implementation should change. This might be what you meant by TDD only helping with changing the internals of a function.
fagnerbrack wrote a wonderful article on this exact subject:
Changing the API for something that is widely used through the system is hard, risky work. TDD helps us here, as a side benefit, of giving us a pretty comprehensive test suite that covers the entire system, since no functionality should be added without tests.
In these cases, one technique I use to improve the design of an existing system, is to write a new test using my "dream api". The API I wish I had.
We start with our existing API.
I might think my design is bad, and I'll start by playing around with the design in a new test:
Maybe this might handle future inputs better?
If I like this design, I now run into a new problem: this API is used in 3000 parts of the system.
To solve this problem, we'll incrementally switch existing consumers over to the new function.
Start by giving the new Sum function a different name: Sum2?
Now, in the existing tests, update all occurrences of Sum to use Sum2.
should be changed to
The test should pass and we know Sum2 is compatible with every behavior that Sum implemented.
Now, you can begin the process of slowly transitioning every consumer of Sum to use Sum2:
should be changed to
After transitioning a few of them, I should pause and think about whether my new design is better or worse than the old one (In this case, I've actually made it worse: back to the drawing board...)
It may not be that TDD makes design suffer, but redesigning the API of heavily used components is painful work.
Hopefully that helps a little bit! ;