Hi @toureholder
. Thanks for the pointer to the study. I was really happy to see TDD as a serious research topic. I read the study, and a couple of the referenced studies as well. But sadly, I'm a bit disappointed by the approach most of the studies used.
My first problem with these studies is the tasks given to the subjects. In the referenced study (and a couple of others), they used the "Bowling Score Keeper" kata as the task. While I'm sure it's not an easy task to come up with a decently sized challenge, I still believe that a simple kata is far from being a realistic example. I mean... it's a popular kata. A finger exercise. Nobody I know is paid to build a simple "Bowling Score Keeper". If the challenge is too simple, an experienced developer might have the solution right in his head. In that case, TDD might indeed "slow him/her down". But in "the real world" (meaning our everyday life), tasks are complex. We have to build systems talking to other systems. Reading things from the database, sending data to message queues, talking to REST services, handling strange user input. We have to deal with connectivity issues, legacy code, and much more. Life is not a greenfield project.
Another interesting thing is the participants chosen by the researchers. In most of the studies, students are the participants. In your referenced study: 21 graduate students.
All the participants, before the study, received 20 hours of training in unit testing in Java, using JUnit as a framework and Eclipse as the IDE
20 hours. Is that enough to be sufficiently proficient? I don't know. Anyway, it's a bit questionable to make general statements about the effectiveness of TDD based on this data.
Sorry, if I may sound a bit grumpy. But, as I said, I was really to see TDD as a research topic. But after reading some studies, I'm kind of disappointed. I don't know if you read it, but "The Effect of Experience on the Test-Driven Development Process" by Matthias M. Müller and Andreas Höfer is a really interesting study. Here is what they found while comparing TDD experts with TDD novices:
The experts complied more to the rules of test-driven development and had shorter test-cycles than the novices. The tests written by the experts were of higher quality in terms of statement and block coverage as well. All reported results are statistically significant on the 5% level.
In any case, I thank you for the inspiration to dive into the TDD research. Reading the studies was super interesting.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Hi @toureholder . Thanks for the pointer to the study. I was really happy to see TDD as a serious research topic. I read the study, and a couple of the referenced studies as well. But sadly, I'm a bit disappointed by the approach most of the studies used.
My first problem with these studies is the tasks given to the subjects. In the referenced study (and a couple of others), they used the "Bowling Score Keeper" kata as the task. While I'm sure it's not an easy task to come up with a decently sized challenge, I still believe that a simple kata is far from being a realistic example. I mean... it's a popular kata. A finger exercise. Nobody I know is paid to build a simple "Bowling Score Keeper". If the challenge is too simple, an experienced developer might have the solution right in his head. In that case, TDD might indeed "slow him/her down". But in "the real world" (meaning our everyday life), tasks are complex. We have to build systems talking to other systems. Reading things from the database, sending data to message queues, talking to REST services, handling strange user input. We have to deal with connectivity issues, legacy code, and much more. Life is not a greenfield project.
Another interesting thing is the participants chosen by the researchers. In most of the studies, students are the participants. In your referenced study: 21 graduate students.
20 hours. Is that enough to be sufficiently proficient? I don't know. Anyway, it's a bit questionable to make general statements about the effectiveness of TDD based on this data.
Sorry, if I may sound a bit grumpy. But, as I said, I was really to see TDD as a research topic. But after reading some studies, I'm kind of disappointed. I don't know if you read it, but "The Effect of Experience on the Test-Driven Development Process" by Matthias M. Müller and Andreas Höfer is a really interesting study. Here is what they found while comparing TDD experts with TDD novices:
In any case, I thank you for the inspiration to dive into the TDD research. Reading the studies was super interesting.