Cost of Code Ownership

If you spent any time buying a car, I am sure you have done your research and asked: How much will this car cost to own? Will I finance the car with a loan from the manufacture or use a bank or credit union? How much will the insurance on this car cost? How much will it cost to maintain it? What is the deprecation? And many more questions. All of this factors into the total cost of owning a car.

While we may spend a lot of time figuring out what the cost of owning a car is, have you ever stopped and thought, “The code I write each day, how much does it cost me? How much does it cost my team? How much does it cost the organization I write it for?” These are important questions we must face. If you look at the industry in general, often we prize the mythical “10x Dev” that can run circles around everyone else. The guy or gal that can get things done and write lots and lots of code. But should we be optimizing for this? Rewarding people for it?

Code should be viewed as a liability

I am reminded by a famous quote from Edsgar Dijkstra:

My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger.

Why did he say this? One reason comes to mind, complexity. As we write more and more code, things get more complex. We see this all the time in the industry with greenfield projects. Have you ever wonder why startups can just get things done and run circles around larger companies? Why is every thing so much easier in a greenfield project? The reason is that we don’t have all the baggage of an existing code base. Before we write a single line of code, our opportunities are endless. There is nothing to try to understand. There is nothing to integrate with. There is nothing to test. It’s all new and and exciting!

Fast forward a few years and what often happens? As we add more and more code, things get harder to understand. We can no longer keep a working model of the system in our heads. And if we have added new people to the project, this only compounds the complexity issues. Also lots of us, including myself, often are learning as we go. This means that we might not even know how to write code that fights this complexity and we accidentally make things worse. Thus even skilled devs armed with the best engineering practices face growing complexity just from the fact our system gets larger and larger with every line of code we write.

Accidental complexity

This takes us to anther point, if just the act of adding code creates complexity, bad code makes the situation even worse. Dan North had this to say:

I call false dichotomy. I've said elsewhere the goal of software development is to "sustainably minimise lead time to business impact". You can't do anything sustainably without taking engineering seriously. Poor code quality is only non-user visible in the short term.
— Daniel Terhorst-North (@tastapod) March 4, 2019
https://twitter.com/tastapod/status/1102570605986103297

Bad code quality does not just effect the developers, it effects the business. If the complexity is not managed and minimized through good software engineering practices, both lead times and through put slows to a crawl. The once lighting fast startup grinds to a halt as features take longer and longer to add. This in turn costs the business money because of all the missed opportunities. It also starts to create a rift in organizations, as managers try to “fix” the issue by issuing deadlines and ultimatums. The developers start to get stressed and want to leave this toxic situation. The business suffers even more as developers start leaving for greener pastures. This in turn creates a powerful feedback loops as either new people are thrown at the problem, or the team is forced by management to start cutting corners with testing or other practices that ensure quality. The few experienced people that remain end up either fixing messes others make, and/or training all the new people.

What can we do?

At this point might seem hopeless situation, as we add more code, things will just get worse. But there is a lot we can do. Here are a few things we will go over now:

• Everyone needs to be aware of the state of the code
• Uphold software engineering best practices
• Do not optimize for code production, focus on outcomes
• Removing features that are no longer bringing value, delete dead code
• Balance is needed

Everyone needs to be aware of the state of the code.

Business needs to have a clear understanding of what shape each major areas of the code. Far too often in agile circles, features are treated all the same. Feature1 and Feature2 are often ranked only in terms of value to the business, but often the complexity of adding any of these features is not well understood by management. Clear and open communication is needed here, and we need clear and understandable ways to communicate the state of each major areas of the code to the business. Michael Feather gives some good ideas in this talk about how to communicate the level of technical debt in the code to the business here. One idea, use hypothetical features you would add to each major area of the code base and track the estimates of the devs on the team over time. As you change the system over time, you can see how those changes affect the estimates.

Along with this clear communication, both developers and management need to take a hard look at why did we end up with a mess in the first place? Is management giving unrealistic deadlines? Are the managers telling developers to start cutting corners to get things finished? Are the software developers giving too rosy estimates? At the end of the day the organizational and team culture have a powerful effect on the kind of code software developers write. If you want to make this better you have to have the right culture in place!

Uphold best engineering practices

As already mention, the worst thing we can do when under schedule pressure is start cutting corners. Managers might think that stopping refactoring, cutting back on unit tests, and other such measures will help get features out the door. While this may work in very short term, such decisions can have a lasting impact on the project. Can't say this better than Mark Seemann:

I've also often observed managers professing preference for a particular developer who 'gets things done.'

What that developer often does is to declare a lot of tasks to be 'done', leaving others to clean up his/her mess.
— Mark Seemann (@ploeh) March 3, 2019
https://twitter.com/ploeh/status/1102301536443600897

What is the answer? Test driven development is so important here. First it ensures that we have good suite of unit tests. This in turn gives us the safety net needed to do refactoring. Often times I see developers afraid to touch certain areas of the code, and never want to refactor it. Why? It is hard to understand and few or no unit tests which makes it easy to break. So we must be disciplined about unit testing, and the one thing I have seen help with this is test driven development. Because we put tests first and depend on them when making changes, this incentives the developers to ensure that first their code is testable, and second that the tests are quick.

But why do we need to spend so much time refactoring? The same reason we need to use drafts in writing. Often the first draft of something is not our best work, and often takes few revisions before we come up with something that is not only better, but easier to understand. The same is true of our code. Often the first draft is not our best work, we where just focusing on trying to get the code to work! Now we need time to make it understandable and maintainable.

Software development productivity has little correlation to how fast you can produce code.

It's closer related to the total cost of ownership of that code.

Some code, you can write in one hour, and then proceed to waste days or months maintaining and troubleshooting.
— Mark Seemann (@ploeh) March 1, 2019
https://twitter.com/ploeh/status/1101442797436055552

Remember what Mark Seemann is saying here, just because we can churn out code fast does not mean we want to or should. We need time to refactor and revise our code. If we just rush through things, there is a good change we will end up creating code that no one understands, increasing the complexity, and creating more work for our fellow teammates.

Also we have to consider the architecture of our system. If we have a small team might make sense to have a mono repo with everyone working out of it to keep the complexity low. If we have several teams working, we may want to pursue a micro-service architecture to allow teams to focus on the parts of the system they own and have well defined contracts to communicate with the services that other teams own. Look at your situation and take time to pick the right solution that will help minimize the complexity your facing.

One final point to keep in mind, focus on your needs today, not what might happen in the future. Often times I see developers get so caught up in making code handle thousand different cases, when we really only care about one of them right now. Write only the code that you need to solve the problem at hand today. Don’t try to anticipate future needs that may never come. If you guess wrong, you end up with a lot of dead code that only confuses people.

Do not optimize for code production, focus on outcomes

Do not optimize for code production, ever. Why such a strong statement? Goodhart's laws states:

Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
https://en.wikipedia.org/wiki/Goodhart%27s_law

This means that a measurement, once it is used to evaluate the performance of anyone will eventually become useless. The reason is people will start to optimize for this measurement. So if we reward developers for the lines of code they produce in a day, or the number of commits they make, this will naturally encourage them to produce more code. They will do this even if it harms the project or destroys teamwork and pairing. People might not even be aware they are doing this. History is filled with such examples. This is also called the cobra effect and if you want to hear what happened to the British when they tried this in India you can listen to Freakonomics podcast on it here.

Focus on outcomes for the business, what problems are trying to solve, and think about ways to measure for this when you release your software. At then end of the day we are trying to help someone solve problems with the software we are creating. If you want to get started in the right direction Jez Humble gives a lot of great ideas in his Goto conference talk here.

Removing features that are no longer bringing value, delete dead code

We don’t want to be a code hoarder, trying to keep every line of code, leaving commented out code through out our code base. Source control is our friend, we can get this code back if we need it later. The important thing here is we need to remove dead code. Less code is less complexity for everyone.

At the same time we need to take a hard long look on the feature set of our application constantly. The natural progression of most teams is just keep adding feature after feature, but each of these features adds to the complexity of the code. Remember that up to 2/3 of our features will either do nothing, or have a negative impact on the business. Thus the business shouldn’t be hording features either. We need to be constantly thinking and analyzing, "Which features do I need? Which can I safely get rid of?" If we don't do this, we will eventually choke on features that no one is using or worse costing us lots of money.

In this article here, Michael Feather proposes a radical solution. The code should auto destruct and get deleted after a few months. This he argues would force teams and businesses to really consider what's important and fine ways to keep only the software that the users want or need.

Balance is needed

In the end we must remember that this is software engineering. Everything has trade offs and balance. There are extremes on either end. Remember that we are not arguing here that all complexity is bad. We can trade complexity in one area to give us an advantage else where. The point is we cannot let these decisions happen randomly or give little thought to them. If we put in practices to help us manage complexity, we can help ourselves and our teams make better software and deliver it with more agility. I hope this article is of use to you and remember that we all need to worry about the cost of code and we can manage it!