How to be more successful at software maintenance

In this post I want to explore the kind of software maintenance that happens after active maintenance ends. This is the stage of the software life cycle where the software has been in production for some time and the initial wave of bug fixes and enhancement requests ends. It's also at this point that managers may cut the number of developers assigned to the project and turn their attention elsewhere.

Keeping this kind of project running safely, securely, and profitability as the years tick by--even if you're not adding or changing functionality--is especially challenging.

I've spent more than a decade in this situation. And I'd like share some of my observations and advice about maintaining this kind of software with you.

The 8 laws of software maintenance

If your software lives long enough you'll experience one or more of the following:

1. You'll have trouble getting enough resources to maintain your project

Keeping old software running isn't sexy. Managers are more interested in acquiring new functionality than maintaining older systems. And, very few programmers want to spend their careers doing maintenance so both talent and money may be in short supply.

2. Your hardware and software dependencies will become out-of-date

The languages, frameworks, libraries, APIs, development tools, build systems, and hardware used by your project are constantly shifting under your feet. Some stacks are shifting faster than others but nothing is standing still. And the more dependencies you have, the harder your project will be to maintain.

3. Your code base will be a mess

I can say this with a fair bit of confidence because most code bases are a mess. But very old code bases can be especially problematic. They might have been developed without unit tests, documentation, static analysis, coding standards, style guides, code reviews, or modern development methods.

Very old code bases have frequently been worked on by several programmers with different programming styles over time. And if your software has been underfunded or developed with speed as the top priority, it probably contains heaps of low quality code.

4. Your memory will fade

Your understanding the requirements, design decisions, and how your code works degrades rapidly after active development stops. People forget the details or take valuable knowledge with them when they move on to other projects. And your documentation and test coverage will almost certainly be inadequate to make up for that missing knowledge.

5. You'll lose something important

Information, software, or hardware critical maintaining, building, testing, documenting, or deploying your software may be lost. Think config files, compilers, compiler switches, hardware configurations, encryption keys, source code for dependencies, drivers, backup scripts, development environments, test environments, etc.

Losing any single critical component of your project could mean that it cannot continue to operate without heroic effort on your part. And some problems--like losing the contents of your database--can leave you with no choice but to shutdown your project.

6. You'll need to change your software

You may not want to change it but, if your software lives long enough, you'll likely discover a good reason to change it.

Reasons include:

• new requirements
• defects
• obsolete dependencies
• security vulnerabilities
• new legislation or regulations (privacy, PCI DSS, etc.)
• hardware failures
• hardware obsolescence

Always assume you'll have to make a change at some point in the future no matter what anybody tells you.

7. Maintenance is difficult and expensive

This is the stage of your project where your chickens come home to roost. The decisions you and your predecessors made earlier in the software development life cycle will determine how painful and expensive keeping this system running will be. But, no matter how much care and foresight everyone exercised, maintaining old software is almost always difficult and expensive.

Cleaning up the code base to make it easier to maintain (aka refactoring) is tempting to most programmers, but it is often a terrible investment.

Large sums of money are wasted on ill-advised code cleanup projects that will never be paid back by faster future development. For example, I currently maintain tens of thousands of lines of code that haven't been meaningfully modified in years and may never be modified again. So any refactoring or cleanup I do on that code is almost certainly wasted. It doesn't matter how ugly I think that code is.

You might be able to make economically advantageous, highly targeted changes if you look hard enough. But you are largely stuck with whatever you have at this point.

8. Your software will stay in production much longer than anybody anticipated

This is certainly not true of all software but if you restrict yourself to the types of projects we are discussing in this post (software in the late stages of maintenance) you'll find that this type of software tends to be hard to kill for various reasons:

• replacing it never gets to the top of anybody's to-do list (regardless of what anybody says)
• it's just too expensive to replace
• the replacement project is late or fails to meet user needs
• the replacement project never delivers working software (because the project failed or was cancelled)
• your users can't or won't upgrade (for various reasons)

How to make software maintenance better

Like I said earlier, keeping this kind of project running safely, securely, and profitability as the years tick by--even if you're not adding or changing functionality--is especially challenging.

Your biggest risks are that:

• you waste your maintenance budget on low value activities
• an unexcepted event cripples the value of your project and/or brings it to a premature end

There are no one-size-fits-all solutions to this kind of software maintenance but there are some things you can do to make your life better. Your first step is to do some analysis.

How and when will you retire this software?

What's the retirement plan for your software? Is it expected to live for another year? 5? 10? 50? Do you plan to develop a replacement product and migrate your users to it? Does this software need to be changed to enable the migration? How realistic is the delivery date? Or will you retire the software without replacement? Or do something else?

What do your stakeholders want from this software between now and the time you retire it?

Don't assume you know what's important to them. Ask them. You might be surprised by what they say.

I did just that with the owners of a project I had been maintaining for over 10 years. I thought I knew their preferences very well. But I setup a meeting to ask them (just to be safe). I knew they didn't care about technical debt or code quality but I was surprised by their views on security, up-time, disaster recovery, and several other issues.

What is the value of your software?

How much money will your software make (or save) you each month or year until its planned retirement? There are several ways to do this but, if you are looking for a recommendation, I suggest you start with a net present value analysis.

You should also account for non-monetary factors in the value of your software. For example, if you promised your biggest customers that you would support the software for another 5 years, it would be wise to take that into account in your analysis.

You may want to get the business types involved in this analysis to help you figure things out.

What will it cost to keep your software in production until its planned retirement date?

What does it cost now? Is it reasonable to project that number into the future with some sort of inflation factor? What unavoidable maintenance tasks will you have to perform? For example, you might expect to migrate to new servers every 5-7 years and migrate your code to a new version of your language every x years. There are also one-off concerns like the year 2038 problem. Or maybe you know you'll need to replace one of your dependencies because its end of life date has been announced.

There are also costs that are harder to pin down. For example, is your project written in a language that is no longer popular? How will you find people with the skills to maintain it all the way to its retirement? Are you on old hardware? How will you keep that running? Or what's the probability that you have a security breach that costs you money or damages your reputation?

It's natural for people to underestimate how much effort it takes to keep software running. So any analysis you do will be more realistic than just throwing a number out there.

How risky is your "maintenance debt"?

Maintenance debt is a relatively new term. It is described in this paper as follows:

We introduce the term "maintenance debt" for maintenance needs generated by an implementation’s dependence on external IT factors such as libraries, platforms and tools, that have become obsolescent. The application continues to run, and the IT department forgets this theoretical liability, focussing on more urgent requirements and problems elsewhere.

Such debt accumulates over time, silently eating away at the value of the software asset. Eventually something happens that makes system change unavoidable. The owner may then discover that the system can no longer be modified--it is literally unmaintainable. Less dramatically, it may take too long, or cost too much, for maintenance to solve the business problem, and an alternative solution must be found. The software has suddenly crashed to £0 value.

Is it worth keeping your software in production until its planned retirement date?

It might become clear to you that your planned retirement date is uneconomical. Maybe you need to move your retirement date up? Or reduce your maintenance budget? Or increase how much you charge your users?

Perhaps you can purchase off-the-shelf software that largely does what your software does for a fraction of the cost and migrate to that.

Or maybe you realize that your software is extremely valuable and it makes sense to devote more resources to its maintenance to make it even more valuable and/or push back the retirement date.

Yet another option is that your software is a money pit and you need to figure out if you should rewrite or refactor it. Or perhaps the best thing to do is shutdown the project immediately without having a replacement (as Google is fond of doing) and work on something completely different.

The important thing is to do the analysis. The business types respond to data. And, even if there's quite a large level of uncertainly in your estimates of cost and value, it's a good starting point to having an intelligent conversation about the best way to maximize the profitability of your software.

It's all about risk management

If you've made it to this point and you've discovered your software is worth maintaining in some way, you'll have to decide how you are going to spend your maintenance budget.

You'll need to assign some of it to your "must-do" maintenance tasks. Then you can take whatever's left and use it to reduce the risk that something bad happens to your software.

For example, if you are worried about a hardware failure, maybe you work on your backup, replication, or redundancy. If you are worried about key people leaving the project, then you might focus on employee retention, letting your developers work on their software pain points, and documenting the system.

My point is that it's unlikely that you'll have enough resources to do everything you might want to do to maintain your software so you'll need to prioritize your efforts.

Additional things you can do

In addition to everything I described above, I think it's wise to do the following two steps periodically.

1. Revisit your analysis from time to time

The value of your software will change over time--often in unanticipated ways--so it's wise to redo your analysis periodically and make adjustments to your maintenance plans as necessary.

2. Build and deploy, even if you have no changes

You might consider updating your dependencies to recent compatible versions, building, testing, and deploying periodically. Even if you have no changes to integrate it's worth something to demonstrate that you can still do it successfully. It's also a good opportunity to look at your hardware and documentation to evaluate how well it's meeting your needs.

How often you build and deploy is up to you but every six months or so seems reasonable to me.

The scenario you want to avoid is that you need to make some small change and discover that its impossible or nearly impossible because you don't have a critical piece of information, software, or hardware required to complete the process. At that point all your options are bad (and probably expensive).

Wrapping up

If you find yourself responsible for maintaining a software system after active maintenance ends I recommend you focus on striking a good balance between squeezing as much profit out of it as you can and taking steps to reduce the most serious risks of something catastrophic occurring to the system.

Following the steps outlined in this post will help get you going on the right path. And, remember, it's your stakeholders who decide what makes your software valuable, not you. Good luck!

Additional resources:

• Software Engineering at Google (book)
• Working Effectively with Legacy Code (book)
• Why governments can't migrate away from cobol (article)
• Maintenance Debt (research paper)

Have a comment, question or a story to share? Let me have it in the comments.

Enjoy this post? Please "like" it.