My survey of the computer science literature suggests there are only two economical ways to achieve extremely low defect rates (< 1 defect per KLOC). The first way is to follow the Personal Software Process (PSP), which was created by Watts S. Humphrey at CMU. The second way is to use languages and tools that make it difficult to introduce errors into your code in the first place and easier to detect errors if you do manage to get some into your code. In this post I'm going to briefly discuss these two options and how I plan to explore them to become a better software developer.
But first, why do I want to do this? Because my experience building and maintaining software roughly tracks the data coming out of the research: defect-riddled code and all the testing, debugging, correcting, and rework that go along with it are incredibly wasteful. Our industry is overflowing with failure. There's got to be a better way.
I've already written at length on working on the most important thing, avoiding waste, code review checklists, optimal pull request size, and so forth. So, in this post, I'm going to focus solely on what it takes to produce software with very low defect rates.
PSP might best be described as the application of lean manufacturing principles to the activities of an individual developer. Programmers tend have poor insight into the various ways they make mistakes and create waste. They make the same mistakes over and over again and think they are awesome (me included). But the truth is that you are the one who is putting all those errors into your code, writing complicated code, writing unmaintainable code, and so on.
PSP teaches programmers how to track their work and their mistakes. And it provides programmers with strategies they can use to improve their performance over time. From what I hear, people who follow this process can improve dramatically.
It's very difficult to write correct code in most languages. They're ambiguous and they allow programmers to do things that are almost certainly errors. Things get more interesting when you switch to a safer language like Ada and/or its ultra-safe subset called SPARK. This fits very well with my idea that programming languages need to become significantly safer by default if we're ever going to improve software security.
Let's talk about Ada first. Strong typing, ranged types, contracts, and a really smart compiler help you avoid the stupid errors that languages like C/C++ and their derivatives ignore. So, if you choose to program in Ada, you're going to have an easier time detecting and localizing defects that you wouldn't even notice with another language.
If you want to take things to another level, you can look into using SPARK, which is a subset of Ada that can apply formal methods to your code.
I know it sounds scary but I've been playing around with SPARK and I think it's awesome. First off, you don't need to know or do any serious math to use SPARK. SPARK can prove (or at least attempt to prove) your Ada contacts and type constraints automatically. If you just stopped right there, SPARK basically acts like that best static analyzer you've ever seen.
But you can go further. You can add assertions to your code and SPARK will attempt to statically prove that they will never be violated. How far you go is up to you but you can get your code--or some important parts of it--to the point that SPARK can demonstrate an absence of run-time errors. That means it can't crash from overflows, divide by zero, etc.
Why would you want to do that? Four reasons:
- Just like with TDD, writing your code in a way that SPARK can prove results in really well designed, simple, easy to read code.
- If SPARK is guaranteeing that whole class of errors can't happen, you don't need to write tests for as many paths through your code.
- There's a strong correlation between absence of run-time errors and the absence of logical errors. That means that code proved by SPARK tends to have fewer logical errors than code that hasn't been proved with SPARK.
- Once you understand what SPARK can do, you can stop thinking about those things when you are coding. It's similar to the way you don't have to agonize about your spelling when you write an email because the spell checker handles that for you. It's really impressive when you see it in action.
There's not much information on the web about Ada/SPARK (compared to more popular languages). And much of what's out there is outdated or just wrong. Both Ada and SPARK went through major changes in their latest releases that addressed some real pain points. Plus, Moore's law has made compiling and proving the correctness of your software orders of magnitude faster than they used to be.
I'm already experimenting with Ada and SPARK. This kind of programming is a little different than I'm used to but I found some good resources and things are starting to click (see resources below). I plan to build a few toy applications with Ada/SPARK solidify my learning.
For a bunch of reasons I won't go into here, it's unlikely that Ada will ever be a mainstream language. So, I'm not under any delusions that I'll get to write production code in it any time soon. But, I can already see that some of the concepts can be applied to any programming language with very little effort. And I'm looking forward to using some of those ideas to improve my code at my day job.
As for PSP, I ordered the book PSP: A Self Improvement Process For Software Engineers by Watts S. Humphrey and I plan to work through it and the accompanying programming exercises straight away. PSP is language agnostic so if you're interested in upping your game immediately, you might want to start with PSP and explore Ada/SPARK later.
I've been studying PSP for the last few months and I've written a summary of what I've learned so far.
Both PSP and Ada/SPARK are more prominent in the safety-critical, mission-critical, and high security sides of our profession. I've never heard any of the web development gurus mention this stuff so, I have no idea how well these tools will apply to my day job as a back-end ecommerce developer. My guess is that almost anything I can do to prevent defects from getting into our code base will be worth doing.
The research is very clear on the rising costs of fixing defects in the later stages of the development lifecycle. So, I can give up some speed on coding to save time on testing, debugging, and unplanned rework. The trick will be finding the sweet spot.
Would you ever consider learning PSP or Ada/SPARK? I'd love to hear your ideas in the comments.
How to achieve ultra-low defect rates:
Paper: The Fumble Programmer by Rod Chapman
Video: Murphy vs Satan: Why Programming secure systems is still hard by Rod Chapman
Should we trust computers? (video or word doc) by Martyn Thomas
Making Software Correct by construction (video or word doc) by Martyn Thomas
Safety-Critical Systems (video or word doc) by Martyn Thomas
Book: PSP: A Self Improvement Process For Software Engineers by Watts S. Humphrey
Video: An Introduction to PSP by Watts S. Humphrey
Pluralsight Course: An Introductions to Ada 2012
Contract-Based Programming in Ada 2012 (video or slides) by Jacob Sparre Andersen
Website: Introduction to embedded programming with Ada
Book: Building High Integrity Applications with SPARK by John W. McCormick and Peter C. Chapin
Website: Ada Information Clearinghouse