Making Software Development More Predictable

Look up data on the success rate of software projects, and you’ll likely be unpleasantly surprised at how often they’re considered failures. There are many reasons software projects fail, from busting budgets to missing deadlines to failing to fulfill the clients’ goals. Whatever the ultimate reasons, the majority of those failures stem from the unpredictability inherent in software development.

In this article, we take a look at some of the causes of unpredictability in software development, and then we outline best practices to help manage that unpredictability and therefore make the project more successful.

Drivers of Unpredictability
Chaotic Systems
Software engineering projects are unpredictable by their very nature, because each one is a complex socio-engineering system. And “complex” in this context means something very specific — much like a double pendulum, these systems aren’t random, and yet they aren’t predictable, either. The double pendulum, with one pendulum attached to another, seems simple enough at first glance, as its movement is governed by a set of straightforward differential equations. But try to predict the pendulum’s position at any point in time, and you’ll realize those equations aren’t at all straightforward after all, but very difficult to solve using traditional methods.

Why? They’re extremely sensitive to initial conditions.

In our experience, software engineering projects are similar in nature. We all know which project management models to use and how to build teams, develop software, and debug systems, but these projects are highly sensitive to the contextual conditions. Those models and strategies and systems aren’t quite as “plug and play” as one might expect. But the good news is that, while the systems aren’t predictable, they have stable “equilibrium” states that we can target as we set up and implement the project.

Speed of Engineering
It seems, these days, that speed and agility are the end all, be all for software development clients. And in that haste, it’s easy to overlook a critical step in the development process: modeling a system before beginning construction.

Engineers are responsible for thinking ahead, imagining and modelling and optimizing their projects before they start building. This doesn’t mean producing detailed, bureaucratic documentation for the entire system upfront, but it does mean thinking carefully and modeling at least the next piece to be built.

When engineers dedicate the mental energy and self-discipline to thinking ahead about the architecture of a system and its components, the build process goes much more smoothly. When we consider what’s coming before we jump in, we run into far fewer road blocks and surprises along the way.

Unrepeatable Processes
Unpredictability is a two-way street: a project may fail, but it can also be a success. And when a project succeeds, the next question is, how do we repeat that? If processes aren’t sufficiently documented, then the next attempt to repeat that success — either in updating the application or replicating it elsewhere — will be just as unpredictable as the first.

As a service-based organization, Syberry has a bit of a leg up. We have a lot of different projects and products under our belts, and we and use the knowledge we gain from each to steer the next ones. When things don’t go well, properly analyzing mistakes and adjusting processes accordingly is critical to consistently improving our service.

Building a single engineering team to design and create a new application is no easy task, but it can be done. It’s like building a Tesla prototype — difficult, but doable. But as soon as you ask that team to start replicating their success, that’s like building a Tesla factory, and it requires a significant increase in resources and experience.

Making Development More Predictable
Many software developers handle all these complexities by putting them on their clients’ shoulders, providing the team of engineers and requiring the client to manage the team and the project. While this works sometimes, it often leads to subpar experiences and can’t rightly be called full-service software engineering. At Syberry, we work closely with our clients, but we are the ones managing our projects and our teams. We believe our clients hire us so they don’t have to do that work.

We know there is no single solution to the challenges of software development — or any challenges, for that matter — but we are constantly fine-tuning our processes and strategies to make software development more predictable and give our clients and our projects the highest chances of success.

Here are four of the drivers behind that success:

Organizational Culture
Culture and processes are the combination that determines an organization’s ability to deliver quality products and services. While the processes are important, they can’t account for everything. You can write books worth of rules and processes and regulations, but there will always be new scenarios, finer details, and unexpected questions that formal processes don’t account for.

This is where culture comes in, and Syberry has worked to cultivate a culture where employees are empowered to make the best decision for the client and the project anytime there is no handy rule to reference. Our employees highly skilled developers and critical thinkers, and we trust them to make informed viable decisions — or seek help when a decision is out of their realm of expertise.

Promises
It’s not so much about making promises as it is about making promises and commitments you can fulfill. We’ve all met “yes men,” who agree to anything, promise the world, and then fail to deliver. That’s not beneficial for anybody, and it’s certainly not the path to success for a software project. When we work with clients, we are careful to make realistic, thoughtful commitments based on our software expertise and our clients’ expressed needs.

To ensure we’re making promises responsibly, we implemented what we call “service models” in our organization. Similar to project management models or methodologies, service models define what we promise to our clients in different kinds of engagements. We use a variety of different models. These are just a few of the most frequent:

Custom software development: When a client comes to us with detailed requirements for a project, and the vision and core functionalities aren’t likely to change, we can promise to build a system with a specific list of features and within a specific budget and timeline.
Product development: When a project is nothing more than a vision, when the features are still flux, and when competitors are breathing down a client’s neck, we can’t make such specific promises. When time is of the essence and the goal is a moving target, we promise a high-performing team to maximize the number of features built within a given timeline. To clarify our promises, despite the flexibility, we work with the client to define clear milestones to track the project’s success.
Maintenance and support: A system that is in production requires a completely different set of promises, including SLOs around response time for troubleshooting and fixing issues or levels of system stability.
There are many types of services, but the key here is that, when we sell something, we have to be sure that both our team and our client understands exactly what we’re selling. Structuring our services this way, rather than just trading hours for dollars, protects our time and our clients’, and it makes our services clear, successful, and scalable.

Client Education
When you go to a doctor (unless you’re a doctor, yourself), you expect her to know more than you do about medicine, and you also expect her to help you understand and improve your overall health. The same is true in software development. Our clients are experts in their businesses, and we’re the experts in software development.

Our clients don’t have to be skilled software engineers or project managers — that’s our job. But it’s up to us to ensure they have a solid understanding of the process, milestones, and end goals, maximizing their confidence and minimizing surprises along the way.

Team Equilibrium
Along with our emphasis on culture, we believe that the most effective software engineering teams are driven by KPIs and evolving ideal processes, not by management. The most effective teams’ function with proper goals and framework, with management playing the role of gardener — finding the right people and giving them the tools they need to flourish on their own.

Because development isn’t predictable, teams have to be flexible and prepared for change, always aiming for equilibrium, not stasis. In other words, they need to be prepared to pivot in order to work in whatever way is best for a given project at a given time. This flexibility and constant change mean our team is always looking for better ways to achieve project goals within the constantly changing business environment.

Software development will always include a level of chaos, but that doesn’t mean it can’t be successful. When you partner with a vendor whose expertise, agile processes, and service-oriented culture minimize the inherent unpredictability, your organization’s software development project is far more likely to be a rousing success.

The article first appears at https://www.syberry.com/company/blog/articles/making-software-development-predictable