Overthinking Software Projects

"Weeks of coding can save you hours of planning."

Most developers learn this truth the hard way. I did when I implemented a feature based on incorrect assumptions, and the only way to save it was to rewrite it.

While junior engineers often fall into the trap of jumping straight to coding without thinking about the problem, more senior engineers fall into a different trap: overthinking.

Senior developers are routinely responsible for projects requiring a handful of engineers and a few months to complete. There is no way to execute these projects without careful thinking and planning. Even if you wanted to start coding on the first day, you couldn't. You simply wouldn't know where to start.

The top priority in the initial phases of bigger software projects is to sort out the ambiguity that blocks development. The most common way to do this is to devise a design that will guide the implementation.

The tricky part is that figuring out the design is by itself an ambiguous problem. There is usually more than one way to implement the solution, and many constraints and requirements are unclear. Even estimating how long it will take to prepare the design can be difficult.

All these uncertainties put pressure on the engineer(s) responsible for the project. Making a bad decision can lead to wasted time (amplified by the size of the team) or even a project failure.

When stakes are high, it is natural to proceed carefully, explore potential problems, and work with others to identify gaps that may otherwise go unnoticed. However, setting limits for these activities is crucial. Failing to do so will inevitably lead to overthinking, which can also put the project at risk due to:

• Delays - analyzing every imaginable scenario takes a long time and will eat into development time, making meeting expected timelines impossible.
• Overengineering - trying to address minor or hypothetical issues leads to overly complex designs that are hard to implement and expensive to maintain.
• Missed opportunity - endless discussions, revisions, and feedback rounds steal time engineers could spend on other project activities or elsewhere.

How do you know if you are overthinking?

One of the problems with overthinking is that the line between productive analysis and overthinking is thin and easy to miss. However, there are signs of getting there:

• Although all critical requirements have been satisfied, new requirements are being added.
• The same topics continue to be discussed repeatedly without reaching any resolution.
• Edge cases, minor issues, and esoteric scenarios start to dominate the discussion.
• The debate moves to future scenarios that are out of the scope of the project at hand.

What to deal with overthinking?

It is important to understand that the goal of the planning and design phases is not to identify and solve all possible problems. First, it is impossible even to list all the issues. No matter how much time you spend thinking you will miss something. Second, many problems will never materialize, or if they do, their impact will be minimal. To focus on what's important, create a list of requirements, decide which are critical (the list should be short), and satisfy those. Leave out the remaining ones and revisit them if they become a major problem.

Many problems have no one correct or even best solution. No amount of debate is going to change that. It may take time to arrive at this conclusion, but once it is settled, you must pick one of the alternatives and live with the consequences.

If you are not sure, prototype. An hour of coding can save you a few hours of meetings. Code does win arguments.

Design for your current needs. If you don't operate at Facebook's scale, don't design for Facebook's scale. Don't build abstractions for your hypothetical future features.

Accept the fact that things may not work out. Fortunately, you are not pouring concrete. This is software, and the ability to modify it is one of its greatest advantages. You can build small and evolve your solution when your needs grow.

Build vertically. A small feature working end-to-end will allow you to discover issues early and course-correct. If you build horizontally, you won't see gaps until very late when all the layers are ready. Fixing bigger problems at this stage will be an undertaking.

Picture: The “Am I Overthinking This?” book cover.

💙 If you liked this article...

I publish a weekly newsletter for software engineers who want to grow their careers. I share mistakes I’ve made and lessons I’ve learned over the past 20 years as a software engineer.

Sign up here to get articles like this delivered to your inbox:
https://www.growingdev.net/