DEV Community

Cover image for How do you identify "over-engineering"?

How do you identify "over-engineering"?

Ben Halpern on July 14, 2020

How do you spot over-engineering as it is happening, how do you communicate around this issue?

Collapse
 
gwutama profile image
Galuh Utama • Edited

I've seen many software engineers implement things that aren't in scope of their projects just because:

  • "hey, what happens if the customer needs this feature in the future? We must adjust the architecture so that it can accommodate that feature."
  • "hey, micro services are all the rage right now, we've got to have it for our next project"
  • "hey, this brand new shiny library looks cool. We must use it for this project."
  • "hey, the customer pays us a lot of money, let's use it to learn new languages and use for that project"
  • and so on ...

These are all warning flags for me, as an engineering manager, that the engineers will potentially over-engineer things that bring zero value for the customer.

When we follow the lessons of V-Model:

  • There's requirement and use case analysis,
  • based on that you build a system design,
  • based on that you build a functional system design,
  • based on that you build technical system design,
  • and based on that you write the code.

What I'm saying is, each step is based on the previous one and each step must be verified against the previous one. Anything else is over-engineering unless there are other business cases behind it.

You shouldn't identify over-engineering as it is happening because it might be too late to change direction. You should prevent over-engineering from happening.

Collapse
 
yerac profile image
Rich • Edited

hey, what happens if the customer needs this feature in the future?

Although there is a big advantage to stepping back during the design process and thinking of extensibility or abstractions which will allow features to be expanded easily, or suit alternate approaches without mass refactoring. Sometimes, these thoughts come at a later implementation phase, and we have had big wins by doing iterative stuff at a later point to "enhance" the offering rather than go back via the planning and design phases.

There is a big difference between "gold-plating" and forward-thinking, and the line between the two is extremely fine and easy to end up on the wrong side of!

Collapse
 
gsto profile image
Glenn Stovall

I agree, and I think the difference is in how you approach it. You want to leave yourself open to extension, but not over-extend.

Collapse
 
xanderyzwich profile image
Corey McCarty

Start from the Minimum-Viable-Product and then iterate from there. If there are pet pieces that the developers would like to make along the way then you let your customer handle prioritizing that work.

There is another side of this issue that you mentioned. If people are trying to push projects into a new language then they obviously have interest in learning something new and are not given an outlet for that energy. Letting them get some down time to work on a small pet project to learn a new language and report back as to how it performs would be great. Help them to understand that there is a time for experimenting and the Customer's product should not be that.

Collapse
 
tsalman80 profile image
tsalman80 • Edited

how do you address over engineering when the person who is doing is the architect/fo-founder.

lets build a user management system with log-queue pattern, you know so each web app can manage their own users (which they dont, all collections need to be in sync otherwise the app breaks). have 3 micro service, one which for crud on the source and the other two are responsible for replication.

WTH.

Collapse
 
puritanic profile image
Darkø Tasevski

Amen.

I know a few people who are overengineering stuff all the time, just for the sake of what if... Meanwhile, there are a bunch of serious problems with the apps they're working on. I guess that most of them hope that the problems would just go away on their own.

Collapse
 
ben profile image
Ben Halpern

Great comment

Collapse
 
javascriptinginfo profile image
javascripting-info

GREAT post!

Collapse
 
spiritupbro profile image
spiritupbro

keren sekali penjelasannya mas galuh saya terpana

Collapse
 
jsn1nj4 profile image
Elliot Derhay

Instructions unclear. Duct tape soaked in WD-40. 😵🤪🙃

Collapse
 
dwd profile image
Dave Cridland

I once saw someone trying to remove marks left on a glass table by duct tape. She thought I was joking when I told her what would remove it. WD-40 - works every time. No idea how to remove that though.

Collapse
 
meatboy profile image
Meat Boy • Edited

Using external libs for trivial problems. Reinventing the wheel isn't good but very often the real problem is easier to solve than people think.

Also, forcing to use design patterns everywhere without thinking if they match requirements makes code crappy after time. They are for special purposes but some people are trying to put them everywhere.

Collapse
 
themobiledev profile image
Chris McKay

I've been guilty of the latter. It used to be that when I found a new pattern I would put it everywhere. I justified it by telling myself that I was learning about it, but all I was doing was stalling a release. Quick and dirty, while not always the best solution, isn't always the worst, either.

Collapse
 
ryanlanciaux profile image
Ryan Lanciaux

I think over engineering can often be synonymous with over abstracting. As developers, we often build abstractions to future proof our software. Unfortunately, we can often plan for a future that doesn't exist. Since we get more information about the future as time goes on, it can be a better practice to defer building abstractions until we have more information. This helps us avoid unnecessary complexity, and make decisions based on data, rather than guessing :D

Sandi Metz has an awesome article about this topic. sandimetz.com/blog/2016/1/20/the-w... This article has some great words of wisdom in it "duplication is far cheaper than the wrong abstraction" and many others.

Collapse
 
jsn1nj4 profile image
Elliot Derhay

I think Jeffrey Way would agree about over abstraction. When I was learning Laravel, a lot of times he would say to not worry about abstraction until it became clear that a part of the code was going to start getting messy. He'd also show a good number of ways to accomplish the same thing, depending on what's needed.

Collapse
 
jenc profile image
Jen Chan

I'm guilty of this a lot. And it requires my teammates to tell me that so far. I feel like maybe it's a step-back-and-look thing. Or maybe stopping before diving straight into a problem and trying to see if there's already something in an existing lib or codebase to do it simply.

Collapse
 
daedtech profile image
Erik Dietrich • Edited

Personally, I find the term "over-engineering" to be about as helpful as describing code as "bad." Baked right into the word is relativism centered on the speaker's axiomatic perception of themselves as the correct arbiters of "Goldilocks engineering." In other words, I would define "over-engineering" as "a descriptor indicating that the speaker doesn't fully understand the context of someone else's design decision, but knows they don't like what they do understand."

To give this a concretion, I spent a bunch of years doing a specialized form of management consulting that involved using static analysis to help enterprise IT leaders make decisions about codebases. In other words, I earned a living analyzing codebases and the decisions of the engineers who brought the codebase to its current (usually bad, since people never called me to tell me how great things were going) state.

This led me to find myself looking at a lot of codebases that had what most would probably agree was "over-engineered solutions." Here are some particularly fun ones:

  • A DTO code generation scheme that... also generated unit tests for those DTOs.
  • An application consisting of multiple "layers" of degenrative pass-through calls.
  • An interesting character that created a new flavor of MVC that he called "MVCD," where the "D" was his first initial.
  • A Winforms app that achieved "multi-tenancy" by generating every token in a SQL query with a complex stringbuilder scheme, tuning them with every token based on the logged in user.

The list goes on.

But here's the thing. "Over-engineering" is a judgement-assigning, dimestore root cause analysis that doesn't matter. Those designs had actual, tangible problems that had nothing to do with what the designer was (over) thinking:

  • Unit testing property bags has little, if any, regression detection capability for the cost.
  • Do-nothing application layers create maintenance overhead without any encapsulation upside.
  • New people onboarding to the team will be familiar with MVC, but not with "MVCD," so you're burning money on staff learning curve for no apparent reason.
  • That Winforms app took something like an average of 30 seconds to do anything that hit the databse.

So, I guess for me the tl;dr is "when cataloging issues with a codebase/design, and especially when communicating them, I try to stay away from personal taste and deal instead in cause/effect reasoning and tradeoffs being incurred by the design choice."

Collapse
 
pavelloz profile image
Paweł Kowalski • Edited

I guess you heard that you overengineered stuff enough to dig yourself in a defense of it. That will be a tough one to defend, especially considering how often startups fail because they are polishing their kubernetes config and polishing CI instead of getting paying clients.

Example:
I can perfectly understand react app, but when i see business card-type of page done in react+redux, its overengineered. Period.
Even if I didnt understand it, i can still judge when someone is clearly using wrong tools for the job.

Collapse
 
daedtech profile image
Erik Dietrich • Edited

Period spelled out? Wow, you must be right.

I imagine that always settles the matter at code review for you, especially when you yell it. Glad you showed up to correct me!

(BTW, what you're describing -- speculatively adding unneeded functionality -- is called gold plating. People commonly use "over-engineering" to describe something else)

Thread Thread
 
pavelloz profile image
Paweł Kowalski

Cool.

Collapse
 
alainvanhout profile image
Alain Van Hout

Overengeneering is when you add architectural complexity for a perceived (rather than actual) future need. The trick to avoid this, though it's by no means self-evident, is to leave open a path for future change, without already paving that path right now.

Collapse
 
adam_cyclones profile image
Adam Crockett 🌀 • Edited

Anything for the future is not for the now so should be looked at with a sceptical eye.

I just spent 2 evenings on a personal project trying out different sub module and other strategies but in the end it all felt insane so I just stopped. No with that distraction out of the way I managed to refine my release scripts so that they at-least work.

Collapse
 
somedood profile image
Basti Ortiz

In my side projects, I know I've over-engineered when not even Git could save me from my spaghetti code... 😔

Collapse
 
ferricoxide profile image
Thomas H Jones II

Simply put, if a given "enhancement" requires more than a fractional- or even trivial-effort to implement but puts delivery timelines at risk for something that's likely not even in scope for the next major phase of the project, somebody's probably lost the ball.

Collapse
 
dfockler profile image
Dan Fockler

Over-engineering is often solving problems that are either future problems that don't actually exist, or solving problems that were caused because of previous code.

An example is a developing a custom templating language instead of using an ERB file. This is an example of a problem that doesn't exist, creating super customized, ultra-flexible systems that most likely will never be used to their full potential. Often these systems use layers and layers of abstractions in order to get 'clean' code.

Also be careful because the term "over-engineering" is used as a cudgel by people who want to talk smack about code they might not really understand or that feels complex to them, when in reality it might solve a problem really well. It takes a while to fully understand a complex problem, and a complex solution that solves it.

Collapse
 
xanderyzwich profile image
Corey McCarty

I actually had a good discussion last night about this in terms of 'perfection v time to market' (which I know is only part of the issue), and I had a realization that the legendary Microsoft 90% rule is a great balance for this debate. It provides a valuable product while allowing the last bit of development to be refocused by actual user feedback.

Collapse
 
n_develop profile image
Lars Richter

Hi Ben. Another cool discussion about a real-world topic. I planned to write a post about over-engineering for a few weeks now.
This discussion reminded me to get back to my draft and finish it.

I published it a few minutes ago. You can find it here:

But to answer your question, I think over-engineering is all about adding additional complexity to code that could be much simpler.
I think most (experienced) developers (including myself) have been guilty of over-engineering. Remember the day when you learned a framework/design pattern/tool that you thought is really cool. That is the same day when you excessively start to look for places where you can use it.
It happened. And it will happen again. We will just have to find a way to deal with it in a solution-oriented way.

Collapse
 
dwd profile image
Dave Cridland

I think this varies hugely on what level of the stack you're operating at.

At the front-end, redesign is relatively cheap, so the ideal way of building things is to be very "lean", and build the minimum possible. If you need to build something more, the old can usually be thrown away without penalty if that's the quickest path.

As we descend the stack, the cost of change increases - so we need more care in the work we do, and there's more "passive provisioning", where we deliberately build gaps for future functionality - and sometimes actually build the future functionality since it's easier than just leaving a gap.

Often, old architectures and API points have to live in the system, and be maintained, for significant amounts of time afterward - for support of mobile apps etc.

Infrastructure is even worse - changing infrastructure architecture can be time-consuming and risky. It's best to get that right first time.

In the worst case, standards development, we're trying to build the design itself, and we're stuck with that for years, if not longer.

We often try to minimize the cost of failure in our professional - "Lean", "Agile", and "MVP" all derive from this. We also, of course, have to minimize the cost of success.

Over-engineering isn't simply doing more engineering than is needed; it's doing more effort than the cost of success warrants.

Collapse
 
winstonpuckett profile image
Winston Puckett • Edited

I'm of the opinion that code quality is the thing that enables us to move quickly in a code base... I get concerned about over engineering when someone anticipates a need for future changes and then sacrifices clarity for something that has a low possibility of happening. I have no idea how to communicate about this though. I'm a junior engineer working with exclusively senior engineers.

Collapse
 
salimchemes profile image
Salim Chemes

when every little change is so expensive and time consuming

Collapse
 
dansilcox profile image
Dan Silcox

This can be a symptom of ‘under engineering’ too - ie just piling features on features on features and never paying down tech debt

Collapse
 
salimchemes profile image
Salim Chemes

yes, this is why I think finding the correct solution is a very valuable skill. Not under, not over engineering.

Collapse
 
dividedbynil profile image
Kane Ong

If code review is not a must and not being value by stakeholders, under engineering is the only concern. Over-engineering can start with choosing a framework or importing 3rd party libraries, but deadlines are always the top priority.

Collapse
 
kspeakman profile image
Kasey Speakman

Usually, an over-engineered solution starts to "feel" awkward and ill-fitting. It takes time to develop a sensitivity to that feeling. Usually the best way to prevent over-engineering is to keep very focused on solving the business problem at hand. And realize within yourself those conditions where you find an interesting technical solution and are trying to force it to fit the problem.

Collapse
 
iimrankhan profile image
Imran Khan

When a co-worker keyboard or mouse comes flying past the head because it took them 8 hours to add a simple database read call.. Just kidding.

i will first ask what is over engineering? What do we mean by it? For a lot of people 'over engineering' means doing thing the way it should be.

Any how, i look at how may calls it makes to do something? How many objects it passes through to achieve something? How many 'layers' it does through to complete a task?

Collapse
 
aghost7 profile image
Jonathan Boudreau

Ask them to explain what problem they're trying to solve. If they say "we might need it later" or "just in case", then they aren't adhering to YAGNI. In general I've found that sticking to principles at least makes it easier for people to accept. It also sets expectations as to what you won't accept.

I've also found that agreeing to revisit the idea to help. Acknowledging ideas is important.

Collapse
 
nbageek profile image
Patrick Minton

If it doesn't benefit the customer, it is over engineered. Not future customers, mind you. Today's customers. I find that if you approach everything with a customer-focus, you'll probably be fine.

Note also, that some of you are thinking "But I don't work on customer-facing code," to which I would say you are wrong. I promise you that you have customers. Figure out who they are, and start focusing on them.

Collapse
 
mxldevs profile image
MxL Devs • Edited

Given the specifications, I focus on a minimum viable product that meets the requirements.

Features and use cases naturally will come up, but if we start talking about things that aren't asked for, the question then becomes whether this new thing will help make it more flexible in case the specs change or new specs emerge that would result in us having to rip apart the code and rebuild certain aspects, or if it's purely because it would be cool.

Collapse
 
tominekan profile image
Tomi Adenekan

Your project is over engineered when:

  • You think "Oh that's a good feature idea" and you add it to your app after planning.
  • You have tons of barely working features
  • You can barely look at your code
  • You start getting thoughts that your project is over engineered
Collapse
 
alexandrudanpop profile image
Alexandru-Dan Pop

have written my opinions about it here:

Go for Serverless or JAMStack approaches

Those technologies allow developers to leverage the cloud as much as possible - hence focus on building applications, not complex backends.

Of course those are not always possible as sometimes the API's are your selling point.

Leverage external tools instead of building your own

Let's be honest. As developers we like to re-invent the wheel. There are hundreds of programming languages. Dozens of libraries that do the same thing, for each language.
Break out of this loop, re-use what you can and focus on delivering the needed value.

The art of keeping things simple

Developers also love to over-engineer things.
Product owner needs to show the users birthday on the settings page? Well of course it can't be done without having a few microservices communicate in the backend through a service bus. And all indexing in Elastic needs to be changed before you can get that data. Frontend... you'd better forget about it... we need to spend a month first to configure webpack for the new birthday-settings microfrontend. Also the frontend will depend on the i18n microservice that is being refactored so that will take some time.


Seriously now, don't do this. Build things in a simple and easy to maintain way. Challenge over-complicated designs.

Collapse
 
brpaz profile image
Bruno Paz • Edited

Well, Serverless or Jamstack can be a form of over-engineering too IMO.

Maybe a "boring" Rails or PHP stack with a MySQL database is all you need. ;)

Collapse
 
alexandrudanpop profile image
Alexandru-Dan Pop

True

Collapse
 
sebbdk profile image
Sebastian Vargr

Over-engineering is like a bad joke.

Too much setup, and isn’t there supposed to be a punchline here?

Collapse
 
mdamaceno profile image
Marco Damaceno

When a team mate reviews your code and spots on code style, not the business rule and its flow.

Collapse
 
dansilcox profile image
Dan Silcox

There’s definitely a tension between giving yourself and your colleagues future architectural ‘runway’ and YAGNI - knowing what tech debt is relatively easy to fix later vs what isn’t...

Collapse
 
slackerzz profile image
Lorenzo

Magento 2?

Collapse
 
xowap profile image
Rémy 🤖

Could you write less code to solve the stated problem(s)?

Collapse
 
abhinavsri360 profile image
Abhinav Srivastava

Learning something strange or maybe special or maybe for the first time than trying to implement and speculate that particular thing for several nights before sleeping. 😅