Software might be eating the world, but DevOps complexity is grinding it to a halt
Releasing software was supposed to be easier by now. We’ve got all these tools, we’ve got resources like Stack Overflow, and we’ve got the cloud. But somehow, most engineering leaders we talk to feel it’s only gotten more complicated
Why has shipping software gotten more difficult and error prone? It’s not writing new features that’s causing the bottleneck - but rather the complexity of deploying code to the cloud.
Yes, you might be able to release an MVP quickly, but creating repeatable and scalable processes is an entire different story. Talking with CTOs and heads of engineering, we keep running into the same issues:
Skilled DevOps engineers are hard to come by: A majority of IT leaders have reported that hiring skilled DevOps people is a challenge, according to a 2021 survey (PDF). For smaller companies that can’t afford to keep up with the salary arms race, this is an even bigger problem.
Dealing with tooling sprawl: Everything you can do on the cloud nowadays comes in dozens of different flavours - from your choice of cloud provider to your container orchestration platform. Tooling choices are endless and confusing; evaluating software becomes a costly distraction.
Configuring the cloud stack correctly: Every platform under the sun promises a complete self-service experience, but the reality is often less straightforward. Something as simple as setting up an S3 bucket to store some documents comes with a host of configuration choices, and the same applies to every microservice, database, and component you use. Many of the people we spoke to had nagging suspicions that things were set up incorrectly and that this would come back to haunt them (they were usually right).
DevOps issues such as managing dependencies, Kubernetes configuration, and CI/CD pipelines end up eating 10-30% of development resources - slowing down the company’s core revenue-generating activities (as was covered in this report from 2021).
For a small company that just needs to ship an app today, getting started can be extremely difficult.
The Pareto Principle and 3 Ways to Overcome DevOps Complexity
The Pareto principle tells us that roughly 80% of consequences come from 20% of actions. In software development, the first 20% of correct design choices will give you most of the benefits when it comes to performance, security, and stability; the rest you can optimise later.
While no two projects are identical, we’ve found that many early stage companies or smaller software teams are implementing the same patterns:
Standard open-source toolchain: For most companies and most projects, there is a set of open source tools that strikes the right balance between control and usability. GitHub, Kubernetes, ArgoCD, Prometheus and Grafana give you what you need to write well-structured code and push it to the cloud.
Good-enough configuration rather than perfectly-tuned everything: Options are endless and fiddling with them might eventually produce a cleaner pipeline or reduce milliseconds of latency, but that’s rarely what makes or breaks a startup. Instead, start with an MVP-type setup that ticks just-enough boxes to enable short cycle times without jeopardising security, stability or resilience.
Best practices for development workflows: There are right and wrong ways to handle namespace management, resource provisioning and dependencies. We would review the way things that were done in the past, fix them when necessary, and ensure that best practices are adhered to moving forward.
After dozens of projects with different companies, we found that these basic patterns are repeated time and again - right down to the choice of naming conventions. Which leads us to ask: Why repeat a process manually, when you can automate it?
Diego - Productizing the 20/80 Stack
Diego is our answer to the DevOps challenge that’s paralysing smaller software teams. Diego is a platform that provides an end-to-end development workflow using a set of open-source tools, mainly Kubernetes and ArgoCD. With Diego in place, developers can build, deploy, and monitor containerised applications in the cloud - on a solid foundation of enterprise-grade DevOps infrastructure.
With Diego, our software would handle cloud and Kubernetes setup, security, infrastructure management, application management, deployment, and observability. You can find the details here.
Our vision is to give smaller teams the ability to ship high-quality software quickly, even if they don’t have large DevOps teams to optimise their cloud deployments. No more trial and error or whim-based configuration decisions that end up creating a mess of repos and dependencies. Instead, Diego provisions the resources and environments needed for each feature, app, or microservice - and lets developers work their magic from there.
We’d love your feedback.
Diego is a new product. We’ve just launched with a small group of design partners and would love to hear your feedback - are we in the right direction? What else would you like to see? How can we further simplify the process of building, deploying, and monitoring apps for organisations like yours? Tell us in the comments.
And if you’re ready to give Diego a spin, get in touch here and we’ll set you up with a free trial.
Top comments (0)