Introduction
Writing software has changed a lot in recent years, and it’s now more complex than ever before. The complexity of the businesses relying on software, ingrains a natural complexity into the domains for which we, as developers, have to write software for.
My career in the field started only a mere 5 years ago, but, I’ve already seen a few different businesses and different ways of working and managing teams to be able to compile this post offering some advice that can help you manage some of the inherent complexity while keeping the accidental complexity at a minimum.
Accidental and inherent complexity
The inherent complexity is the complexity that you have to face because of the nature of the business you are writing software for. If you are writing a guidance system for an airplane or devising a complex algorithm to help you synthesize a new drug, then, the complexity of how a guidance system works, how a plane orients itself, how it manages lift, drag, responds to pilot commands, or a new drug is deemed efficient or not.... All of these facts about your domain, in one form or another, will have to end up making it into the code. This is the inevitable piece of the puzzle, there’s no escaping it.
The art of writing good software is in how we can embed this necessary complexity into the “accidental one”: the language and/or tech stack we are using to encode these real-world facts into code that can deliver the real value for the business. Anyone would agree that writing a guidance system for an airplane in Assembly is a terrible idea: we have better tools. Yes, better tools like...Python. Except that Python is a terrible idea for a high-performance, real-time system like a guidance system. Any programmer reading this can relate.. It’s just not fast enough, not performant or memory efficient enough to be running in constrained environments. C or C++ or Rust are better choices. Much faster execution cycles, the generated native code can be fully optimized to a target architecture chip, full control over memory, etc.
Knowing that we need to always choose the right tool for the job is a key aspect of managing accidental complexity.
The rest of this post will focus on how we can write good code, following best practices and using modern technologies from the perspective of non mission-critical domains. Google’s search engine, a hotel reservation system, tools to analyze code, CAD tools, etc. are all non mission critical domains. Failures in these systems can have consequences and are very serious, but not as serious as a self-driving car brakes failing or an airplane falling off the air…
Taming complexity: a toolset to be efficient in the modern software development age
I will approach this section from the perspective of someone who works with Java, Springboot, Git and uses Docker and Kubernetes to manage and provision infrastructure, but the underlying principles will hopefully be general enough that anyone will be able to get something out of this post.
Teams and processes
Writing software professionally, for a very large amount of developers, is at its core a collaborative activity. This means that you don’t work alone, and you need to collaborate with other developers and report to managers, product owners, follow the Agile methodology (if your company uses it) and most importantly, you need to communicate and agree on how to work and on how to prioritize what to build next.
This is really the crux of the matter: agreement, low barriers at entry, and managing friction and conflicts between teams and individuals is the most important thing. It’s independent of any methodology you or your company decide to adopt but, it is the single most important aspect that you need to get right to enable yourself and your colleagues to work efficiently. All of the other things, be they methodologies, processes, certifications, whatever, only exist to enable and foster transparency and clarity of requirements between team members and other relevant stakeholders. Remember: nobody cares about story points on a ticket. People kind of feel like they need it because it is something that you can measure. But, focus instead on what you CAN’T measure: your grit, your commitment, your will to do good by your colleagues, to keep improving the product you work on, to take pride in a job well done. If you focus on these things, you end up embracing your work in a different light that can effectively let you work under ANY company using ANY type of methodology. Because, at the heart of all of it are people. And, fostering good relationships between teams and peers is the most important thing when it comes to working as a team. The rest will naturally follow and fall in place. Be kind, attentive, work hard, listen, express your points of view and everyone will benefit. Teams are not defined by their governing methodologies, they are defined by people. This is important.
Ride the devOps wave
There is no such thing as writing code in isolation anymore. Continuous Integration and Continuous Delivery are key indicators of successful development teams and businesses in today’s world. Any merge to the master branch needs to be successfully released to production with ZERO manual intervention. Build reproducibility is no longer a luxury or a nice to have. It’s key to the efficiency of a development team.
Docker and Kubernetes were released, respectively, 7 and 6 years ago. A developer who started working professionally in 2012, SAW the way build reproducibility changed. He or she saw how this could shape the way entire teams work and manage their code. It was a revolution.
However, the technology is complex and it requires effort from the developers to learn it and leverage it in the best possible way. I like to think of devOps as “developers assuming responsibility for Operations”. This is critical, and, it’s one of the best investments you can make today that will keep paying itself off for the future to come.
Learn the basics of Docker and Kubernetes, and leverage it to be efficient and help your team be efficient as well. There are a plethora of aspects that you can approach the devOps wave for yourself and for your company:
improve local build reproducibility by putting some of your services into a private container registry for your team or company. Backend services can be dockerized so that a new developer joining the team can run an entire suite of integration tests with a click of a button.
Test data inside SQL databases can be dockerized using a mounted volume that can enable a script to load some static test data into a container. Your front-end team can leverage this to run visual tests, to test changes at specific levels, etc.
Create “templated” docker compose files, where you can add certain services, for example, from different branches, or for example different databases, like managed replicas created by your devOps team, or clusters from Google Cloud Platform, you name it. Docker provides you with the flexibility, safety and isolation your developers need to be as productive as possible locally. Embrace it as much as you can.
Kickstarting Kubernetes usage at your company? Look into branch deployments. Kubernetes can be used as a great tool to enable developers to spin up a complete “branch environment”, that behaves like the real production thing, but contains your surgical changes made in your branch. This is invaluable. “Will this be performant enough?”, “Does changing this endpoint mes up the front-end?” Answering these questions by enabling a full environment to be spun up with a git push is a real enabler. It’s hard to get right and takes effort, but, it’s a great way to be at the forefront of current development practices and very useful for developers. Gitlab offers Kubernetes integration, and Amazon Web Services has kubernetes clusters available, and has backed tools like cdk8s, to enable a declarative approach where we can use Java and the Builder pattern to programmatically generate yaml files that can be deployed on a running k8s cluster. Possibilities are endless.
Docker and Kubernetes tie closely with devOps in the sense that they enable developers to manage and provision resources and environments in a way that was typically reserved to “the devOps guys” a few years ago.
Writing good code
Writing good code is important, because when you’re working under tight deadlines, or when requirements shift, your code needs to be able to cope just as easily. Code needs to be easy to adapt, change or remove, and it needs to be concise and simple. Obviously, writing such code is hard and it’s influenced heavily by the technology your company is using and also by your level of familiarity of the language and the type of architecture you’re using. However, there are some guidelines that can help you shape your code so that it can remain flexible and maintainable even in the face of constantly changing requirements. This will be an incomplete, biased list, but, it should give you important insights:
Keep your services as generic for your domain as possible. If you feel that you have several areas in the code where you need to apply the same logic more than a couple of times, seek for opportunities to extract that logic into a separate, reusable service that can both increase code clarity and reduce duplication.
State and state propagation are the source of numerous bugs. If things happen over which you have no control, isolate these things proactively, and ensure that state does not leak any further than it has to. Data processing, like reading from a database, is always a stateful action, and we can’t always make assumptions that are always valid about our data and its structure: “will this field never be null?”, “Can this list be empty here?”, etc.
What I believe to be a very suitable approach, one that indirectly frameworks like Springboot guide you towards, is to leverage the concept of “Onion Architecture”. Borrowed from functional programming, this architecture, as the name states, is composed of layers, where the outer layers are the stateful ones, and where information flows from the outer to the inner layers. Isolate the data fetching and processing at a certain layer (for e.g. Spring’s repository abstraction over the database you’re using) and ensure that no information can flow from that layer to layers that are directly responsible for the logic of your business domain, like sending responses to a client.Keep your classes and methods short, and use meaningful class names that map directly to the concepts of your particular business, use them and leverage these as first-class citizens in your code. Concept mapping builds understanding.
Try and structure the areas that handle the business logic as data transformation pipelines. Functional Programming abstractions like data enrichment, mapping, transformation, filtering, grouping and sorting, usually make handling the data all the way from the data layer right to the area where it is needed, very smooth work. Convert from database entities at the outer layer into immutable representations for your domain at the inner layers, and handle those with the functional programming tools that are best fit to do the job.
If a method or class is becoming too long, look for ways to refine your logic that allow you to compose something from smaller parts, that are hopefully, nice candidates for reuse, especially because many domain concepts are usually closely related together.
Leverage the tools existing in your ecosystem to their fullest potential: Java streams, the Function interface, Optionals and the object-oriented paradigm can help you shape your code in a certain structured way that can grow organically.
Springboot can be very powerful when we can leverage its Aspect-Oriented programming side, in order to design and build our own custom annotations and validators, literally enhancing the existing set of annotations with additional ones that are suited to our domain.
Jackson can handle serialization and deserialization very efficiently out of the box and it’s a great abstraction that we can leverage to take away most of the plumbing involved in pushing data across the network.
Write your code in this way, and you will see it lends itself well to tests. Write unit tests AND integration tests. Both types are important and they can test different layers of your architecture.
The closer you move to the “outer shell” of the onion, where things can get more stateful (like calling an endpoint or interacting with the database) the harder it is to test, you may need test data, mocks, you usually also need a whole “test application context”. This happens because when dealing with state, the number of moving pieces increases and that complexity gets reflected in the tests as well.As you move to less stateful layers, like certain services that leverage the “data pipeline” style of code, you will see that simple, stand-alone unit tests are usually enough and that’s because usually those services are simple data transformation classes, with very few to no external dependencies, so the set of testing cases is reduced a lot. Even more when the classes and methods are small in both size and responsibilities.
With these guidelines in mind, hopefully you can strive for a design that is easy to maintain, extend and test.
Conclusion
Modern software development fuses together code and operations as never seen before, and that can bring many challenges to teams that have to adapt to new ways of working.
Leveraging simple principles across the board, from your teammates to the responsibilities of your methods, you can thrive in this modern day and age and enjoy doing it!
Strive to always keep learning, get involved in all the aspects of the software development lifecycle, stay up to date on current practices and you will improve your skills and inspire others!
Latest comments (0)