I wanted to explain, why enterprise software development comes with a lot of overhead and why it still makes sense.
We recently needed to set-up a new micro-service that will be the basis for new features in the future. You could argue, that you would only need a git repository and there you are, but that's not quite it. In fact, there is a lot more to it that I am going to explain in this article.
So let's start:
OK, that's the most basic part, we set up an empty git repository that will contain the new service.
Then, we create an empty application stub. For spring boot, we'll use start-dot-spring-dot-io, cheers to Josh Long.
Now we need build jobs on a continuous integration server like Jenkins, not just one of them. We need a continuous (C) job that builds the server after a code change (push trigger), we need a nightly (N) build that will regularly build the code even without code changes and that does some extra work as well as a couple of extra scan jobs, we'll come to that later.
During both C and N builds, we'll perform a static code analysis (e.g. SonarQube). Enabling a quality gate here will fail the build if code coverage is to less or there are too many issues (code smells, bugs, complexity etc).
Based on my experience, both coverage and found issues should not be treated too seriously (like "We need 100% coverage"). However low coverage (< 60%) and a high number of issues is usually is a good indicator for code that is less maintainable and has a high risk of producing bugs in production.
This part is crucial and legally obliged. Otherwise, companies could get sued.
Another legally-induced scan will check if the written source code contains parts or whole files that have been copied from open source projects without referencing to the project and it's license. This BTW also applies to StackOverflow. It's license is CC BY-SA which requires proper attribution.
Violations will also fail the build.
This is the next step in the build pipeline. The used libraries will be checked automatically against the vulnerability database. Any CVEs will show up in the build result. Depending on the severity, it may also fail the build. This part is obviously very important but also very tedious. This scan could break a build at any given time without any code changes just because a new vulnerability has been found. This happens very often every one or two weeks. The scan will usually show what part is actually affected, if there is a new version without the issue or how to work-around the vulnerability.
The final deployment artifact is virus scanned and the report attached to the binary artifact.
For every backend service, we also build and deploy a docker image. This is used for testing as well as deployment.
Software quality is also very important. See my article Everything That's Not Tested Will Break if you're interested in details.
- Unit Tests
- Integration Tests
- System/Use Case Tests
- UI End2End Tests
- Performance Tests
Even for an empty service, we will create a dummy endpoint that will be tested in system and performance tests.
Since we are not the one-product-start-up, we need to support a couple of different deployment targets that range from zip-based on-premise over docker based deployment up to multi-cloud deployment.
That will be mixed with multiple databases that we support. and results in a wide range of Docker and non-Docker-based test environments.
Depending on the features planned for a service, we will also need to apply for a new license module that will allow customers to use the new features.
Each service will get one ore more new permissions that allow users or other services to communicate. Those will be created at service start-up automatically. So even for a stub service, we will set this up.
Phew, all of this needs to be done before writing the first feature!
I want to show people interested in how this might look like in larger companies.
Moreover, I'm curious how you actually handle the above topics. I could imagine, that in a couple of smaller companies this might just be filed under "We don't need that". Others might solve it in different ways. We are always looking to automate individual tasks as well as the whole process but it's not that easy since all steps involve a lot of different system.
So, please comment if you have questions or suggestions!
I am very confident, that when writing software that will be delivered to customers and run on the Internet, most - if not all - of the above is actually required to create software products of high quality without a high legal and security risk.
Photo by langshoots on Unsplash.
As software gets more and more integrated into our lives, the industrialization of its crafting process becomes inevitable. But the over-generalization of software engineering can be crushing the creative side of programming.