Everyone seems to think ever more unit tests and Continues Integration are good things. This is often a fallacy. It is part of the often-mistaken Silicon Valley culture that gave us Scrum and believes you can somehow iterate yourself to quality by doing ever more incomplete work in ever shorter fixed time sprints, thereby adding incomplete work using ever smaller piles of garbage, rather than in actually solving problems.
CI is the backbone of this culture. In the CI driven development cycle, this means you not only get to figure out and debug build systems issues in your local development environment, but in an entirely remote system that may have very different behaviors, and to which you have very limited access. Debugging a CI issue often feels like the blind man trying to understand the elephant. And all you can do is create ever more commits to try and hope to fix your CI build when it does break.
For this reason, I prefer to do all my production / release work for my things locally on a development workstation. It's far easier to resolve broken builds, especially if the number of people who actually do releases is small to begin with. It also means you have better idea how to setup local dev environments for everyone else correctly. Anyone else with a correct / complete setup can then do production releases, too. We have all these local resources that often are far faster as well as far more accessible, whether for running lint, running pre-release tests, etc.
Where I do find CI handy is for accepting code from other arbitrary external people submitting public merge requests, as a kind of minimal pre-verification, since they probably don't know how I setup production or what my expectations would be in advance. Strangely, for a long time, this one obvious use case (integrating CI with merge requests) was actually rather poorly supported in the CI cycle and even actively discouraged in the cult of CI.
Instead, we often have this practice of verifying every single commit. This can be rather useless and a big waste of resources. If people have problems locally testing code before submitting, make sure its forced in their commit system locally, rather than exporting their bugs for CI to work out later.
Do you run deploy on tags from your CI? The horrors of that elephant again, because tagged production workflows are only used during the few releases, and so are rarely used. They usually do get broken with product changes. Then you find yourself hiring devops engineers, and other workers you didn't need because the workflow creates labor needs you wouldn't have had, and all those CI resources also have to be maintained. Yet, it is these useless workflows CI was originally optimized for. This workflow does at least guarantee full employment for all skill levels, though...
Unit tests are often also part of the CI driven scrum culture and has it's own issues. For old fashioned 20th century applications with well-defined linked API's, they can be very useful to validate and prevent API regressions, and this even remains true for a lot of embedded work today, too. For many real-world things made today, however, for example, that interface over networks, or involve interacting components that have no purely isolated operation, that have human interfacing, etc, they are often rather useless. Proper integration and release testing is what you most often want instead.
Many that buy into the SV CI cult also go for code coverage. The code coverage subcult tends to reject code if the coverage falls below a certain project manager defined minimum. The problem here is that while it encourages / requires unit tests to be written, they often are made to satisfy management needs. This lends to poor quality and testing practices. This is especially true for systems, as noted, that cannot effectively be unit tested in isolation to begin with.
There certainly are cases where CI is useful, where unit tests and test driven programming do work, and these are good things, but they are far more cases where they are blindly followed practices where they can at best be useless, and at worst cause real harm. The next thing I do want to talk about is how AI can help bring back the single person software shop of the long lost 70's (and 80's?), but before we can even talk about that we need to regain control over those practices which make that impossible.
Top comments (0)