Every programmer has used packages in some way or another during his/hers professional career. Think of famous frameworks and languages like Vue, React, NodeJS, Python. Even though they are so ingrained in the developer infrastructure, most people haven't though too deeply on how they can benefit from introducing them in their own codebases.
This article dives a bit deeper in why you would do this, what you should keep an eye on and how they can help your cases.
Why packages
In my previous article (which you can read here), I explained why packages can be a good way of decoupling your code from each other. This makes your logic more easily swappable with new behaviours without needing to update all consumers in one go.
Now, of course this is not the only upside of packages. More abstractly, they are a great way of defining code to be used over multiple repositories. Imagine a large repository of models that is used across your entire organisation. Implementing these models in all consumers is practically very tiring. Keeping them all in sync is going to be a living nightmare.
Packages easily fill the role of mediator here. They contain certain pieces of logic & are static based on their version number. As such, if at one point a new entity would be released or a change would happen to a pre-existing entity, it's as easy as releasing an incremented version of the package & upgrade your consumers one-by-one. You could even upgrade them ad-hoc, when they would start using this new entity.
In the wild, we see this behaviour a lot. Think of any library you have been using, I'll use Vue as an example, as I have some history with it. Vue 3 was implemented on at the end of 2019. Only in a few months of writing, Vue 2 will be fully deprecated. Even though Vue 3 was out in the wild, consumers still had the option of using Vue 2 until maintenance of the second version didn't make sense anymore. I'm sure, if you think a moment on it, you will be able to collect a few cases in your own domain. Most frameworks and even programming languages, use the same versioning behaviour.
Lastly, a major upside of using packages, is that the consumer is still in full control. You want to overwrite a certain behaviour, that's completely possible by extracting the function & overwriting it. Want to debug a certain issue, well, as you are running this code locally, you can put breakpoints wherever you want. Relate this to microservices, which are inherently made blackbox, it is much easier from a developer perspective to work with packages than it is to understand someone else's microservice.
All of this is very well, but using packages the right way isn't always given. Especially on the operational side, it can be hassle.
Operating a package infrastructure
Introducing packages in your organisation isn't as easy as you would think. Sure, packaging it & hosting it is definitely doable with a few lines of CI/CD definitions and a private package repository. Sadly, that's not where the lifecycle of a package stops. Keeping it up to date & notifying people of updates can quickly become cumbersome. Your organisation (or at least your team) needs to think about how to introduce it easily in the organisational processes. In the following paragraphs, we'll discuss a few things to keep in mind, keeping your processes in sync after the introduction of packages.
We'll dive a bit deeper into how you can easily deploy & keep your packages up to date with your code repository. I will explore a few ways of dealing with ownership over packages. And link that to strategies lowering work for either consumers or producers of packages, or somewhere in between.
Maintaining versions
Keeping things up to date is hard. Most things that need constant maintenance are dreaded by even the most organised people. Think of documentation, the value is obviously clear, but still most programmers prefer doing other things over creating documentation.
The same holds for packages. If you spread your logic over more consumers, all of those consumers will need to have their versions upgraded and will need to follow your patches, bug fixes and deprecations. Especially the last 2 points are the hardest to get right, as it might require a lot of well-executed communication to keep all consumers in sync.
There are a few strategies to deal with this, which we will explore in the next few paragraphs. Obviously, no single solution will be the golden nugget for your organisation. If you decide on a strategy, make sure you have cross-team buy-in as well, as misusing the strategy might spread like a contagious decease across you engineering department.
Broadcasting Strategy
The first strategy is to do as most open-source projects do, communicate about upgrades and deprecations in advance. Each upgrade that comes out can be checked on a certain branch and changes can be included in this unpackaged version. This helps communication, as consumers can follow along with the changes for the new version.
Secondly, as the version gets released from this branch to a new version on master, the changelogs are communicated in a uniform format: changes, fixes, deprecations and potentially a migration guide for major releases. This helps the consumers to understand if the changes impact them, either in a positive way (a bug got fixed) or a negative way (a function got deprecated).
Finally, for major increments with complex migrations and multiple deprecations, both versions stay maintained during a certain time window, which gives consumers more time to do the migrations before the prior version becomes unmaintained.
One can already spot a few inefficiencies in this model, depending on the use case. Maintaining 2 versions of the same framework, just to keep old consumers running might be a drastic waste of time for your developers. Both doing the communicating and maintaining both version takes focus away from the team.
Secondly, communication is hard, especially if it is needed cross-teams or even cross-department. The risks stays present that a certain team didn't receive or ignored your deprecation warnings.
In general, the open communication style lends itself well for fully decoupled projects for key projects. The massive energy drain of double maintenance and broadcasting communication, might need organisation changes that might not be worth the effort. If on the other hand, your company is already fairly well structured with regards to cross-team communication, you might find the low touchpoint of the broadcasting strategy interesting for your use case.
Subscription strategy
The next strategy can be seen as a watered down version of the broadcast strategy. If your package will only be used by a select number of consumers, maybe you don't need to broadcast your changes to all consumers. If you can have people subscribe to your package, listing the functions they are delegating to your code, you can compile a list of teams to functionalities. If your new version then changes this functionality, you can let the people know and define maintenance timelines and deprecation windows together with them.
Reading between the lines, this means that consuming teams must have the diligence to subscribe and, a bit harder to ask, keep the functionalities they consume up to date. If this is not the case, you might forget to involve a consumer regarding the windows outlines above.
Summarising, this is a great low-touchpoint way to allow reusability across teams, while not spamming those people with updates on the state of the package. If you are not up for endless communication, maybe this strategy is for you. This puts a lot of responsibility in the consumers' shoes, maybe you don't want that. Especially in a single team context, this wide communication might not fully make sense.
Push strategy
A final strategy to explore is the completely opposite of the broadcast strategy. Only major version upgrades, in most cases with deprecations, will be communicated, if necessary. Only those cases can actually form issues with consumers, so why bother with communicating every small detail?
There is another way to go about this, which works quite well in small teams or simple bounded contexts. Why not, as part of your release process, plug in the new version of your package across all consumers? Especially if you added for example an improvement to a certain functionality, why update the version numbers one-by-one in each consumer. Why not put that power into the hands of the person merging with master?
This can easily been done with a CI/CD pipeline. Imagine your second to last step is the packaging and deploy of a certain version. Then, you can make the last step of your pipeline checkout the consuming repos, make the change to upgrade the version and finally, push the changes to the original repo. After this step has run, the normal CI/CD pipeline of the consuming repository will be triggered due to a code change and will eventually be deployed with the new version.
This strategy assumes a few things, making it not viable for too large contexts. First, it assumes you know exactly what the producers use and the impact a rollout of a new version has. Secondly, you should have access to the consuming repositories, as you need to upgrade and potentially roll back a version if necessary. Finally, you still need a lightweight form of either of the above strategies within your bounded context.
If you do know your context is small, for example, a single team, it should be fairly straightforward to recycle common definitions used in multiple of your owned repos by putting them in a shared package. You could then, after a definition is changed, update all of your downstream repos. In general, this works quite well, as the entire team knows what the impact is of changing a certain piece of functionality. If not, an MR review should catch such issues.
To summarise, if you are working in a small context, where information of impact is already quite shared, it might be an option to put the power over the consumers in the hands of the producers of the package. Deprecations still follow their own path, but in most shared packages, this happens only sporadically and in a well organised way.
Conclusion on strategies
In the paragraphs above, we saw 3 possible strategies you can employ in your own team or company to allow for easier sharing of code via the means of packages. Each strategy exists on a spectrum, from a lot of power to consumers, to more power to producers. Each came with its own benefits and downsides, shifting your ideal solution possibly somewhere on the spectrum.
All of the above strategies require some form of buy-in from your team members. The first strategy a lot more so than the last. This means it requires some change management to look into this, but starting small, with the most simple practicalities, helps building a more complex strategy in the future.
Top comments (0)