It absolutely astonishes me how many software engineering practices I didn't encounter until I'd been working for a while. I mean, when you go to school to study a field, you expect to at least have heard of things that people in that field do, right?
Continuous Integration is one of those topics for me - I encountered it LOOOOOOOOONG after it would've been useful to know. So hopefully by writing about it here and sharing with you, I can save you some of the long wait in your own journey!
Definition
Continuous Integration is defined as the practice of frequently merging all developers' working copies to a shared mainline.
Ok, definitions can be... unapproachable sometimes. Let's do this in Story Mode:
A team of programmers are writing a complex application. It has 4 main features, and 2-3 people are working on each feature semi-independently of each other.
Well - they can't all be working in the same file. They'd constantly get in each other's way and stomp on each other's work and nothing would ever get done! So the feature teams go off by themselves, build the feature they're each responsible for, and then have a Merge Party where the features are combined into one cohesive code base.
Merge Parties are painful, though. Because as they've been working independently for several weeks, some assumptions cropped up that are out of alignment. Team A is passing a UserID to Team B's feature, but Team B was expecting to receive a full UserProfile object. Now the application is broken until someone realigns... and the teams are each going to expect the other one to just do the work. Conflict and contention arise... and in the meantime, the whole release process grinds to a halt because we can't roll out broken code!
Merge Parties are also expensive affairs. At least one person (maybe more) from each feature team now has to be present to collaborate on the fixes of all these misalignments - and depending on the size & complexity of the application that could take a LONG time. Now you've got multiple members of the team who have been taken off the productive task of building and living in the hellscape of merging. There's gotta be a better way...
And that way is Continuous Integration (often called "CI")! The general axiom of CI is that the merged codebase is ALWAYS in a deployable state.
Well how the heck do you do that when you have large features to merge together???
You don't.
The secret to CI is thinking of the work that gets delivered in a different way. Instead of "build the whole feature", CI promotes the idea of "make the units of work smaller and merge them constantly".
Merges are hard because they're large. In fact, there's a pretty direct correlation between the size of the code that you're merging in and the difficulty of making the merge successfully! So the trick is to change the rules by merging smaller things more often. This is a fundamental DevOps principle that I think applies to life in general:
If anything is painful to do, you should do it more frequently in smaller increments. -- Ben's Law
So how do I merge smaller increments???
This can get tricky - after all, does that mean you merge parts of features? How does that mean that the merged result is always deployable??? Well, there are a couple strategies...
Think Agile
One of the best things about Agile project management methods is that it encourages defined work to be partitioned in small units. To use some Scrum terminology, a card with 85 story points won't fit in a sprint! Those smaller increments of work, if defined properly, would be standalone units of value that are delivered... and since value is defined by the customer who uses it, that means they have to be things that customers can actually use! So as the new, smaller-definition units of value are completed, they can be merged - which means more merges that are individually smaller.
Feature Flags
Wrapping parts of your code in Feature Flags can help you keep always-deployable code active and keep partially-written sections inactive. This technique has great benefits, not only for the development process but also during maintenance later on! If your application can turn off features easily, your operations team will be able to prevent users from going to certain places when there's scheduled maintenance or external outages that aren't in your control to resolve - for example, if you had a 3rd-party web service that was called in a feature and the other company's service was offline. Do you really want your application to break every time someone calls that external service? Having a feature flag around it can help you handle the external service's downtime gracefully and provide a better user experience.
Uh Ben, doesn't lots of merges == lots of work?
There's a sneaky little implication in the world of CI, isn't there? In order to properly "do CI", you need your merge product to always be deployable. In order to know that something's deployable, you have to test it!
...you DO test your code, don't you?
Automated Code Testing is something you should be thinking about anyway - at the very least you should be unit testing!
I know this sounds overwhelming if these concepts are all new to you. Hang in there - they're all interconnected and even though at first it seems like you're having to re-learn everything you ever knew, you're going to see how these items all come together.
The Continuous Integrator's Checklist
So what does your continuous integration practice need to have in order to be successful?
Frequent merges. This is a discipline you train. Get in the habit of merging your code as quickly and as often as possible.
Build / Compile automation. There are loads of tools, like GitHub Actions, or Cloudbees Jenkins, or CircleCI, or Earthly Technologies, that will allow you to set up automatic builds of your application when you merge code. Shop around and find the best one for your needs, and even if you don't have it do anything else, let it compile & build your application and tell you if there were errors. You'll also notice that these tools call themselves "CI/CD"... I'll talk about the CD part in a future post!
Unit Tests! I've known so many programmers who never learned the value of unit testing... so many, in fact, that I'm going to have a future post just on the practice of Unit Testing. If you do this already, you know the value of those tests to ensure that you didn't break anything with your changes... make sure that running your tests is part of your compile/build process in the CI tool you picked for #2, and you'll thank yourself later on!
A notification system you won't ignore. The results of your CI builds should alert you if something's wrong... and they should do it in a channel you're not going to mute. My general recommendation is to only alert if there's a failure, to minimize the number of messages you receive - whether you use email, or SMS, or Slack... just make sure you pick a place that gets your attention! The reason for this is simple:
When the build fails, everyone should drop everything and ensure it's fixed before proceeding.
The fundamental CI principle is that the main branch is always deployable code... so when it isn't, it's everyone's first priority!
Wrapping up
Continuous Integration makes us better at what we do by establishing some guard rails... our product is always ready to deploy, our work is broken down into sufficiently small units for delivery, and we quickly diagnose and respond when things aren't right.
CI is the basic first step for DevOps because it provides a solid foundation upon which we can do a lot of amazing other things. Tune in next week for a discussion of CD... which stands for, well, maybe a couple of things! You'll just have to come find out!
Top comments (0)