In my 5 years of working as a DevOps engineer, the speed of CI/CD pipelines has been among the biggest complaints.
Here are 10 tips to ensure that your pipelines run well within 20 mins:
1. Instrument Everything:
Instrument your build pipeline with spans and traces so you can see where all your time is going. ALWAYS. Instrument.
2. Order Your Tests:
Order tests by time to execute and the likelihood of failure.
3. Test Only Changes:
Don’t run all tests, only tests affected by your change. It will improve CI/CD pipeline significantly faster.
4. Reduce Scope:
Similarly, reduce build scope; if you only change front-end code, only build/test/deploy the front end, and for heaven’s sake don’t fuss with all the static asset generation.
5. Don't change availability regions:
Don’t hop regions or zones any more than you absolutely must. This may lead to unpredictable latency issues when running the pipelines.
6. Remove unnecessary tests:
Prune and expire tests regularly, don’t wait for it to get Really Bad
7. Refactor tests:
Combine functionality of tests where possible — tests need regular massages and refactors too.
8. Care and Intention:
Optimized CI/CD pipelines require care and intention to make them faster. Make sure to setup a cadence to do it regularly.
9. Use Shared Environment:
You do not need multiple non-production environments in your CI/CD process. Push your artifacts to S3 and pull them down from shared environment.
10. Measure Time Elapsed:
Set a time elapsed target for your team, and give it some maintenance any time it slips by 25%.
In my experience, some of these tips end up reducing the time for pipeline runs by more than 50%. Incorporating them from the start may help you to not face them at all.
Thanks for reading this.
If you have an idea and want to build your product around it, schedule a call with me.
If you want to learn more in DevOps and Backend space, follow me.
If you want to connect, reach out to me on Twitter and LinkedIn.
Top comments (0)