At a high level, a CICD pipeline has two goals - to build the code and deliver the code. Although this may seem very straightforward to some, building a successful, repeatable pipeline takes a ton of testing and visibility.
When you're building a successful pipeline, you need to ensure that:
- All code prerequisites and dependencies are met (libraries, modules, packages, etc.)
- You understand the ins and outs of the CICD platform you're using. Although the technology itself is always the same, the different CICD platform features and requirements can vary.
- All engineers understand what the pipeline is doing
- Different stages are available (dev, staging/UAT, prod) for testing purposes to ensure the code being deployed is ready for production
After you've built a successful pipeline, how can you maintain its success? How can you ensure it's running as expected without watching it every five minutes?
In this blog post, you'll learn about three key ways to maintain pipeline visibility for GitHub Actions.
Logs From A CICD Pipeline
The first place that you can pull information from, although very Adhoc, are the logs in GitHub Actions. When you create a new GitHub Action, you can see, watch, and review the entire pipeline step-by-step. The output is very crucial because it's going to tell you exactly what's happening, success or failure, in a pipeline.
In the first screenshot below, you can see a list of CICD pipelines (called Workflows in GitHub Actions) that were run. Some succeeded and some failed.
The one negative thing about this approach is if you need these logs later down the line, you won't be able to see them as the logs expire after 90 days. Because of that, it's not exactly a reliable source. If you want to hold onto these logs, you'll want to store those logs elsewhere.
If you want to take a look at pipeline logs that are new, then this approach is fine from an Adhoc perspective. The below screenshot is of a failed pipeline from a Weatherchecker app written in Go (golang) on my public GitHub.
Notice in the below screenshot the error.
You can now see from the GitHub Action logs that this is due to an HTTP 403 error, which is most likely from an API key that has expired, so no authorization can occur, making the pipeline fail. The logs from the CICD pipeline are great for troubleshooting purposes.
Monitoring and Logging a CICD Pipeline
Although looking at the logs via GitHub Actions is great, you'll typically need something more long-term. A solution that you can go back to later, query the results, and even get some alerts on. That's why you should set up some monitoring and logging for CICD pipelines.
Depending on your monitoring/logging solution, it's typically going to consist of a few steps:
- Authenticate from wherever the data is (the pipeline) to a logging and monitoring solution.
- Send the data via some integration. For example, there's a Gitlab integration to send pipeline information to Datadog.
- Get creative. Write a script that can, for instance, collect the data from GitHub and send it to New Relic using New Relics API.
Another great way to do this is with a Value Stream Management (VSM) platform. VSM is still pretty new, but there are several platforms and tools out there. VSM at a high level is a way to show value from an engineering perspective. A lot of the tools and platforms can collect data about pipelines.
Here are some great links for different monitoring and logging tools on how to collect pipeline data.
- https://docs.newrelic.com/docs/new-relic-solutions/new-relic-solutions/measure-devops-success/measure-code-pipelines/
- https://docs.datadoghq.com/continuous_integration/?_gl=1*qltzfp*_ga*MTYwMzQ1ODI0MS4xNjMxNjM1MDUw*_ga_KN80RDFSQK*MTYzNDA0MjY5NS4zLjEuMTYzNDA0MjcxOC4w&_ga=2.77239314.603174302.1634042696-1603458241.1631635050
- https://www.splunk.com/en_us/devops/cicd-pipeline-monitoring.html
Messaging Results
Messaging apps (Slack, Discord, MS Teams) are, at a rapid rate, replacing email and how we communicate. Although you shouldn't spend your entire day on these messaging apps because they severely negatively impact productivity, they're great if you want to set up a Channel to capture pipeline out.
There are some great ways to set up integrations for Slack. You can have a message come into a specific channel every time a pipeline is run, a pipeline fails, a pipeline is successful, etc.. You can even show who ran the pipeline. It's a great way to keep tabs on what's going on from a CICD perspective and depending on the Slack plan that you have, those messages will be permanent, so you can go back later and review them.
Here are two ways that you can integrate GitHub Actions and Slack:
- https://github.com/marketplace/actions/slack-notify
- https://github.com/marketplace/actions/slack-github-actions-slack-integration
Closing Thoughts
A lot of organizations at this time are still trying to fully understand CICD and how to use it effectively. Because that's such a big goal in itself, they aren't really thinking about the long term, like how to view data about the pipelines and see productivity/value.
Logging and monitoring pipelines are crucial for any organization that wants visibility into the tools that are making their applications get out into the world, which is what's making the organization money.
Latest comments (0)