Best Practices for Onboarding Security & Compliance Scanning Tools

Introducing, adding, or replacing a new binary security and compliance analysis tool into your SDLC, if not handled correctly, can be very disruptive to the SDLC and organization.
This article will go over what I believe is a best practice for onboarding such tools in order to reduce disruption, improve adoption, and shift left to create a DevSecOps behaviour.

First, let’s begin by describing the scenario I think about when talking about onboarding security tools. What usually happens, especially when introducing a new tool, is that all screens go red and alerts appear from every direction. In such scenarios, it is a reasonable knee-jerk reaction to require a system lockdown; disqualify builds, reject dependencies, etc. However, such a reaction, even though it may seem valid, is counter productive. Such a behaviour, in theory, will bring production to a halt, cause frustration and conflicts. Naturally, in real life, in most cases, no one will really stop the business. Rather, the organisation may develop Alert Fatigue and will ignore the tool, which is definitely not something we want. Keep reading to find out how to avoid this scenario and address onboarding DevSecOps tools with minimal disruption and maximum gain.

The following is a list of 5 suggestions when onboarding SCA (Software Composition Analysis) tools:

1. Involve R&D

   When you involve R&D in the process, you have a real chance at achieving a real DevSecOps process that is manageable and productive. There is approximately one security engineer per 100-200 developers. Reviewing and controlling all security issues by a single engineer is only possible when it is attached to development. Otherwise it will cause bottlenecks, delays, redundant work, and a lot of frustration.

2. A “watch” per application team

   Each security tool uses a different name, but most of them have some notion of scope. By scope I mean ‘some way to group a set of resources (repositories, folders, builds, etc.) for the application of relevant security and compliance governing rules.
   Creating a “watch” per application team allows giving every team their own “world” and responsibility. In turn, this provides the ability to “shift left” the responsibility of security governance.

3. Dev tools integration

   Try to bring the information into the development tools (i.e. CI server, IDE), which will provide governance related information into the developmer’s environment.

4. Start small and work in cycles

   • Choose one team to start with.
     • Considerations for choosing the right team:
       • A vanguard team - a team which is open to changes and testing new ways to improve (especially if they care about security).
       • A new app team - if you have the option to, choose a team which is starting a new project/service (i.e. starting “greenfield”)
       • Or a team with less “integrations”/dependency with other teams.
     • Work with the chosen team to establish and improve the process before expanding to other teams.
   • Start with “Critical” issues - As mentioned in the introduction, introducing a new SCA tool will most likely generate tens or hundreds of alerts of all types and severities. Even in small scopes, trying to handle everything at once might not be reasonable. If you see that even the number of critical issues is too high and your tool supports it, try to be even more granular than Low/Medium/High/Critical. Try to go by CVSS score; start with 9.9-10, then 9.8-10, 9.7-10, etc. The granularity of this filtering should be based on your needs.
   • Make a decision for each “critical” - Decide whether the issue needs and can be fixed, or whether to whitelist the issue. This might be a decision to temporarily, permanently whitelist the issue, or define a deadline for fixing the issue.
   • Do not use “brute force actions” - Most tools allow you to take actions. Email notifications, webhooks, Slack, Jira, etc. Some tools provide additional capabilities such as blocking download, failing builds, or similar. Until you finish the cleanup of manageable existing issues (i.e. issues you can resolve or ignore) avoid any action that will disrupt the development lifecycle such as blocking download of a needed dependency or failing a build based on the new tool’s scan results.
     • Only when the build passes without violations consider a hard action future.

5. Add external notifications (Jira, Slack, etc.)

   • You probably want to add external notifications only for new issues, otherwise you may be risking “Alert Fatigue”.

After handling criticals repeat the same process for major issues. For minor issues, you should consider if you want to handle them at this point in time, or rather move to the next project/team and only then deal with them.

This post provided a general best practice flow for introducing a new SCA tool into your organisation/environment. Hopefully this will help you remove any tension that you may have between Development and Security, as well as provide some DevSecOps foundation.

Note: A version of this blog post is also published here