API's From Dev to Production - Part 10 - CodeQL

Series Introduction

Welcome to Part 10 of this blog series that will go from the most basic example of a .net 5 webapi in C#, and the journey from development to production with a shift-left mindset. We will use Azure, Docker, GitHub, GitHub Actions for CI/C-Deployment and Infrastructure as Code using Pulumi.

In this post we will be looking at:

• SAST - Static Application Security Testing is used to secure software by reviewing/scanning the source code of the software to identify sources of vulnerabilities.

TL;DR

Static Application Security Testing is a very important step in the journey from dev to production. With an abundance of tools out there on the market, there's bound to be one that suits you. With GitHub CodeQL, we can achieve this in little time with little effort.

You can increase your chances of finding security vulnerabilities before they reach production

GitHub Repository

Requirements

We will be picking-up where we left off in Part 9, which means you’ll need the end-result from GitHub Repo - Part 9 to start with.

If you have followed this series all the way through, and I would encourage you to do so, but it isn't necessary if previous posts are knowledge to you already.

Don't forget to ensure you have setup Code Climate Quality with your repository.

Introduction

As we have discussed in this blog post series and in shift-left, security testing is part of actually shifting-left! We need to carry out security testing, and of course we want to automate it, and ensure its early on in the process. There are many tools out there in the market that can scan your code (statically); the newest entrant is from an acquisition of Microsoft's GitHub - CodeQL.

In this post we are going to go through the steps on how to add SAST using CodeQL to our API.

A bit of history about CodeQL

GitHub acquired Semmle in 2019 for an undisclosed sum, it originally came out of research completed at Oxford University.

The technology is based around queries and its language CodeQL, the analysis engine enables the query of large codebases to identify code patterns and search for vulnerabilities and their variants.

GitHub has now baked this into its platform and has simply made it oh so easy 😁

Why is SAST so important?

SAST makes discovering common vulnerabilities simple, and more critically, automated. SAST is a really good way of improving overall good-practice among your engineers.

Usually engineers outnumber security staff and therefore, it can be rather challenging for any company to find the people to perform code reviews, specifically those with a focus on security. SAST tools provide the ability to analyse 100% of the codebase and are certainly faster than manual code reviews. These tools can easily scan millions of lines of code in a matter of minutes, and automatically identify critical vulnerabilities; such as SQL injection, cross-site scripting, and more with a high level of confidence.

SAST helps integrate security into the early stages of the software development lifecycle and enables the culture of shifting-left; to detect vulnerabilities in the proprietary code in the design stage or the coding stage when they are relatively easier to mitigate.

Step 1

Navigate to your GitHub Repo → Security → Code Scanning Alerts → Click Set up this workflow

Step 2

This will generate a new GitHub Actions Workflow file called, codeql-analysis.yml - You're free to change the name, but I've kept it the same.

The intelligent thing here is that GitHub will detect the languages inside your repo, in our case, it's picked-up C#; which is of course correct 🤓

One thing to note is the cron, for me, it did generate an invalid value according to GitHub. The last digit is the day of the week, 0 = Sunday; for some reason GitHub doesn't like Sunday... In this case, I've changed it to the below, whereby I've set the day of the week to 1 = Monday.

schedule:
    - cron: '44 23 * * 1'

For more information about the crontab format, please see here

Commit your changes and head on over to the Actions tab to see it running.

Here you can see I have a new workflow on the left and it has executed in 4 minutes and 4 seconds!

You'll notice it has done an autobuild; based on its language detection, CodeQL attempts to build your code, its image has loads of the .NET SDK's to cover everything.

It is worth looking through the build output, even just to get a general understanding on what is going on here.

📄 Autobuild
This means it will build our API twice, yes, twice! Is that OK?
Not ideally but... I have no issues with it at present, it will build in parallel to our container build, the key is to scan the code, anything built is discarded anyway, we don't need it, just the results.

It is not optimal given we are using Docker containers, however, there is a way to bake this into Docker, but the documentation is a little light at the moment and requires a decent deep-dive into it - I will most likely investigate it soon, and of course, create a blog post about it.

For more information about CodeQL in containers, please see Running CodeQL code scanning in a container

Step 3

We can configure a few things that are not enabled by default such as additional queries

I have left the default queries and listed some extensions, in our case, I'm adding extended security and security and quality queries, and of course C# 👍

This configuration simply goes into a yaml file.

1. Create a folder called, codeql under the .github folder.
2. Copy & paste the following code into a new file called, codeql-config.yml - no need to commit your changes yet, as there is another change in Step 4.

Code
codeql-config.yml

name: "default"

languages: csharp

queries:
  - name: Extended Security
    uses: security-extended
  - name: Security and Quality
    uses: security-and-quality

Step 4

We need to reference our new CodeQL configuration file in our Workflow.

In the Initialize CodeQL Workflow Step → Add a new line to its with clause like so below:

Code
codeql-analysis.yml

config-file: ./.github/codeql/codeql-config.yml

Step 5 (Extra)

If you're like me and really like branch protection rules and GitHub Status Checks, you'll want to create/modify your branch protection rule to have both your workflows as required to pass before merging.

Finally, commit your changes.

Step 6

If you navigate back to Code scanning alerts under Security, hopefully you'll see no alerts!

Here, other tools can publish their results too, by supporting the open SARIF standard.

Which means even if you expand your toolset later, you can have the same GitHub native experience! Pretty cool right?

Step 7 (Extra)

While we are here in the Security tab, I applaud you to ensure you have Dependabot alerts on.

What have we learned?

We have very quickly enabled our API to be scanned by GitHub CodeQL, a SAST analysis engine. It's easy and provides a single pane for you to see any security vulnerabilities in your code. If you end up using another product, these products can push their results into the GitHub Security area of your repository; so long as they integrate with GitHub and the open SARIF standard.

We understand that it will build our API again (but in parallel), but this is not a huge, big deal, as we are just after the results, not the build from that workflow.

Security should always be part of your engineering process.

Dynamic Application Security Testing is another form of security testing, the key point is it being dynamic VS static - We won't cover this here, but there is value to be had with the combination of SAST and DAST. I may cover it in another post in the future 😉

Next up

Part 11 in this series will be about:

• Infrastructure as Code
  • Pulumi

More information

• https://docs.github.com/en/code-security/secure-coding/about-code-scanning
• https://docs.github.com/en/code-security/secure-coding/configuring-code-scanning
• https://www.azure.com
• https://dotnet.microsoft.com/
• https://www.github.com
• https://www.docker.com
• https://www.pulumi.com
• https://www.codeclimate.com/quality