As a developer, you build new features, fix bugs, and ship code. That's what the original job description included. Software developers bear additional responsibilities today, such as expertise in deploying or securing software. You could make a convincing case that security is a production-only concern. As long as you're working on your features, security is not essential (yet). But what if security issues arise in production? These kinds of tickets return to engineering and interrupt everything you're doing. While these situations can happen anytime, you can play an active role in preventing them in the first placeโthe result: Fewer high-priority tickets and interruptions.
In this blog post, let's explore tools you can use to be proactive about a class of security incidents.
What Security Issues are we talking about here?
Over the last few years, companies like GitHub invested heavily in making software more secure. Dependabot automatically scans your dependency lists (i.e., package.json
) and submits pull requests if you're using a vulnerable library version. Since it's such a low-effort workflow, developers can quickly patch vulnerable packages as part of their regular workflow.
In this blog post, we want to shed some light on another aspect of software security: Production runtime. If you're building Docker Images as part of your CI/CD workflow, this blog post is for you. No matter what specific build strategy you use for your images, you likely install additional packages to ensure your code runs without issues.
Insecure Docker Images?
Have you ever considered the security of the Docker images you run in production? If we use the current Ubuntu LTS image, it has 106
packages installed:
docker run --rm ubuntu:jammy dpkg -l | wc -l
106
Not using Ubuntu? Alpine only has 15 preinstalled packages:
docker run --rm alpine:latest apk list -i | wc -l
15
If a Docker image has 15 or over 100 installed packages, any of these packages could potentially be vulnerable to an attack. If you think, "Ok I'll patch once I see a report in the news" (like Heartbleed or Log4Shell), you're "missing out" on most patch opportunities. Most (severe) security vulnerabilities don't get a front-page listing on Hacker News.
There's no need to go through the CVE database manually. Instead, we can work towards a workflow that resembles GitHub's Dependabot. We can scan Docker Images as we build them to find out if there are any vulnerable packages and patch them. While we will never achieve 100% security,
we're getting a step closer with a low overhead process.
How to scan Docker Images
Introducing KubeClarity. KubeClarity is an open-source project to help you ship more secure software. While KubeClarity covers many different use cases, let's focus on image scanning for now.
Installation
Check out the KubeClarity README to find installation instructions for your specific platform. In this
tutorial, we mainly use the CLI but feel free to install the Dashboard for visualization.
Scan your Image
To illustrate image scanning, we use this repository,
containing an example Rust workload with a vulnerable OpenSSL version. Run the following command:
kubeclarity-cli scan ghcr.io/schultyy/rust-workload:0.0.3 --input-type image -o table
NAME INSTALLED FIXED-IN VULNERABILITY SEVERITY SCANNERS
perl-base 5.28.1-6+deb10u1 CVE-2023-31484 HIGH grype
libgcc1 1:8.3.0-6 CVE-2018-12886 HIGH grype
libsystemd0 241-7~deb10u9 CVE-2019-3844 HIGH grype
openssl 1.1.1n-0+deb10u3 1.1.1n-0+deb10u4 CVE-2023-0215 HIGH grype
ncurses-base 6.1+20181013-2+deb10u3 CVE-2023-29491 HIGH grype
libudev1 241-7~deb10u9 CVE-2019-3844 HIGH grype
openssl 1.1.1n-0+deb10u3 1.1.1n-0+deb10u5 CVE-2023-0464 HIGH grype
libstdc++6 8.3.0-6 CVE-2019-15847 HIGH grype
openssl 1.1.1n-0+deb10u3 1.1.1n-0+deb10u5 CVE-2023-2650 HIGH grype
openssl 1.1.1n-0+deb10u3 1.1.1n-0+deb10u4 CVE-2023-0286 HIGH grype
libc-bin 2.28-10+deb10u2 CVE-2020-1751 HIGH grype
openssl 1.1.1n-0+deb10u3 1.1.1n-0+deb10u4 CVE-2022-4450 HIGH grype
libss2 1.44.5-1+deb10u3 CVE-2022-1304 HIGH grype
libsystemd0 241-7~deb10u9 CVE-2021-3997 MEDIUM grype
libsystemd0 241-7~deb10u9 CVE-2022-3821 MEDIUM grype
openssl 1.1.1n-0+deb10u3 1.1.1n-0+deb10u5 CVE-2023-0466 MEDIUM grype
libudev1 241-7~deb10u9 CVE-2022-3821 MEDIUM grype
openssl 1.1.1n-0+deb10u3 1.1.1n-0+deb10u4 CVE-2022-2097 MEDIUM grype
libudev1 241-7~deb10u9 CVE-2021-3997 MEDIUM grype
libsystemd0 241-7~deb10u9 CVE-2022-4415 MEDIUM grype
libudev1 241-7~deb10u9 CVE-2022-4415 MEDIUM grype
openssl 1.1.1n-0+deb10u3 1.1.1n-0+deb10u4 CVE-2022-4304 MEDIUM grype
openssl 1.1.1n-0+deb10u3 1.1.1n-0+deb10u5 CVE-2023-0465 MEDIUM grype
libpcre3 2:8.39-12 CVE-2020-14155 MEDIUM grype
libgcrypt20 1.8.4-5+deb10u1 CVE-2019-13627 MEDIUM grype
login 1:4.5-1.1 CVE-2023-29383 LOW grype
bsdutils 1:2.33.1-0.1 CVE-2021-37600 LOW grype
(Output truncated for brevity)
The kubeclarity scan
command outputs a list of packages for which CVEs have been filed. openssl
comes up several times, with a version that fixes the issue (See FIXED-IN
column).
Fix Vulnerable Packages
Open the Dockerfile
and move to line 19. Line 19 right now installs a specific openssl
version:
RUN apt-get update && apt-get install -y openssl=1.1.1n-0+deb10u3 && rm -rf /var/lib/apt/lists/*
Update this line to install the latest version according to the kubeclarity-cli
scan result:
RUN apt-get update && apt-get install -y openssl=1.1.1n-0+deb10u5 && rm -rf /var/lib/apt/lists/*
It indicated that issues would subsequently be fixed in
1.1.1n-0+deb10u4
and 1.1.1n-0+deb10u5
. Therefore, we'll go with the latest version available.
Let's build a new image:
docker build -t ghcr.io/schultyy/rust-workload:0.0.4 .
To verify the issue has been fixed, we rerun KubeClarity, this time with the LOCAL_IMAGE_SCAN=true
variable set. We want to scan our local image before pushing, verifying that OpenSSL does not have any outstanding vulnerabilities:
LOCAL_IMAGE_SCAN=true kubeclarity-cli scan ghcr.io/schultyy/rust-workload:0.0.4 --input-type image -o table | grep openssl
openssl 1.1.1n-0+deb10u5 CVE-2010-0928 NEGLIGIBLE grype
openssl 1.1.1n-0+deb10u5 CVE-2007-6755 NEGLIGIBLE grype..
The scan still lists two openssl
entries, though only with severity of NEGLIGIBLE
. Therefore, the issue has been fixed.
What's next?
Check out the KubeClarity GitHub repository to learn about additional use cases, and make sure to star the repository!
Top comments (0)