Why is it that we’re seeing the term “Software Supply Chain Security” so frequently? The usage of third-party packages and libraries within software has been on the rise in recent years. You’ve taken steps to build a secure application, but are you certain that the third-party packages that your software depends on are secure as well?
Software supply chain security is the field of ensuring security not only for your application but also the components that go into your software and the entire process of how you build and deliver the software. Another often-overlooked field of software security is data infrastructure security; especially authentication and access control for your data related resources.
The 7th annual All Day DevOps, or “ADDO”, took place November 10, with 180 speakers and almost as many sessions. Many of the brilliant speakers, including the opening keynote speaker Sean Wright, focused on software security. While the videos are available on demand, my Aiven colleague Dewan Ahmed and I summarized our learnings in this blog.
The Rise of the Supply Chain Attack
Sean Wright, Principal Application Security Engineer at Featurespace, kicked off the event with his keynote “The Rise of the Supply Chain Attack”. Sean referenced Sonatype’s State of the Software Supply Chain report which shows a staggering growth of open-source components in use, and paints a picture: the attack surface has increased in size as a result.
From the 8th annual State of the Software Supply Chain report (2022)
Sean remembers his early days in the industry where most dependencies were downloaded manually and the choice of open-source needed justification. While open source brings speed, Sean reminds us that speed opens the door for a whole range of attacks, including dependency confusion, typosquatting, and, recently, protestware.
Your production workload might be behind a secure firewall. But what if your software uses a package and malicious code was part of that package? Now the software is running in production including that piece of malicious code and the attacker could have the keys to the kingdom.
This is not fiction and many of us are aware of the SolarWinds incident. While incidents like these have served as a driving force behind strengthened software security and SBOM (Software Bill of Materials) adoption, Sean thinks we need ever more awareness as and across the industry around validating transitive dependencies.
Mitigation, according to Sean, is a combination of appropriate (network) access control, SCA (Software Composition Analysis) tooling to manage your policies around CVEs, and purging “all the things”. He also thinks MFA (multi-factor authentication) for authors of (critical) packages should be required. Sean gets his vulnerability insights from deps.dev, ossindex.sonatype, and cvedetails.com, and closely monitors interesting initiatives such as the OpenSSF Security Scorecards - a tool to assess open source projects for security risks through a series of automated checks.
State of the Software Supply Chain
In order to harden your applications, you need to know what you’re dealing with. Dr. Stephen Magill, VP of Product Innovation at Sonatype, describing the results from the latest Sonatype State of the Software Supply Chain Report, says that while the 1st generation exploits were hackers using existing flaws in packages, this generation is introducing vulnerabilities. Moving to the latest version of a component is no longer the solution.
The State of the Software Supply Chain reports that there are as many as 97,334 malicious packages in October 2022, and a 742% average YoY growth.
While the perception of Open Source that it is risky is true in some part, 98% of projects have safe versions available, and most vulnerabilities are patched before they're disclosed. Log4j was patched in 15 days.
We are “just” terrible at managing open source. 68% of the report’s respondents are confident that they are not using vulnerable versions, but 68% of applications use a component with a known vulnerability. Although media attention improves remediation, the industry is generally slow in adopting patches.
In his talk, Laurent Simon, Security Engineer in the Google Open Source Security Team (GOSST), elaborated on the OpenSSF’s Scorecard. Cool data point: since its v4 release in January 2022, Scorecards has been installed on over 800 GitHub repositories as of March 2022 and is recommended by the GitHub documentation to harden workflows.
Signed, slimmed, SBOM-ed
A Software Bill of Materials is a complete, formally structured list of components, libraries and modules that are required to build a given piece of software and the supply chain relationships between them. A good SBOM is a list of ingredients that is also a recipe for a reproducible build, says Shelley Lambert, Software Engineer at Red Hat and PMC member at Eclipse Adoptium.
The "list of ingredients" is a great analogy. Reading the ingredients list in your hand before using it prevents using ingredients in case of food allergy or intolerance. The (continuous) exercise is more important than the format (SPDX, CycloneDX, SWID, …) you chose to start creating your SBOM with.
Lars Lefebvre, among other things, contributes to the OWASP project. And he’s all about signing your containers. Lars also encourages organizations to improve their security posture by scanning container images for known vulnerabilities, using up-to-date and lightweight images, among other things.
In “A Different Kind of S3: First Line Security of the Supply Chain” Pieter van Noordennen, Senior Director of Growth at Slim.AI, focused on three core areas to secure the supply chain: SBOMs, Signing, and Slimming.
US president Biden's Executive Order of May 12, 2021, requires any vendor selling software to the US government to include an SBOM for each product. The CNCF is teaming up with industry to bring more standardization to SBOMs. The IETF is working on standardizing SBOMs in a new working group - Supply Chain Integrity, Transparency, and Trust (SCITT).
The purpose of signing is to provide cryptographic proof that the author is who they say they are and that the code has not been changed or corrupted since it was signed. Likely registries will adopt / enforce container signing, and the OCI (Open Container Initiative) proposal for encrypted images is just a sign (get it?!) on the wall.
The safest software is the software you never ship. Only ship what you need to prod. While SBOMs and signing are necessary - without slimming, the solution is incomplete. Slimming focuses on reducing the attack surface, which helps you with both known and unknown vulns (zero-day attacks).
Data Infrastructure and Dynamic Secrets
Keeping secrets is hard for us humans. How about computers? How good are they in keeping your private keys… private? Jeroen Willemsen is the project lead of OWASP WrongSecrets, a project which came out of the 2020 All Day DevOps edition. A vulnerable app and secret detector testbed, WrongSecrets’ goals are to educate on secret management and its pitfalls, help people reflect on their secrets management strategy, and to promote secrets management as an important facet of security. The WrongSecrets project is sponsored by the likes of GitGuardian, JetBrains, Docker, and 1Password, and welcomes feedback, ideas, and contributions.
Aiven’s own Dewan Ahmed, posed the question “Who’s Managing the Credentials for Your Data Infrastructure?”, and opened with one of the largest data breaches in history in terms of the number of people affected - the 2022 Shanghai National Police database breach.
Figure: Visualization of world’s biggest data breaches (source)
Dewan raised a key question about data infrastructure security and proposed the use of dynamic credentials to provide time-bound access to cloud resources for your human and machine users. If your human user is working for 8 hours, you could provide them a credential that is valid for 8 hours. However, if your application is accessing the database for a few seconds, you can generate a dynamic credential that might only be valid for a few minutes. None of these cases require you to keep the same database credentials valid for months or years.
He used an open source secret management tool, HashiCorp Vault, to address Authentication, Authorization, and Auditing challenges (the 3 “A”s of the AAA security model). The appeal of Vault is in its many features - flexibility of using API/CLI/UI, a variety of secret engine backends to generate credentials of (from cloud providers to databases and more), ability to handle X.509 certificates, and more. However, the biggest reason why businesses adopt Vault is because it has been running in production by a large number of customers and is battle-tested.
Dewan reminded the audience that a break-glass-procedure is still required besides adopting a secret management tool. A break-glass-procedure is a bypass mechanism in case the traditional security processes fail. A handy take away from his talk is a link to this blog which contains the demo Dewan showed to secure a PostgreSQL service.
Is Open-source Helping or Hampering Your Security?
Now that you are aware of the software supply chain and data infrastructure issues, how do you take advantage of open-source technologies to solve the problems? Do you still have some doubts about open-source and how secure the technologies are?
Neha Sharma is part of the DataOps team at Xenonstack, and according to a report by Open Logic she references, there has been an increase in the use of Open Source Software over the last year in organizations, most notably when it comes to frameworks, data tech, and version control:
Image reference: https://www.openlogic.com/resources/2022-open-source-report
Neha’s checklist aims to separate the healthy projects from the risky packages. A project scores one point for each "yes". You'll have to determine how many points are required to pass, based on your risk tolerance. We took the liberty to convert the checklist into a flowchart:
Image description: Project readiness and risk factor assessment (credit: Neha Sharma)
Security Champions
Chris Romeo spent 10 years at Cisco, and is the co-host for The Application Security Podcast. He was one of a number of speakers who talked about Security Champions, who he defines as security-passionate persons engaged with your security team, interested in expanding their knowledge and experience with security.
There’s a need for security champions, says Chris. According to Building Security In Maturity Model BSIMM-12, for every 135 developers there's only one Software Security Group (SSG) member. SSG is the internal group charged with carrying out and facilitating software security within a company. The goal of a championship program then is to create a community of security force multipliers. Chris discussed how to create such a program in his talk, so be sure to check that out.
Neelu Tripathy (Security Practice Lead at Thoughtworks India), Ann Marie Fred (Senior Principal Software Engineer at Red Hat), and Siddharth Pareek (Senior Vice President Consulting at the NatWest Group) all talked about similar programs. Tracy Bannon, Senior Principal in MITRE Corporation’s Advanced Software Innovation Center, in her session, talked about building a shared cybersecurity culture.
Turja Narayan Chaudhuri (Assistant Director Cloud Platform at EY GDS) explained in his talk “Your developers decide your security posture, not your security teams” that it’s paramount for a security initiative to involve the development team in order to succeed. He added that security processes and frameworks should not conflict with developer experience or productivity.
In conclusion
With your newly gained knowledge about the (open source) software supply chain and data infrastructure security, are you feeling worried or empowered? We sure hope it is the latter. Being aware of your security posture, and armed with best practices like container signing, slimming, and modern secrets management, you’re well underway on your security journey.
Conferences like AllDayDevOps play a key role in fostering an open discussion around these critical security topics. If you liked the trailer after reading this blog and would like to watch the full talks, all talks are available for your on-demand viewing pleasure.
We would wholeheartedly recommend saving the date for next year’s event, once announced.
Top comments (0)