It appears that the internet is on fire again. This time in a story reminiscent of Cliff Stoll's hunt for a 75 cent accounting discrepancy, a software engineer doing some profiling noticed slightly elevated CPU usage where it shouldn't be. He tugged on that thread and discovered a cleverly obfuscated backdoor in the XZ compression utility that leads to unauthenticated SSH logins.
ℹ️ tldr; if you don't have time to read the full post, we have released a Puppet module that can help detect the current known xz backdoor.
What makes this compromise so concerning is that it was perpetuated by a long-term known contributor, with maintainer access to the XZ GitHub repository. This malicious actor has been working hard for at least two years to lay the foundation for this backdoor. They utilized sockpuppet accounts to pressure the original maintainer to accept help from a relatively unknown contributor and then later to weasel the compromised library into popular Linux distributions.
This attack was not only technical in nature, but also compromised the social network foundation of the open source community. We will be learning and evolving from this attack for years.
Our current understanding says that the XZ backdoor is the only active compromise, but due to the convoluted and long-term nature of the attack, everything they've touched for the last two years is suspect. And because the malicious actor had admin access to the XZ repository and could have easily spoofed commits, all activity in the repo is also suspect.
We'll be untangling this for a while. What we have today is a quick script to detect the known compromise.
Nick Burgan, a software engineer at Puppet whose name you might recognize from their community engagement, took the initiative to build a quick module which orchestrates that detection script across your infrastructure.
All the usual disclaimers apply. We currently have no way of knowing how complete that detection script is. The nature of the compromise means that our understanding of it will continue to evolve for weeks and new detection methods will be discovered. Your help in keeping the module current with the latest detection methods would be greatly appreciated!
This module provides both a task which you can run interactively across nodes in your infrastructure and can also set up a scheduled task to run the detection script daily. We encourage you use this scheduled task and to pin the module to the latest release in your Puppetfile to ensure that you get updates. This will ensure that when we add improved detection methods, your infrastructure will be running them shortly.
# Puppetfile
mod 'puppetlabs-xzscanner', 'latest'
Then classify all nodes with xzscanner. You might do that by putting it in a base profile class, or by adding it to the global site.pp.
Header photo from https://www.flickr.com/photos/jeremybrooks/2398999602/
Top comments (1)
Sadly this has happened before, and will keep happening, again and again, because not enough people or entities even care enough to make any other outcome really possible. It makes it even harder to get newer software accepted, too, meaning ever fewer people will have time. Instead of making software exciting again, we drive away more people, and rather than appreciating skills, the community now even turns on and actively excludes those disabled, too.