DevOps practices have significantly transformed the software industry, leading to faster release cycles and more streamlined workflows. The enduring presence of the DevOps model is undeniable, and its influence on modern development methodologies is profound. However, this accelerated pace introduces challenges, particularly regarding DevOps security. Outages, human errors, cyberattacks, data breaches, ransomware, and security vulnerabilities, often leading to data loss, are the realities that DevOps has to confront every day.
Our The State of DevOps Threats Report, an expert’s analysis of GitHub, GitLab, and Atlasian-related security challenges, shows that the number of incidents with the greatest negative impact on DevOps services didn’t exceed 14%. Moreover, none of the DevOps providers experienced a complete downtime, as it happened in 2022 when Atlassian experienced a massive outage for almost a fortnight and the estimated number of affected users might vary from 50K to 800K. However, the majority of issues affected components’ operation, resulting in degraded performance at various levels.
Constantly rising threats and the necessity to embed security within development processes have given rise to DevSecOps where development and operations teams work together with security teams and all their processes are converged. In DevSecOps, security is not an afterthought but an integral part from the outset of the software development lifecycle. Thus, in the report we’ve covered not only practical examples and use cases of threats and vulnerabilities, but also the best, thoroughly tested recommendations to guide you to security.
The number of incidents in GitHub saw a 20%+ growth
In 2023, incidents that impacted GitHub users grew by over 21% compared to the previous year, with the first quarter seeing the highest activity.
This year, GitHub recorded 13.94% of events that had a major impact on the service.
One of the major vulnerabilities that GitHub users were facing in 2023 was Repojacking, a supply chain vulnerability with a subdomain takeover-like conceptual underpinning. Thus, in June 2023, the researchers from AquaSec concluded that 9 million of GitHub repos might be potentially vulnerable to Repojacking. Later on, in December, the Checkmarx team discovered that GitHub’s vuln could have exposed over 4K packages to RepoJacking attacks, and finally VulnCheck had been investigating this issue and found out that over 15K Go module repos are vulnerable to this kind of an attack.
Additionally, hackers exploited GitHub by hosting malware on the platform, using it as a dead-drop resolver to access the actual command-and-control (C2) address. This tactic allowed threat actors to build a reliable, low-cost attack infrastructure, posing significant risks to other users and their data.
One of the toughest times for GitHub users appeared to be May 2023 when GitHub had to handle 3 incidents in 3 days in a row. GitHub experienced a major outage when 8 of 10 services were degraded due to configuration changes and a lack of possibility to read replicas.
About 32% of incidents in GitLab affected service performance and impacted customers
About 32 percent of events in GitLab were recognized as having an impact on service performance, preventing customers from performing with full capabilities. However, compared to 2022, the number of incidents in 2023 slightly decreased – in 2022, GitLab recorded 84 events, and in 2023 there were 76 of them.
The “hardest” months for GitLab users were June and August when the service provider experienced 10 incidents per each month with very high CVSS scores. In June, there was one major service disruption and four smaller incidents classified as partial disruptions. August saw the highest number of degraded performance events over the past year, with five recorded incidents.
Moreover, in August, GitLab faced a sophisticated attack that not only compromised its security but also enabled a new Proxyjacking scheme. The attackers initially exploited the CVE-2021-22205 vulnerability (with a CVSS score of 10.0) to gain access to the container. This vulnerability opened the door to potential ransomware, data theft, and other subsequent attacks. GitLab’s response was to advise following their Security Incident and Disaster Recovery procedures to revoke the compromised instance and restore the latest clean backup to a new GitLab instance.
Other severe incidents that GitLab users could feel include remote code execution (RCE) vulnerabilities, a social engineering campaign targeting employees’ personal accounts at tech companies, and critical account takeover flaws within GitLab.
Atlassian accounted for one-third of the major impact incidents, with Jira users experiencing disruptions roughly every 7 days.
About one-third of incidents in Bitbucket and 44% of incidents in Jira Atlassian were recognized as the major impact, meaning they might directly affect users. While Bitbucket saw a slight decrease in incidents in 2023 – down by just 2.04%, – Jira users weren’t as fortunate. They experienced a 50% increase in incidents compared to the previous year – 75 events in total, which gives us worrying statistics of one incident every 5 days.
Last year, Atlassian’s struggles were largely related to high-severity vulnerabilities – with many recognizing the CVSS score of 9 and higher. Those security flaws included template injection vulnerability, critical Remote Code Execution (RCE) bugs, etc., which could allow hackers to access and compromise users’ data.
Just take a look at these magazine titles!
However, not only vulnerabilities were trying Atlassian. In February 2023, Atlassian also fell victim to an attack on one of its employees within Envoy’s app. Thus, the malicious gang could access staff information, including names, emails, departments, and Atlassian office floor plans located in San Francisco, USA, and Sydney, Australia.
Stay proactive with in-depth security measures
Understanding that the rapid pace of DevOps, combined with the complexities of modern cyber threats, demands a proactive and informative approach, we’ve covered in our study all the necessary in-depth security measures for protecting users’ DevOps tool data.
Thus, while building your security strategy, you should not only include security testing, such as static application security testing, automated security testing, dynamic application security testing, and interactive application security testing but also you shouldn’t forget about other security processes. Among them, there should be DevOps backup, implementation of software composition analysis, controlled privileged access, application programming interface management, vulnerability management, cloud security, the possibility to automate software provisioning, secure code practices, configuration management, short and frequent development cycles, secrets management, security scanners, etc. Moreover, it should include constant checking for the effectiveness of your DevSecOps approach, for example, using static code analysis.
The final thought
The report also analyzes the most serious incidents of all time, including the infamous Atlassian outage that lasted over 2 weeks and affected over 50K users, the GitLab database incident caused by human error that resulted in the loss of data from over 5,000 projects, and 700 new user accounts, and the ransomware attack and repository wipes that happened to GitHub, GitLab, and Atlassian.
It’s worth also noting here that in recent years, the frequency of cybercrimes targeting popular Git hosting platforms has been on the rise. These attacks range from leaking secrets and stealing tokens or source code to wiping the entire repository (or repositories!) and demanding a ransom. Moreover, there is also an increase in a number of supply-chain attacks that are specifically targeting developers.
Read the full report – The State of DevOps Threats – and stay tuned for the latest risks and dangers in DevOps. Remember, we have your back(up)!
✍️ Subscribe to GitProtect DevSecOps X-Ray Newsletter – your guide to the latest DevOps & security insights
🚀 Ensure compliant DevOps backup and recovery with a 14-day free trial
📅 Let’s discuss your needs and see a live product tour
Top comments (0)