Reducing Downtime Risk with Blue-Green Deployments

Many of the world’s most common and trusted applications- for both personal or business use – are now delivered as SaaS applications, whether on mobile, desktop, or other devices. End users assume these SaaS applications – whether paid for via subscription or invoked free on demand – to be available 24/7/365. Users trust application builders to provide a dependable and uninterrupted service experience. When software isn’t available, customers get frustrated and businesses’ reputations suffer.

Leading content platform and application software companies are increasingly using progressive software deployment strategies such as ‘blue-green’ to reduce the risk of downtime, enhance customer experience, and build brand loyalty. However, in many companies, there still exists the misconception that initiatives to improve quality and security translate into increased investment. Therefore, a business case for making the technology investments required to significantly reduce the mean time between service failures to avoid downtime will likely be required.

The Costs of Downtime

According to a 2019 report from Statista, the average cost of downtime can range from $100,000 to $5,000,000 per hour.

Content customers and software users are working all hours, locally and around the world. Therefore, there is no ‘good’ time to have an outage. They expect and/or require the applications to be available all the time.

A recent Accenture study found poor service quality is the top reason for customer churn. In addition to compromising customer experience, jeopardizing renewals, expansions and upsells, excessive downtime can trigger direct costs if service-level agreements (SLAs) are in place for operational performance. In addition, if a service is delivered to regulated industries like banking, insurance, and stock trading, outages may violate compliance requirements. Said violations can result in fines, penalties, or increased scrutiny from auditors and regulators. Resultant scrutiny is very expensive in terms of time, consuming resources that could be working on revenue generating activities instead.

There is also a sizable intangible cost to organizations that face continuous downtown incidents. Service breakdowns impact employee productivity, morale, and job satisfaction. This often results in higher employee turnover.

Implementing practices to achieve service uptime performance consistent with customer expectations is critical to sustained success. Blue Green deployments are a technique increasingly viewed as an operational best-practice for deploying applications reliably into production and ensuring minimal downtime.

Blue-Green Deployment

During a blue-green deployment, the system shifts traffic routing from the currently-deployed version to a new application version. This practice requires two concurrently-running application instances:

• A current, or “blue,” version of the application
• A new, or “green,” version of the application

Containerization and cloud-native development augment the blue-green deployment model: Kubernetes provides a cost-effective, flexible, and automated process of generating similar application instances, and label-based traffic routing provides the necessary foundation to switch between different application versions quickly.

I work on Armory’s new SaaS-based deployment offering, Project Borealis. We just added blue-green support. In Borealis we deploy the new version, then expose it to a preview or “dark” URL that the user can leverage for any needed validation. Once validated, they can issue an approval, and the new version will become live. The old version is kept around for instantaneous rollbacks until a final approval is issued, or a given amount of time has elapsed.

Application Updates

Let’s say a company has a running application with version V1 – this is the “blue” instance. When it’s time to deploy the next application version, V2, the company deploys without exposing this new application instance to production traffic – this is the “green” instance. Instead of receiving production traffic, it is exposed to a dark URL that can be used for manual or programmatic validation.

The process of deploying and configuring applications can be elaborate. Typically, teams organize the process using steps in runbooks. Execution times vary from a few minutes to a few hours.

The team tests each step without interrupting the blue application, eliminating any downtime. The blue application continues to serve customers until the green application is ready to go.

Next, a QA pipeline runs automated tests to validate the green version. The team can triage any issues without the stress of a ticking clock. Post validation, the DevOps team or an automated tool configures the load balancers to route new requests to the green version. Once all traffic has been redirected to the green application, the blue application can be repurposed, torn down, or kept alive for a quick rollback.

A blue-green deployment in Project Borealis

This process can readily be applied to stateful applications as well. However, if your application has a database, there is additional thinking to be done about how to handle roll forward and backward compatibility.

Application Rollbacks

Blue-green deployments significantly reduce the costs associated with a broken release. If a defect is detected during or after the deployment, the DevOps team can configure the load balancer to direct traffic to the still-running blue environment, as convenient as flipping a switch. Even if your application takes significant time to start, rollback is instant since the old version is still running.

Blue-green deployments are more complicated than downtime deployments. Companies must support these environments with release automation, testing, infrastructure provisioning, configuration management, and the associated CI/CD practices. Because the deployment logic is also significantly more complicated, such strategies are often impractical to adopt without leveraging a production-ready deployment solution such as Project Borealis.

The development teams must also invest the time to make these practices work for all teams that must collaborate to successfully ship software. Often, this requires some tooling standardization. While a custom-made continuous delivery process may work for some teams, it can be challenging to apply it to all teams in an enterprise. It can also be challenging to support and maintain an extensive, homegrown solution. For example, a new engineer may not find it easy to figure out how to use all the tools available for deployment, even with simple changes. Or if a connected tool or service is upgraded, it may require troubleshooting or code rewrites before it can work.

Armory Delivers Continuous Delivery

Armory provides enterprise-grade cloud-native continuous delivery, the next evolution of the open-source Spinnaker platform. With Armory, teams can accelerate the adoption of a blue-green model using:

• Pipeline building based on Spinnaker
• Role-based access control
• Deployment compliance checks using policies
• Infrastructure provisioning using Terraform
• GitOps, secret management, and more

These features enable collaboration between the developer, QA, DevOps, and SecOps teams for building end-to-end deployment pipelines. New deployment pipelines can be executed in different phases like code tagging, application release, tests, security checks, deployment, and smoke tests. This results in a green environment being set up without all the usual manual work.

Armory provides a battle-tested, flexible pipeline management system with integrated CI tools, cloud providers, security policies, and third-party libraries. Pipelines can be fully automated or user interactions can be set up representing complex and customized delivery workflows to control the pipeline. Armory enables the creation of pipelines with stages to deliver software in support of any process or policy an organization requires. Teams can go from fully automated to manual checks at each gate.

In addition to Spinnaker, Armory recently announced a new intelligent deployment-as-a-service offering, Project Borealis, that supports blue-green strategies so that developers can focus on building great code rather than deploying it. My team recently added blue-green deployments to this SaaS offering and is helping our design partners to leverage it. A demo of blue-green in Project Borealis is available. If you’d like to try it out, spots are still available for additional design partners.

Next Steps

The blue-green deployment model can eliminate the risks and toil of application release. To reap the model’s benefits, different enterprise teams must collaborate closely and have a high level of readiness.

Armory provides a foundation for safe, reliable deployments using release and deployment pipelines with compliance checks. Its policy engine and external integrations reduce the deployment model’s overall complexity, enabling development teams to operate at peak efficiency.

Are you ready to reduce your downtime and keep your application users happy while saving deployment time and effort? Explore armory.io to learn how customers use the product to solve these deployment challenges and contact us for a demo and whitepaper to see how Armory can help you. If you’d like to try out Project Borealis, our new intelligent deployment as-a-service solution that makes advanced deployment strategies simple to adopt, you can sign up to learn more and become a design partner.

The post Reducing Downtime Risk with Blue-Green Deployments appeared first on Armory.