Testing in Production with Canary Deployments: A How-To Guide

Testing in production may not seem right, does it? 🤔

With feature flags, it’s possible. So what are feature flags?

Feature flags enable developers to enable or disable specific features within an application or service without modifying the codebase. It's like turning a switch on and off.

But why would anyone want to test in production?

Sometimes, we want to roll out a new feature and have some initial users try it out or provide feedback. For example, imagine you're introducing a chat feature in your app. Instead of releasing it to all users at once, you may want only 5% of your total users to see that feature initially, so that you can verify whether it's working fine or not.

In software development, there are various ways to test in production; we will take an example of canary deployment.

What is canary deployment?

In simple words, canary deployment is a way to release and test new software by trying it out on a small group of users before deploying it to everyone. It's the same thing that I mentioned in the chat example above. The benefits of this include easy rollback, A/B testing, etc.

Still confused? How can I set up canary deployments??

Glad you asked. Canary deployment, A/B testing, etc., processes were always limited to DevOps personnel. Now, tools like Flagsmith give this power to software engineers who can do it themselves with the clever use of feature flags.

Flagsmith is an open-source tool that lets you easily create and manage feature flags across web, mobile, and server-side applications. Through Flagsmith, you can wrap features in a flag and then toggle them on or off for different environments, users, or user segments.

With feature flags or multivariate flags set up, we can show 5% of users the chat feature, as we saw in the above example.

So now you don't need to run those creating pipelines and branching strategies. You can do all the testing with a clean and simple UI and let Flagsmith handle everything for you. Just go into an interface to change the percentage values—so you can set up and change your canary size without needing to redeploy.

How to set up a canary deployment

Let's take a sneak peek at how you can easily set up canary deployment with Flagsmith.

Once you set up an account and create a project, click on the Features button on the right panel and then click on the "Create Your First Feature" button.

To create a feature flag, you have to provide some information. For example, here we are creating a chat feature. The ID is essential; it will help to identify and get the value when we access this flag in our code.

The control value can be anything like a font size or footer size. In this specific use case, we will treat it as a percentage of how many per cent of the audience can access that.

Variations are pre-defined values; here we are setting how many people will have access to that chat option. After filling in all things, click on the Create Feature. So, here only 10% of people will be able to see the chat menu.

Now click Test Again to see if this is working or not. We can see we get 90. If you're wondering why Variant weightings are not viewable, it's because we have to pass in an Identity while using the SDK.

To check, let's change values from 90 to 85% and 5 to 10%. So now, 10% of the total users will now be able to see the chat feature.

Let's click on the test button to verify. It's now changed to 85%.

Now you can use SDKs to access the flag value and state. Flagsmith supports a wide range of SDKs. After configuration, our canary deployment is live, and 15% of users will experience the chat popup. We can analyze its performance and make adjustments to percentage values without the need for redeployment.

To sum it up, Flagsmith makes testing in production easier. With Flagsmith, developers have the power to test and improve their software, making the whole process smoother and more manageable. It's a handy tool for developers to make their work better!

It’s also open source! Check out the repo here and don't forget to give it a star to show your support and join their amazing community to get technical support.