How can feature flagging help you during release and customer experiments?

Shared from Danubius IT Solutions' tech blog.

Have you ever been working in a situation as a project manager, business owner, product owner on a project or in an organization, when a major change you have been waiting for so long was deployed to production related to an application you were responsible for, and what worked perfectly yesterday’s now broken, but you couldn’t intervene? Are you familiar with the terms rollback, while you shouldn’t have to? Perhaps you have developed new functions with IT guys, but when you went live, you couldn’t make sure everything was functioning properly, before opening it to the full audience of customers?

Have you ever done market segmentation as a marketing expert, but it never occurred to you that the app could be used to target specific users, and display differently to some groups than to others? Did you want to try something new, but weren’t sure if it would have the same effect that you expected? Has IT supported A/B testing so far?

As a developer or operator, have you ever not slept the night after a release, because it was impossible to turn off non-operable features, systems were slow to revert to a previous state, or there were interdependencies between pieces of code that only a full revert helped and lost all new features? Have you missed proper testing ever? Has it taken days to do a sweaty merge after months of development on several different branches even after following some well-written strategy like Gitflow?

Feature flagging is not the Holy Grail, but it can be very helpful for all involved stakeholders in the software development life cycle.

What is feature flagging?

There are many definitions out there, but probably the most precise one if we say that feature flagging is an opportunity to change the application’s behavior and feature sets after the code has been deployed to the running environment, e.g. production one. This leads to many other great chances during and after the standard development process.

Main benefits of feature flagging

There are basically 3 (maybe 4) important categories, where this mindset and strategy can bring in great values.

Release management

Probably the most used use case. With the feature flag approach changes can focus on smaller segments of customers – why would you roll out the product to the entire audience after weeks or months of tests, when you can push features to production and receive feedback from only a chunk of people and then iteratively increase the userbase? They can be even more targeted, such as enabling risky functions only for the patient ones at first. If something is not working, one click, and the new features become unavailable. It is easy to manage and no need for any IT involvement to do that. (Further goods here: setting the exact timing and audience of the release, iterative rollout, friends and family, highly decreased release risk, beta/canary release, etc.)

Development and Operation after deployment

It is not enough to have a successful release, systems need to be kept up and running. In most cases feature flagging is easy to implement from a development point of view, operators also like to work with, because even though organizations using feature flagging tend to release more and more frequently over time, this still doesn’t cause more rollbacks from production, quite the contrary: the rollbacks’ number decrease due to the fact, that switching off a feature set does not mean new deploy or code change at all. Even later, if a team responsible for a certain application finds, that some elements of the infrastructure or a component of a software stack is not functioning, which causes performance issues, they have a chance to intervene immediately and switch off high resource–consuming features. (We do not cover topics like software architecture design and high availability - hereinafter HA - here, obviously feature flagging cannot replace all these steps later if they were skipped earlier. Further goods here: tweaking on-the-go, direct access from business PoV to operation (need to handle with care), no rollback, etc.)

Experimenting with real customers

Marketing or Sales can work with feature flags to get more insights about their customers built on top of the data company already owns. Let’s imagine they want to show a different advertisement picture to people above and under 30 years, but they do not know how long this promotion or the differentiation will last. With feature flags, the problem can be solved easily, and after development and deployment, no IT assistance need to change the features, targets, scopes, etc.

“Functionalities”

Some features can be available only for a smaller set of customers by design, and feature flag SaaS solutions can be part of the design. It is not recommended, but a real option.

Possible threats

Security

Never publish a possibly vulnerable, not properly tested feature to production even with a switched-off flag, because it is very easy to miss-click, allow it accidentally, and provide an easy-to-break honeypot and optional attack vectors to possible attackers.

Consistency

While more and more features go to production with the approach mentioned, later all these can behave as a technical debt, which has a price. Do not forget about it.

On the other hand, a feature flagging strategy is good to have at the start, because later it can be difficult to introduce. Consider defining processes, controlling access of features, audit of switching on and off, maintaining documentation, guides and setting expectations from feature flagging and measure to have a chance to improve.

Availability

What happens if the feature flag service is not available? Answer these questions and introduce local cache, especially if you use a SaaS flagging system, but it can be beneficial also if you have your flagging application because this way you do not need to build or buy a HA infrastructure for your internal feature flagging solution. Latency and tolerating failures are also worth thinking about.

Cost

In general, it is a cheap solution compared to other IT techs, but it depends on how you develop the environment: do you ask internal devs to provide an internal feature flagging system, or pay for a SaaS? Whichever it is, usually the cost/income ratio is quite good.

Where to start

We will use ConfigCat in the example, but any other SaaS provider will work in a very similar way. First of all, register on the webpage and set our first feature.

To connect the API to our local code, follow the steps described in the FAQ. As we use Python right now, install configcat-client.

pip install configcat-client

Use your favourite IDE, and try it out.

If this works, the good old if-else statements do the rest of the magic, and marketing specialists, business owners can switch features on and off from now on with a single click.

Example opportunities

Iterative rollout, A/B testing

Start with like 5% of the customers, and if everything seems fine, increase again and again.

Geolocation based appearance

Feature 4 is so great, and only Germany can reach it.

Some providers on the market (SaaS too)

• ConfigCat (we use this one)
• Featureflags.io (Opensource)
• LaunchDarkly
• Vwo
• Cloudbees
• Hackle
• Split.io

Summary

Feature flagging is a key element of giant tech companies and using it may help your business and team grow, transform to a fast-paced environment, where release, testing and experimenting are not fearful words, but something you are familiar with from the daily routine.

At Danubius, we used the approach in different domains (education, banking, and many local product development), and in different countries (Belgium, The Netherlands, Hungary, UK, etc.), feature flagging was a great help for every projects and every customers.