Sometimes we found ourselves driven by the hype of a new feature or product. This might lead us to make decisions in a hurry, which usually is not a good thing, guts are good, but numbers and research are better. We've talked about some qualitative methods to evaluate some product fit, before by using personas.
But in some cases, just qualitative information is not enough and you will require quantitative data. There are plenty of ways and techniques to get quantitative data about a product or feature before releasing, in this article, we'll talk about a Smoke Test or fake landing pages.
What is a Smoke Test?
You create a new page that describes your new feature or product you are planning to build, and then you ask your visitors their level of interest by adding some call to action to buy/use, but instead of leading them into the product you display a "coming soon" page thanking the user for their time and asks for contact information to keep them in the releasing loop.
What's the main metric you'll get from this test? The conversion from a visitor to a user interested in your new product/feature. You can also measure the people leaving their contact details as highly intent people. But you should not centre just in them, even the curious people are people who will convert from visitor to lead.
You can also add multiple levels to your smoke test. But you should take in consideration that you'll add a frustration point at the end by not really offering the product, so you should keep it as simple as possible. Probably not more than 2 or 3 clicks before the "coming soon" page.
A new idea arrived at the company
To make it simple let's take the Shoes Selling App from this previous post. Someone arrived on Monday with a new idea, adding a subscription to receive new shoes periodically. This new feature might generate big income to the company but it also requires a lot of work from multiple departments to be realized.
After analyzing our personas and making some qualitative analysis it seems to fit into our current user-base. But to be sure we'd also like to have some quantitative analysis that validates the proposal, we want to reduce the risks of spending lot of time and effort developing as a company a new feature that doesn't really fit in the market.
Smoke Test time
To get the user intent we'll design a Smoke Test, this would consist of a simple landing page explaining the value proposition. Basically, it's a paid subscription to get new shoes every three months, the shoes will be chosen by our fashion specialists based on your previous orders and your profile so you don't need to care about buying new shoes.
Another important factor is: should we offer different plans? It means adding different categories as Basic, Plus, and Premium. Each category will have different prices which will affect the amount/kind of shoe you'll receive. All these things can be hard to decide just based on qualitative data.
Designing our Smoke Test
It's going to be a two layers test. First of all, we'll have a landing page explaining our new service, our main call to action is to subscribe to this service. From this first layer, we'll get the fit into our visitors based on interest, this is important as it can validate that the feature is a necessity in our visitors' life.
As a second layer, we'll add the different tiers. Here we want to know how much a user interested in this service is ready to spend, this might help our finance prediction to determine the profitability of the new feature. It might be a nice feature but maybe it's not worth it for our company offering this new feature.
Running our Smoke Test and getting results
Running it is the simplest part, you just need to use low resources to set these landing pages, some data tracking, and you'll get all you need to retrieve information to analyze.
After doing this you'll get your results, let's say that from every 2000 visitors we had 1000 clicks on the call to action from the first landing page, that means that 50% of our visitors showed interest in getting this new feature. After adding this information to your qualitative analysis you can make the decision of building this new feature…or not.
Final thoughts
It's important to validate features or products before investing time on developing them. Sometimes just a good idea is not enough, you need to validate your new features or products before spending a lot of time in development and releasing them.
Also, it's a good idea to add some quantitative data to your qualitative data, doing it you can decide easier if it's worth it to materialise the idea. You are also minimizing the risks to fail.
Top comments (0)