Tracking Customer Usage in Your Software - Getting Started

You're thinking of changing a feature or adding a new one to your app, but you don't know how important it is to implement it.

Does anyone need it?

Does anyone use something similar already?

Are you focusing on the right aspects of your app, or are there areas that are being neglected without your knowledge?

For me, one of the most frustrating aspects of trying to develop and prioritise my roadmap is knowing what to focus on. I'm in a fortunate position in that I get to inform what features should be going onto the roadmap and which bugs should be prioritised as well as being able to write a lot of the code that builds those features or fixes bugs. This is all great, but there's nothing worse than when I am halfway through a project when I start to doubt its validity. When I eventually ship, what if nobody uses what I've written? It would be fair to say that I have wasted a considerable amount of time developing something that might be pretty cool, but ultimately redundant.

With SaaS applications or E-Commerce sites, this data is usually available from the beginning. The cloud model is more conducive to tracking the behaviour of the people who grace us with their presence. When it comes to desktop or even mobile apps, it's less likely that we have mechanisms in place to collect usage data from our users.

It became too tempting for me to ignore. I know people are out there and are getting something valuable from my code but I don't know what specifically they see the value in. I have to build a metrics system.

The Serverless Approach to Gathering Data

I built a metrics system for tracking customer usage in a desktop app and I can honestly say that it has changed the way I manage the application roadmap dramatically.

Here is a basic architectural overview of how I used Microsoft Azure as an environment for gathering usage data.

Fishing for raw data

To start with, you're going to want to know which aspects of application usage you are most interested in tracking. In my case, I'm responsible for an app which provides many varying types of automation. One feature might automate inserting records into a spreadsheet, while another might post to Twitter. It's an application which holds an eclectic array of features. As it stands, I know from experience talking to some customers which are the most utilised features. With some features though, I'm not sure if anyone uses them.

To begin with, identify the main groups of data you need. If your app is a CRM, you might want to track how many people are using a specific filter on their list of Sales Leads. If your app is an FTP server, you might want to see which file format is most commonly uploaded before you decide to stop supporting it. In order to reach the point where this data is available for you to post to Azure, you need to capture it.

Event-Based Approaches

There are two main types of data domains to track. Events that happen in real-time and metadata. For tracking things as they happen, an event-driven model works pretty well. The two domains have one thing in common however, in that they will most likely be persisted as something worth tracking. A metric.

Say I'm using MVC, and one of my controller classes (we'll call it 'InvoiceController') controls the flow of data from a pre-defined invoice model to our front-end. The controller may have events that are fired from the click of a button. This would be a perfect place to track the click of a button which approves an invoice for payment.

I find it useful to use interfaces to describe specific attributes for objects. This way we can implement specific interfaces on classes containing things we want to track to say that they can be tracked and what needs to be implemented on that class for that to happen. I'm a .NET guy primarily, so I'm going to use an example in C#. Let's create an interface called ITrackable. This will describe what needs to be implemented in a trackable class in order to track something. (This is only a very basic example for the sake of simplicity. It can become way more verbose as needed)

interface ITrackable
    {
        void SaveToDB(Metric metric);
        void CreateMetric(object sender);
    }

Wherever we implement ITrackable, we have to include two methods. One to create our metric, sending the actual object that was caught in a click event, and then SaveToDB, which will evidently serialize our metric data. Speaking of metric data, what would that be? Well again, I want to give a simple example, so I'll create a very basic class which can hold information about something we are tracking.

class Metric
    {
        string Name;
        DateTime CapturedAt;

        public Metric() { }

        public Metric(string metricName)
        {
            Name = metricName;
            CapturedAt = DateTime.Now;
        }
    }

So with the Metric class, we have a Name property, which is just the name of the 'thing that was clicked,' (We'll get this from the object sender on the original click event.), and then we have a DateTime in CapturedAt, which is just set to the current date and time when Metric is constructed.

This means that we have a way to grab the object sender from a click event, get its name and name a Metric object after it, noting the time the event was raised and then calling a method to save this data to a DB.

Simple right? Now let's make one our InvoiceController trackable.

 class InvoiceController : ITrackable
    {
        public void SaveToDB(Metric metric)
        {
            //save to DB logic here.
        }
       
        public void CreateMetric(object sender)
        {
            var objectName = sender.GetType().Name;
            var metric = new Metric(objectName);
            SaveToDB(metric);
        }
        

        btnInvoiceApprove_Click(object sender, EventArgs e)
        {
            CreateMetric(sender);
        }

From the top then, we have our InvoiceController, which before we did anything to do with metrics had various click events from the front-end (namely our btnInvoiceApprove_Click event which, you guessed it, gets fired when we click a button to approve an invoice) now implements our ITrackable interface, meaning that it is required to include CreateMetric and SaveToDB.

You've probably noticed that we have called one of our ITrackable interface methods on the click event. This is kind of like raising another event. We pass the sender argument for the click event to CreateMetric, which in turn uses GetType().Name to get the name of the object as a string. In this case the name may be 'btnInvoiceApprove' or 'ApprovalBtn' but either way we will recognise this as a button that was pressed.

Following this, we create a new metric, named after our button and set to the current date and time, before calling our SaveToDB method to persist the data.

Congratulations! It might not be super clear what you actually tracked, and it might be a teeny weeny bit generic, but we have a starting point for tracking something in realtime and then saving it somewhere for use later.

The other domain we talked about was metadata, which is usually a snapshot of the environment at a given time. For example, you might have something that says 'at the same time every day / or if I am open at a certain time, create metrics objects for x and then save them to the DB.' If we use the example of a CRM app again, it might be that this timed event goes to the DB and checks how many leads are saved at a given time and how many were changed.

Once your data, whether it be realtime events or meta has been saved somewhere, you need to be able to view it. At this point however, it's still on our intrepid user's machine. How do we get it to it? Well, we don't. We make it come to us. I did this using the serverless approach with Azure Functions.

Similar to the timed approach of capturing metadata at a set regular time, I then package that data into a format like JSON or XML and call my Azure Function's public endpoint from the app, posting the data to it. The function exists simply to parse the data I send and then put it into cloud storage for me to render in an app on my side.

Always be Open with your Users about Tracking, even if it's Anonymous.

Even if you take a totally different approach to tracking, it's extremely important to remember that you need to provide your users with a manner of consenting to have their usage tracked.

You may decide that you don't want to capture any data that contains personal information, just behaviour within the UI. You still need to provide a way for the user to opt-out. Trust me, no amount of data is valuable enough that it is worth compromising someone's privacy not to mention making you liable to legal action. Make it clear what you are doing and make it easy to turn tracking off.

Getting the Data into the Cloud.

I'll go into more details on how we can ship analytics data to the cloud for analysis in another post. Depending on your app and how you wish to track and analyse its usage, your metrics system could get quite complex. Needless to say, if you don't currently engage much with the people who use your product, it will certainly provide you with a new means of planning product roadmaps for the better.