API providers need to understand how their consumers are using their APIs. Usage metrics are essential because they tell you about API adoption, how your API is growing over time, and which endpoints are seeing more (or less) use. When you look at API usage metrics, you should be measuring the active users on your API in the sense that most closely aligns with your service. This informs where your organization allocates compute resources, which API endpoints you decide to develop, and which API endpoints you document.
What is an active user? At first glance it may seem like simply tracking the number of users using an endpoint. But API users can’t be tracked in the same way as website users. Instead, you need to look at how your users access your API in the context of your product.
What does an active user on an API look like? An active API user is any user that regularly makes calls to your API to achieve a real outcome. This doesn’t mean that they have to constantly make API calls to be considered active. The frequency with which an API consumer makes API calls depends on the context of your API. An active user might make many API calls everyday, if that’s what they need to gain value from your API, or they may only need to make API calls once a week if all they need from your API is weekly data processing. One common practice is to divide your API usage metrics into daily active users and monthly active users to get a clearer picture of how often your users are active over different time frames of interest.
While it can be tempting to take a one size fits all approach to measuring active API users, you need to look at the context of your API when you decide what to measure. Your metrics should reflect how your API creates value for users in the real world. For example, an API that allows you to send SMS messages should track the number of SMS messages a user sends. An API that compresses video would track the size of the data files compressed and sent for each user. The metrics should directly relate to the value of the service being provided.
Bad Metric, Good Metric
While it’s common for API providers to measure active users by tracking the number of unique users, this is a vanity metric that makes things look good without really conveying much information. Since the number of unique users will usually increase over time, it’s tempting to point to metrics like this because it can tell a positive story to internal stakeholders. Unfortunately, a simple count of unique users active is a bad metric because it doesn’t give you details about what your users are doing with your API. It’s not too different from counting the number of window shoppers if you’re a department store. Interesting, but it does not tell you how many came in to buy something, what they bought, or how much they spent. Correspondingly, you want to know if users go on to use your API in a meaningful way, how much value is being created for your API consumers, and which API endpoints and functionality are most valuable to your users. Without meaningful information about your API consumers, your organization can’t decide where to apply new development or how to allocate resources.
Scaling your API metrics as your user base grows is important, but it's often overlooked. When you have a small number of API consumers and a small number of usage patterns, measuring active users can be straightforward. But, as your API gains complexity, it becomes more challenging to accurately measure meaningful user activity. You need a forward-looking metrics strategy that develops alongside your API as you grow. This has the added benefit of making sure that your organization doesn’t leave old users behind as your API grows. Metrics like the unique number of API keys your organization has created, or the total number of calls made to your API endpoints, don’t always indicate what they originally measured. A single developer may create multiple API keys, for example.
Let’s examine what good metrics look like. Good metrics should give you meaningful information about how your API consumers interact with your API. Your metrics should be able to answer questions like:
- How much data is delivered in your API responses? Is it all useful?
- How long does each user remain connected to your API?
- Which endpoints are getting the most activity?
- How successful are users when making API calls?
Good metrics tell you more than just how many developers have used your API. The questions above are different ways to look at how engaged your API consumers are with your API. If your API is focused on transferring data between different devices, then the volume of data transferred is a great indicator of how engaged your users are. On the other hand, if your API provides a real-time service, like live captions for video content, the length of the connection would be a better engagement metric. Learn more about how Moesif can help you track engagement metrics.
API Active Users, For Real
A good starting point for good metrics is aggregating your API calls into meaningful events, instead of looking at the raw number of API calls, which may be a noisy signal. An event can be a more detailed look at an API call, combining an API call and its outcome.
Measuring events instead of API calls also allows you to look at how different types of events compare, including what they have in common. Looking at failed events can reveal common sources of failure. If your users encounter failed events because there’s too much traffic going to your API endpoints, for example, your organization can handle this by allocating more resources to your API. Alternatively, a pattern of failed events associated with your developer onboarding may point to trouble your users have getting started with your API. This may point to a need for more comprehensive documentation, or your users may need additional support to integrate your API.
Let’s look at some real-world examples of API event metrics. An API that performs two-factor verification would have at least four basic API events. A successful send verification SMS would be the first API event, paired with a failed to send SMS event. The other two events are when a verification code is successfully validated, and when the verification code fails to be validated. These events taken together show how often API consumers succeed at using the API to achieve their goal (in this example, verifying a phone number). Distinguishing between successful and failed API interactions by bucketing them into events makes it easy to understand the real activity happening with your API.
Best Practices
While we’ve discussed how measuring active API users with events can better inform how your API consumers are using your API, there are additional benefits. For example, you can use your active API usage metrics to bill your users based directly on the value your service creates for them. Your organization can bill for successful events, thereby tying billing directly to real-world outcomes. For example, an API that makes phone calls can bill based on time spent on a call, and an API that sends text messages can bill for the number of SMS messages sent.
Real-time reporting is critical when relying on active user metrics. There are many valuable insights to gain from measuring your users while they use your API. For example, you can spot failed events as soon as your users encounter them, making it possible for your team to address failures in real-time. Users are less likely to leave your service if they receive timely support for their issues. Real-time analysis also allows your organization to look at how the number of successful events compares to the number of failed events at a given moment, such as during times of heavy load. With real-time event metrics, your team can rapidly respond to your user's needs, monitor failed events, and see which endpoints provide the most value to your users. Build better API products with deep insights powered by Moesif.
Success in Real-time
Measuring the total number of API users can be misleading because it doesn’t give you contextual information about how your API consumers gain value from your API. Batching your API calls into events will give your organization informative metrics at scale for your API. You’ll be able to evaluate failed events so that your organization can understand which of your API endpoints needs more attention and resources. You’ll also be able to compare patterns of successful events so that your organization can understand which endpoints are creating the most value for your API consumers.
To really understand your users, focus your metrics on what they use your API for and whether they are truly achieving their desired outcomes. Your metrics should focus on the API events that create the most value for your users so that you understand where to focus your resources. Understand your users by measuring their activity in real-time, giving yourself the best opportunities to address their needs. If you’ve been measuring API active users wrong all this time, now is the time to rethink your strategy. Your internal stakeholders and your customers alike will thank you for it.
This article was originally written for the Moesif blog, by Matt Tanner, Head of Developer Relations at Moesif.
Top comments (0)