This post originally appeared on the Scout blog.
There is no doubt that looking at response times and memory usage is essential to understanding the general health and performance of your application. But as I am sure you are aware, there is more than one way to monitor an application. Approaching monitoring from a different angle can be a powerful way of gaining new insights. If all you did was watch for high response times or areas of memory bloat, then you might overlook something far more simple: the user’s general level of satisfaction. So how can we monitor this rather broad concept of user satisfaction? Well, we can monitor this with a rather useful metric known as the Apdex score...
What is Apdex?
The Application Performance Index, or Apdex, is a measurement of a user’s general level of satisfaction when using an application. For example, if part of the system takes a long time to respond after a feature update, then the user naturally starts to become frustrated. This is what we are trying to measure when we talk about apdex, and in this scenario we would see the apdex rating fall, which would be a clear indicator that a recent change has introduced a potential issue.
There are three levels of customer satisfaction in regards to apdex (summarized in the table above): satisfied, tolerating and frustrated. The satisfied level is when the system response time is equal to or below some arbitrary value which we know the user would be happy with. We call this arbitrary value the Threshold. We can assume that a user will get frustrated with the system when the response time goes up fourfold from this threshold. The tolerating level, then, is when the response time falls between these two scenarios, when the user is not yet frustrated but there is a risk that they will become frustrated if the situation continues or worsens.
If we take all the requests made over a set period of time and divide them up into two groups: satisfied/tolerating users vs. frustrated users, then we come out with a ratio. This ratio is known as the apdex score, and it can clearly show us at any given time how satisfied our customers are. An apdex score of 1 would indicate that all of our customers are fully satisfied, whereas an apdex score of 0 on the other hand, would tell us that none of our customers are happy. The equation shown above shows how Scout calculates the apdex score, you can see that we put less weight on tolerating requests, because we assume that these users are not 100% satisfied in this situation.
Using Scout to measure Apdex
The Apdex score of your application is visible throughout Scout on our main charts. You can toggle this metric on or off at anytime using the APDEX option shown in the image below. You can then try experimenting by combining the Apdex score with another metric in order to identify related patterns.
How to improve your Apdex score?
In order to improve your Apdex score, you first need to be aware of it. So you will need to monitor the situation with an APM solution like Scout. Once you start monitoring the Apdex, you might find that patterns begin to emerge, such as sudden drops after particular deployments (shown by the rocket icon). Or another example might be repeated high spikes in response times during certain time periods rapidly drive down the Apdex, showing you that perhaps you have an infrastructure limitation.
Summary
As you can see Apdex is an interesting way to monitor your applications. It allows you to ask questions like: “Did this feature annoy people?” and ”Have our users been happy with their experience recently?”. These sorts of questions can be hard to answer by other means, and so Apdex is definitely something that we recommend that you keep an eye on.
Top comments (0)