DEV Community

loading...
Cover image for Right Tail Outlier Detector in JMeter

Right Tail Outlier Detector in JMeter

NaveenKumar Namachivayam ⚡
Performance Engineer. I break stuffs 🔨. I know nothing.
Originally published at qainsights.com ・7 min read

In this blog post we are going to see about Right Tail Outlier Detector in JMeter. Analyzing results is one of the critical phase in any performance testing project. Most of the results you encounter might have outliers. In commercial tools such as NeoLoad has a built-in feature to remove the noise. In JMeter, you can isolate the outliers using the Right Tail Outlier Detector plugin.

Installing Right Tail Outlier Detector in JMeter

Prerequisites

  • Stable version of JMeter
  • Plugins Manager

In this demonstration, I am using JMeter 5.4.1 and Plugins Manager 1.6. Download the Plugins Manager jar from here and place it into JMETER_HOME\lib\ext folder.

(Re)Start JMeter to install the Right Tail Outlier Detector plugin.

Launch Plugins Manager by clicking on the icon at the top right.

Search for "Right Tail" under the "Available Plugins" as shown below.

Right Tail Outlier Detector in JMeter
Right Tail Outlier Detector in JMeter

Hit Apply Changes and Restart JMeter. This will download the relevant jars and place it in the appropriate locations and restart JMeter.

Usage

Launch JMeter and add the Right Tail Outlier Detection element which is located under Non-Test Elements as shown below.

Add Right Tail Outlier Detection
Add Right Tail Outlier Detection

You can see the element configuration as shown below.

Right Tail Outlier Detection Configuration
Right Tail Outlier Detection Configuration

What is an Outlier?

From performance test result perspective, an Outlier is a data point which is significantly different from the other data points.

E.g. following are the response time of a page load during the test run. The data point at 17:19:00 has 1.061 seconds which is different from the other data points. Here the outlier is 1.061 seconds.






























Time Stamp Response Time (s)
17:15:00 0.05
17:16:00 0.07
17:17:00 0.06
17:18:00 0.045
17:19:00 1.061
17:20:00 0.046
Sample Data with an outlier

About Right Tail Outlier Detector Plugin

This plugin uses Tukey's technique to find the upper bounds using two options: 1.5 and 3 Tukey's constant k.

k 1.5 will identify all the outliers and k 3 will identify only the extreme values.

Right Tail Outlier Detector Plugin Source Code

<!-- /wp:button -->

To detect the outliers, first you need the performance test results either in jtl, csv or tsv format.

I have the sample test result which I ran against my local app. First let us detect all the outliers using k 1.5.

Select 1.5 (detect all outliers) and browse the results file as shown below and then click on Detect.

Outliers Detection
Outliers Detection

For my test result, I have got 5 Outliers and the Outliers and Trimmed files have been generated in the bin folder.

Outliers and Trimmed Files
Outliers and Trimmed Files

You could discard the Outliers file and consider only the Trimmed file for further analysis. In case if you want to analyze the root cause of potential performance problems, you can consider Outliers files.

You can also see the Upper fence and the Trimmer count in the table view. To save the table data, click on Save Table Data.

k 1.5 Detection
k 1.5 Detection

For k 3, there are no Outliers detected for my test results, as it will detect only the extrement values.

k 3 detection
k 3 detection

Command Line Usage

You can leverage command line to identify the outliers on the fly. Issue the below command to detect the outliers for k 1.5.

java -jar lib\cmdrunner-2.2.jar --tool RightTailOutlierDetector --input-file bin\Run2.jtl --tukey-k 1.5

Command Line Usage
Command Line Usage

Conclusion

Detecting outliers helps you to analyze the performance test results. Outlier detection should be your first step in your checklist. This plugin saves time and helps you to detect in few clicks. Do you use any other method to detect the outliers? Please let me know in the comments.

Discussion (0)