DEV Community

JMeter, Prometheus, and Grafana Integration

NaveenKumar Namachivayam ⚡ on March 05, 2020

This post originally published at QAInsights Blog. You can check out my videos about LoadRunner integration with Grafana in my YouTube channel. Re...
Collapse
 
satya1119 profile image
satya1119

How to integrate Jmeter with promotheus+Graphana and my main requirement is how to fetch the JMeter metrics using external IP address because here i am able to see the JMeter metrics using local host 9270 port but i am not able to see the metrics using IP address of that VMmachine (linux),Could you please help me out with your suggestions

Collapse
 
qainsights profile image
NaveenKumar Namachivayam ⚡

I think you need to open the port in your VM, so that Prometheus can scrape. Try to configure the IP as 0.0.0.0 in your JMeter properties prometheus.ip=0.0.0.0

Collapse
 
satya1119 profile image
satya1119 • Edited

Hi Naveen,
Thank you for your quick reply, Apologies for the delay.
I configured the prometheus.ip = 0.0.0.0 in jmeter properties file in VM machine, even though I am facing the connection refused error. Please find the attached screenshot FYR.
Could you please help me out.

Thread Thread
 
qainsights profile image
NaveenKumar Namachivayam ⚡

are you able to access the Prometheus URL ? yourip:9270/metrics

Thread Thread
 
satya1119 profile image
satya1119

No, i am not able to access the Prometheus URL with
ip Address (curl -q -6 -g http://[240b:c0e0:101:54e2::5b]:9270/metrics) but when when i am trying with local host in (curl -q -6 -g localhost:9270/metrics) it is working as expected

Thread Thread
 
satya1119 profile image
satya1119

dev-to-uploads.s3.amazonaws.com/i/...

Please find the error screen shot

Thread Thread
 
qainsights profile image
NaveenKumar Namachivayam ⚡

Why there is a square bracket in your URL?

Thread Thread
 
qainsights profile image
NaveenKumar Namachivayam ⚡

Where are those properties stored? It should be in the user.properties.

Thread Thread
 
satya1119 profile image
satya1119

with out square brace's i am facing this error

curl -q -6 -g 240b:c0e0:101:54e2::5b:9270/metrics
curl: (3) IPv6 numerical address used in URL without brackets error
[2]+ Exit 1

Thread Thread
 
qainsights profile image
NaveenKumar Namachivayam ⚡

Not sure about the property remote hosts. Probably you can remove and try only with the Prometheus IP.

Thread Thread
 
satya1119 profile image
satya1119 • Edited

as per your comment i removed the remote hosts and tried even though facing the connection refused error
dev-to-uploads.s3.amazonaws.com/i/...

Where are those properties stored? It should be in the user.properties. ---- it means????

Thread Thread
 
satya1119 profile image
satya1119

can you please let me know if any other way to sort out this issue
Thanks

Thread Thread
 
nityasantoshi profile image
SantoshiDevata

I am having the same issue, when I try to use external IP? Do you have any pointers for me?

Thread Thread
 
qainsights profile image
NaveenKumar Namachivayam ⚡

Sorry, I do not have a Docker environment to reproduce the issue. Can you please raise the issue in GitHub?

Thread Thread
 
nityasantoshi profile image
SantoshiDevata • Edited

I solved this issue. I can provide you details if you want

Thread Thread
 
qainsights profile image
NaveenKumar Namachivayam ⚡

Great, please do share, let me add it to my blog and quote you :)

Thread Thread
 
nityasantoshi profile image
SantoshiDevata

Sure I will share the solution over the weekend

Thread Thread
 
qainsights profile image
NaveenKumar Namachivayam ⚡

Many thanks :)

Thread Thread
 
nityasantoshi profile image
SantoshiDevata • Edited
  • Add the prometheus.ip=0.0.0.0 in user.properties file of JMeter at apache-jmeter/bin

  • If you want to run Prometheus on a different machine, use the attached Prometheus_jmeter.yml in the VM(assume VM ip address is XX.XX.XX.XXX) to run the Prometheus.

Run below docker command to get the Prometheus image from docker hub and volume the Prometheus_jmeter.yml file in VM to Prometheus container.

     docker run  --net=host  -d  /Prometheus_jmeter.yml:/etc/prometheus/prometheus.yml  prom/Prometheus

-On local machine, run the JMeter file which has Prometheus listener in it.
Do the Port Forwarding from your local machine JMeter port to remote machine using below command
ssh -R 9270:localhost:9270 usernameofVM@XX.XX.XX.XXX

  • In your remote machine (VM) run the below command to verify that the port forwarding is working. You can to see all the Prometheus metrics show up in remote machine.
    curl -v localhost:9270

  • You can checks all the targets by using this XX.XX.XX.XXX:9090/targets

Yml file:

scrape_configs:

  • job_name: 'prometheus' static_configs:
    • targets: ['localhost:9090']
  • job_name: 'jmeter' static_configs:
    • targets: ['localhost:9270']

If you setup the Prometheus server at one place. You can run the Jmeter test from any where if you use the Remote port forward and with adding the "--net=host" while getting the prometheus docker image.

Let me know if you have any questions. you can reach out to me for more details at santoshi.nitya@gmail.com

Collapse
 
sandeepk1646 profile image
Sandeep

HI, It's very good information.

I have configured as per the steps that you mentioned above. Here are couple of issues.
1) Metrics are showing in 9090 port instead of 9270 port.
2) After running my jmeter script i don't see the result in metrics.

Can you please help me on this?

Collapse
 
qainsights profile image
NaveenKumar Namachivayam ⚡

Are you using the latest version of Prometheus plugin?

Collapse
 
dhanyanayak24 profile image
dhanyanayak24

hi Naveen,

I am following this thread to integrate my jmeter test result with prometheus. I did all the setups as suggested in this thread but i am not able to see the data in localhost:9270

I placed the jar file under jmeter and then added the yaml file as suggested in my local n started prometheus in my docker.

But i am unable to get any input in the jmeter url.Attaching the image.Could you please help me.My prometheus url is up localhost:9090/ but i am not seeing any jmeter parameters there to execute.

Thread Thread
 
qainsights profile image
NaveenKumar Namachivayam ⚡

Please check the log from prometheus to debug.

Collapse
 
amarnath_n_58856c9590f4e1 profile image
Amarnath N

HI Naveen - I have a very basic setup of jmeter where i have 1 master machine and 4 slave machines, I tried running jmeter on master machine with prometheus listener, it working fine not issue able to see metrics both in http://:9270/metrics and in Grafana as well, but when I started running distrubuted testing (tried to run a test on all 4 slave machines) that is when I am noticing issues the test is running fine but unable load http://:9270/metrics or unable to see metrics. I am not using docker, its a very basic setup and 5 windows machines 1 master and 4 slave machine. I tried adding prometheus portand ip address in jmeter properties of slave machines but no luck.

Collapse
 
leibson profile image
leibson

Great information; thank you!
I really like the idea of pulling server resource metrics with Prometheus.

I want to run JMeter in headless mode for better performance. The JMeter people all say that it is bad to run JMeter with GUI components in a live load test.

How do you recommend using this component when running in headless mode?

It would seem that the right way is to just view the data in Grafana.

Thanks again.

Collapse
 
qainsights profile image
NaveenKumar Namachivayam ⚡

Thanks. You can launch your test using jmeter -n -t <test.jmx> ..... for headless mode.