How To Monitor Http traffic Of Third Party Service

Why

The company purchased a third-party service and deployed it on the company's machine in the form of a jar package.This service is on a critical data link, and the concurrency is very high. The service http request appears 500, which is what we cannot receive.

Even without the source code, we also need to monitor HTTP requests to provide detailed information to the provider.

Ideas to Solve the Problem

My first thought was to use java agent to monitor.Java agents are a special type of class which, by using the Java Instrumentation API, can intercept applications running on the JVM, modifying their bytecode.

Another solution is to capture network traffic. My core requirement is to monitor requests with a status code of 5xx, so capturing traffic is more suitable.

Capture network traffic

Tcpdump can capture traffic but needs parsing to be more suitable for reading.

httpry is a tool designed for displaying and logging HTTP traffic.

Nice!This is the tool I was looking for.

Install httpry on Centos

sudo yum install epel-release

sudo yum install httpry

Filter Response with a Status Code other than 200

sudo httpry -m post -f timestamp,source-ip,dest-ip,direction,status-code -o test.log 'tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2)+8:4] != 0x20323030'

The output in test.log like this:

2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       403
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       403
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500

Manager Large Data Volume Output File

All the information is typed into one file, this file will be very large.

You need another tool:logrotate.

I want to organize the files by date and I create the file logrotate.conf

/xxxx/xxxxx/http.log {
    notifempty
        copytruncate
        dateext
        dateformat .%Y%m%d
        olddir /xxx/backup
}

The following command can debug the process

logrotate -d xxx/lograte.conf

Then add timed tasks through ctontab，crontab will execute the rotate according to the config and We will get a file named http.log.20220815.

More Resources

httpry
logrotate