Verifying SafeLine’s Effectiveness: A Comprehensive Evaluation
BlazeHTTP
BlazeHTTP is a straightforward tool designed for evaluating the effectiveness of WAF protection.
- Extensive Sample Set: Currently offers 33,669 samples with regular updates.
- No Configuration Required: Available in both GUI and command-line versions. You can directly download precompiled versions from Releases or clone the repository to compile locally.
- Exportable Reports: Generates detailed reports on the execution of all samples, including attributes, execution time, status codes, interception status, and more.
Testing Metrics
| Metric | Description | Calculation Method |
|---|---|---|
| Detection Rate | Measures how well the WAF detects threats. Missed detections indicate incomplete coverage. | Number of attack sample interceptions |
| False Positive Rate | Assesses the WAF’s impact on legitimate traffic, with false positives indicating incorrect blocking. | Number of normal sample interceptions |
| Accuracy | Combines detection and false positive rates, offering a balanced view of WAF performance. | |
| Detection Timing | Evaluates WAF performance, with longer detection times indicating slower response. |
Sample Instances
- Normal sample:
GET /rc-virtual-list@3.5.2/lib/hooks/useHeights.js HTTP/1.1
Host: npm.staticblitz.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36
Accept: */*
Origin: https://stackblitz.com
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://stackblitz.com/
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7
- Malicious sample:
GET /vulnerabilities/sqli_blind/?id=1%27+or+%27%27%3D%27&Submit=Submit HTTP/1.1
Host: 10.10.3.128
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Referer: http://10.10.3.128/vulnerabilities/sqli_blind/?id=1%27+and+%27%27%3D%27&Submit=Submit
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7
Connection: close
SafeLine vs CloudFlare
| Metric | CloudFlare (Free) | SafeLine (Free, Balance Mode) | SafeLine (Free, Strict Mode) |
|---|---|---|---|
| Total Samples | 33,669 | 33,669 | 33,669 |
| Successful | 33,350 | 33,669 | 33,669 |
| Errors | 319 | 0 | 0 |
| Detection Rate | 10.70%(Total Malicious Samples: 570, Correctly Intercepted: 61, Missed Detections: 509) | 71.65%(Total Malicious Samples: 575, Correctly Intercepted: 412, Missed Detections: 163) | 76.17%(Total Malicious Samples: 575, Correctly Intercepted: 438, Missed Detections: 137) |
| False Positive Rate | 0.07%(Total Normal Samples: 32780, Correctly Passed: 32757, False Positives: 23) | 0.07%(Total Normal Samples: 33094, Correctly Passed: 33071, False Positives: 23) | 0.22%(Total Normal Samples: 33094, Correctly Passed: 33021, False Positives: 73) |
| Accuracy | 98.40%(Correct Interceptions + Correct Passes) / Total Samples | 99.45%(Correct Interceptions + Correct Passes) / Total Samples | 99.38%(Correct Interceptions + Correct Passes) / Total Samples |
| Average Time | 288.96 ms | 70.05 ms | 64.34 ms |
SafeLine vs ModSecurity
| Metric | ModSecurity (PARANOIA 1) | ModSecurity (PARANOIA 4) | SafeLine (Free, Balance Mode) | SafeLine (Free, Strict Mode) |
|---|---|---|---|---|
| Total Samples | 33,669 | 33,669 | 33,669 | 33,669 |
| Successful | 33,669 | 33,669 | 33,669 | 33,669 |
| Errors | 0 | 0 | 0 | 0 |
| Detection Rate | 69.74%(Total Malicious Samples: 575, Correctly Intercepted: 401, Missed Detections: 174) | 94.61%(Total Malicious Samples: 575, Correctly Intercepted: 544, Missed Detections: 31) | 71.65%(Total Malicious Samples: 575, Correctly Intercepted: 412, Missed Detections: 163) | 76.17%(Total Malicious Samples: 575, Correctly Intercepted: 438, Missed Detections: 137) |
| False Positive Rate | 17.58%(Total Normal Samples: 33094, Correctly Passed: 27275, False Positives: 5819) | 52.46%(Total Normal Samples: 33094, Correctly Passed: 15732, False Positives: 17362) | 0.07%(Total Normal Samples: 33094, Correctly Passed: 33071, False Positives: 23) | 0.22%(Total Normal Samples: 33094, Correctly Passed: 33021, False Positives: 73) |
| Accuracy | 82.20%(Correct Interceptions + Correct Passes) / Total Samples | 48.34%(Correct Interceptions + Correct Passes) / Total Samples | 99.45%(Correct Interceptions + Correct Passes) / Total Samples | 99.38%(Correct Interceptions + Correct Passes) / Total Samples |
| Average Time | 31.15 ms | 28.89 ms | 70.05 ms | 64.34 ms |
Top comments (0)