This is a follow up to my earlier article: Concept: High Performance web server w/ Promises.
I decided to follow this up for two main reasons. First and foremost, there was an error in the original code I posted and I couldn't remember if that error existed in the code I actually tested with or if it was introduced during my writing of the article. Second, I didn't have any of my detailed data to share and I feel that is relevant to this type of article.
The Concept
The TL;DR of the concept is: Can we get higher HTTP server throughput (higher RPS) by using a promise even when not needed. That is, no async operation exists.
Please see the first article for more details.
Methodology
I recreated my project but this time I saved it as open source. It's a basic WebStorm express template with handlebars as the view manager. I've modified it to introduce my concept and that looks like:
const myData = { title: 'Promise Express' };
/* GET /promise/ */
router.get('/', function(req, res) {
new Promise(function(resolve, reject) {
resolve(myData); //setup data
}).then(function(d) {
res.render('index', d); //render the view in the 'then'
});
});
I used Artillery for the load testing aspect. Artillery was run from my desktop computer (not on the server running the express project). Using Artillery, we create the following scenarios:
- 10 users each making 20 GET requests
- 50 users each making 30 GET requests
- 100 users each making 30 GET requests
- 500 users each making 100 GET requests
The exact server details are: Ubuntu 16.04 LTS DigitalOcean Droplets, Flexible Droplets - 2 GB 2x vCPUs ($15 a month).
Note: I also did one run on a 1 GB 1x vCPU droplet as well which is documented at the very bottom.
Summary of Differences
There are a few technical differences from my initial article and this version aside from the depth of content. ( Format : first article vs this article)
- Promise library: Pinkie vs Bluebird)
- The server: 1x CPU, 1 GB VPS vs 2x CPU, 2 GiB VPS
- I did check the prev. config though without getting errors.
- Handlebars: No view template engine vs Handlebars
Results
The results are different than my first article. Basically, it doesn't help but it doesn't hurt...
No scenario threw any errors (which was different from my first run as I recall.) I've made the assumption that this has to do with my promise library of choice; bluebird is known and written to be very fast.
Raw Results
Following are the "cut and paste" raw results from Artillery for detailed reference and review. Also, in case I mistyped something into a chart or table.
2x vCPU, 2 GiB RAM
Basic - 10 "Virtual Users", 20 GETs
λ artillery quick --count 10 -n 20 http://159.65.99.177:3000/basic
Started phase 0, duration: 1s @ 15:00:41(-0600) 2018-03-06
Report @ 15:00:44(-0600) 2018-03-06
Scenarios launched: 10
Scenarios completed: 10
Requests completed: 200
RPS sent: 68.49
Request latency:
min: 64.3
max: 139.3
median: 71.8
p95: 119.8
p99: 137.9
Codes:
200: 200
All virtual users finished
Summary report @ 15:00:44(-0600) 2018-03-06
Scenarios launched: 10
Scenarios completed: 10
Requests completed: 200
RPS sent: 68.03
Request latency:
min: 64.3
max: 139.3
median: 71.8
p95: 119.8
p99: 137.9
Scenario counts:
0: 10 (100%)
Codes:
200: 200
Promise - 10 "Virtual Users", 20 GETs
λ artillery quick --count 10 -n 20 http://159.65.99.177:3000/promise
Started phase 0, duration: 1s @ 15:01:44(-0600) 2018-03-06
Report @ 15:01:47(-0600) 2018-03-06
Scenarios launched: 10
Scenarios completed: 10
Requests completed: 200
RPS sent: 82.64
Request latency:
min: 62.3
max: 144.8
median: 71.1
p95: 116.5
p99: 140.7
Codes:
200: 200
All virtual users finished
Summary report @ 15:01:47(-0600) 2018-03-06
Scenarios launched: 10
Scenarios completed: 10
Requests completed: 200
RPS sent: 82.3
Request latency:
min: 62.3
max: 144.8
median: 71.1
p95: 116.5
p99: 140.7
Scenario counts:
0: 10 (100%)
Codes:
200: 200
Basic - 50 "Virtual Users", 30 GETs
λ artillery quick --count 50 -n 30 http://159.65.99.177:3000/basic
Started phase 0, duration: 1s @ 15:06:18(-0600) 2018-03-06
Report @ 15:06:22(-0600) 2018-03-06
Scenarios launched: 50
Scenarios completed: 50
Requests completed: 1500
RPS sent: 373.13
Request latency:
min: 63
max: 376.4
median: 84.5
p95: 130.4
p99: 149.2
Codes:
200: 1500
All virtual users finished
Summary report @ 15:06:22(-0600) 2018-03-06
Scenarios launched: 50
Scenarios completed: 50
Requests completed: 1500
RPS sent: 372.21
Request latency:
min: 63
max: 376.4
median: 84.5
p95: 130.4
p99: 149.2
Scenario counts:
0: 50 (100%)
Codes:
200: 1500
Promise - 50 "Virtual Users", 30 GETs
λ artillery quick --count 50 -n 30 http://159.65.99.177:3000/promise
Started phase 0, duration: 1s @ 15:06:57(-0600) 2018-03-06
Report @ 15:07:01(-0600) 2018-03-06
Scenarios launched: 50
Scenarios completed: 50
Requests completed: 1500
RPS sent: 374.06
Request latency:
min: 63.2
max: 360.3
median: 82.6
p95: 132.8
p99: 166
Codes:
200: 1500
All virtual users finished
Summary report @ 15:07:01(-0600) 2018-03-06
Scenarios launched: 50
Scenarios completed: 50
Requests completed: 1500
RPS sent: 372.21
Request latency:
min: 63.2
max: 360.3
median: 82.6
p95: 132.8
p99: 166
Scenario counts:
0: 50 (100%)
Codes:
200: 1500
Basic - 100 "Virtual Users", 30 GETs
λ artillery quick --count 100 -n 30 http://159.65.99.177:3000/basic
Started phase 0, duration: 2s @ 15:11:15(-0600) 2018-03-06
Report @ 15:11:21(-0600) 2018-03-06
Scenarios launched: 100
Scenarios completed: 100
Requests completed: 3000
RPS sent: 543.48
Request latency:
min: 62.3
max: 220.3
median: 114
p95: 155.4
p99: 174.5
Codes:
200: 3000
All virtual users finished
Summary report @ 15:11:21(-0600) 2018-03-06
Scenarios launched: 100
Scenarios completed: 100
Requests completed: 3000
RPS sent: 541.52
Request latency:
min: 62.3
max: 220.3
median: 114
p95: 155.4
p99: 174.5
Scenario counts:
0: 100 (100%)
Codes:
200: 3000
Promise - 100 "Virtual Users", 30 GETs
λ artillery quick --count 100 -n 30 http://159.65.99.177:3000/promise
Started phase 0, duration: 2s @ 15:12:23(-0600) 2018-03-06
Report @ 15:12:28(-0600) 2018-03-06
Scenarios launched: 100
Scenarios completed: 100
Requests completed: 3000
RPS sent: 542.5
Request latency:
min: 63.6
max: 240.4
median: 117.1
p95: 196.8
p99: 209.9
Codes:
200: 3000
All virtual users finished
Summary report @ 15:12:28(-0600) 2018-03-06
Scenarios launched: 100
Scenarios completed: 100
Requests completed: 3000
RPS sent: 540.54
Request latency:
min: 63.6
max: 240.4
median: 117.1
p95: 196.8
p99: 209.9
Scenario counts:
0: 100 (100%)
Codes:
200: 3000
Basic - 200 "Virtual Users", 50 GETs
λ artillery quick --count 200 -n 50 http://159.65.99.177:3000/basic
Started phase 0, duration: 4s @ 15:21:01(-0600) 2018-03-06
Report @ 15:21:11(-0600) 2018-03-06
Scenarios launched: 200
Scenarios completed: 0
Requests completed: 5737
RPS sent: 592.91
Request latency:
min: 62.1
max: 528.2
median: 272
p95: 403.4
p99: 431.9
Codes:
200: 5737
Report @ 15:21:17(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 200
Requests completed: 4263
RPS sent: 735.08
Request latency:
min: 66.1
max: 552.1
median: 214.4
p95: 272.9
p99: 289.2
Codes:
200: 4263
All virtual users finished
Summary report @ 15:21:17(-0600) 2018-03-06
Scenarios launched: 200
Scenarios completed: 200
Requests completed: 10000
RPS sent: 641.03
Request latency:
min: 62.1
max: 552.1
median: 245.8
p95: 382.5
p99: 424.9
Scenario counts:
0: 200 (100%)
Codes:
200: 10000
Promise - 200 "Virtual Users", 50 GETs
λ artillery quick --count 200 -n 50 http://159.65.99.177:3000/promise
Started phase 0, duration: 4s @ 15:22:10(-0600) 2018-03-06
Report @ 15:22:20(-0600) 2018-03-06
Scenarios launched: 200
Scenarios completed: 0
Requests completed: 5664
RPS sent: 585.81
Request latency:
min: 65.7
max: 468.6
median: 290.4
p95: 384.7
p99: 414.3
Codes:
200: 5664
Report @ 15:22:27(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 200
Requests completed: 4336
RPS sent: 589.17
Request latency:
min: 62.7
max: 433.5
median: 249.1
p95: 375.4
p99: 408.7
Codes:
200: 4336
All virtual users finished
Summary report @ 15:22:27(-0600) 2018-03-06
Scenarios launched: 200
Scenarios completed: 200
Requests completed: 10000
RPS sent: 585.14
Request latency:
min: 62.7
max: 468.6
median: 270.6
p95: 381.6
p99: 410
Scenario counts:
0: 200 (100%)
Codes:
200: 10000
Basic - 500 "Virtual Users", 100 GETs
λ artillery quick --count 500 -n 100 http://159.65.99.177:3000/basic
Started phase 0, duration: 10s @ 15:27:51(-0600) 2018-03-06
Report @ 15:28:01(-0600) 2018-03-06
Scenarios launched: 499
Scenarios completed: 0
Requests completed: 2969
RPS sent: 347.05
Request latency:
min: 68.3
max: 1459.7
median: 657.3
p95: 1170.9
p99: 1273.4
Codes:
200: 2969
Report @ 15:28:11(-0600) 2018-03-06
Scenarios launched: 1
Scenarios completed: 0
Requests completed: 3221
RPS sent: 322.3
Request latency:
min: 1228.9
max: 2406.5
median: 1474.4
p95: 2157.3
p99: 2234.8
Codes:
200: 3221
Report @ 15:28:21(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3465
RPS sent: 347.19
Request latency:
min: 1170.7
max: 1972.4
median: 1472.9
p95: 1591.9
p99: 1623.6
Codes:
200: 3465
Report @ 15:28:31(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 2944
RPS sent: 295.29
Request latency:
min: 1336.3
max: 2194.1
median: 1659.1
p95: 1851.4
p99: 1917.3
Codes:
200: 2944
Report @ 15:28:41(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 2695
RPS sent: 269.5
Request latency:
min: 1511
max: 2925.6
median: 1708.7
p95: 2599.9
p99: 2763
Codes:
200: 2695
Report @ 15:28:51(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3173
RPS sent: 317.3
Request latency:
min: 1215.4
max: 2177.5
median: 1605.9
p95: 1855.3
p99: 1914.4
Codes:
200: 3173
Report @ 15:29:01(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3103
RPS sent: 311.55
Request latency:
min: 1365.4
max: 2064.8
median: 1575.2
p95: 1790.9
p99: 1839.5
Codes:
200: 3103
Report @ 15:29:11(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3926
RPS sent: 392.21
Request latency:
min: 1007.3
max: 1879
median: 1298.1
p95: 1652.6
p99: 1695.8
Codes:
200: 3926
Report @ 15:29:21(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3500
RPS sent: 351.05
Request latency:
min: 1035.3
max: 2560.6
median: 1223.3
p95: 2261.8
p99: 2317.5
Codes:
200: 3500
Report @ 15:29:31(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3762
RPS sent: 375.82
Request latency:
min: 892
max: 2616
median: 1372.8
p95: 1992.1
p99: 2263.5
Codes:
200: 3762
Report @ 15:29:41(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 4787
RPS sent: 478.22
Request latency:
min: 872.3
max: 1304.4
median: 1029.1
p95: 1211.3
p99: 1256.7
Codes:
200: 4787
Report @ 15:29:51(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 2
Requests completed: 4288
RPS sent: 429.89
Request latency:
min: 955.4
max: 1546.2
median: 1154.3
p95: 1316
p99: 1342.4
Codes:
200: 4288
Report @ 15:30:01(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 43
Requests completed: 3345
RPS sent: 330.86
Request latency:
min: 1169.9
max: 2025.5
median: 1439.5
p95: 1695.3
p99: 1735.2
Codes:
200: 3345
Report @ 15:30:11(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 128
Requests completed: 3091
RPS sent: 297.19
Request latency:
min: 1062.3
max: 2158.7
median: 1313.9
p95: 1688.9
p99: 1725.6
Codes:
200: 3091
Report @ 15:30:17(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 327
Requests completed: 1731
RPS sent: 256.67
Request latency:
min: 67.3
max: 1544.1
median: 812.7
p95: 1134.7
p99: 1194.8
Codes:
200: 1731
All virtual users finished
Summary report @ 15:30:17(-0600) 2018-03-06
Scenarios launched: 500
Scenarios completed: 500
Requests completed: 50000
RPS sent: 342.4
Request latency:
min: 67.3
max: 2925.6
median: 1343.2
p95: 1868.3
p99: 2289
Scenario counts:
0: 500 (100%)
Codes:
200: 50000
Promise - 500 "Virtual Users", 100 GETs
λ artillery quick --count 500 -n 100 http://159.65.99.177:3000/promise
Started phase 0, duration: 10s @ 15:23:49(-0600) 2018-03-06
Report @ 15:23:59(-0600) 2018-03-06
Scenarios launched: 499
Scenarios completed: 0
Requests completed: 2840
RPS sent: 334.13
Request latency:
min: 73.3
max: 1409
median: 838
p95: 1189.6
p99: 1291.6
Codes:
200: 2840
Report @ 15:24:09(-0600) 2018-03-06
Scenarios launched: 1
Scenarios completed: 0
Requests completed: 3281
RPS sent: 328.3
Request latency:
min: 1109.5
max: 2140.4
median: 1508.1
p95: 1780.6
p99: 1811.6
Codes:
200: 3281
Report @ 15:24:19(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3164
RPS sent: 316.4
Request latency:
min: 1341.6
max: 2064.7
median: 1568.2
p95: 1747.3
p99: 1788.5
Codes:
200: 3164
Report @ 15:24:29(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3389
RPS sent: 338.9
Request latency:
min: 1130.1
max: 2303.4
median: 1441.1
p95: 1931.9
p99: 2009.6
Codes:
200: 3389
Report @ 15:24:39(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3653
RPS sent: 365.3
Request latency:
min: 1129.9
max: 1912.8
median: 1344
p95: 1672
p99: 1750.4
Codes:
200: 3653
Report @ 15:24:49(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3239
RPS sent: 323.8
Request latency:
min: 1177.4
max: 2181.4
median: 1541.4
p95: 1996.8
p99: 2080.1
Codes:
200: 3239
Report @ 15:24:59(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3167
RPS sent: 316.7
Request latency:
min: 1211.1
max: 2267.6
median: 1557
p95: 1819.5
p99: 1874.5
Codes:
200: 3167
Report @ 15:25:09(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3285
RPS sent: 328.6
Request latency:
min: 1291.1
max: 2083.3
median: 1534.9
p95: 1748.2
p99: 1818.5
Codes:
200: 3285
Report @ 15:25:20(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3217
RPS sent: 322.02
Request latency:
min: 1195
max: 2412.1
median: 1554.5
p95: 1967.8
p99: 2017.9
Codes:
200: 3217
Report @ 15:25:30(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3744
RPS sent: 375.53
Request latency:
min: 1024.8
max: 1831.5
median: 1325
p95: 1604.3
p99: 1667.5
Codes:
200: 3744
Report @ 15:25:40(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3376
RPS sent: 337.6
Request latency:
min: 1266.4
max: 1897.7
median: 1509.7
p95: 1641.8
p99: 1688.3
Codes:
200: 3376
Report @ 15:25:50(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 3311
RPS sent: 331.1
Request latency:
min: 1124.3
max: 2060.3
median: 1534.3
p95: 1781.6
p99: 1822.9
Codes:
200: 3311
Report @ 15:26:00(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 10
Requests completed: 3122
RPS sent: 312.45
Request latency:
min: 1254.5
max: 2007.5
median: 1627.3
p95: 1750.8
p99: 1789.1
Codes:
200: 3122
Report @ 15:26:10(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 50
Requests completed: 3369
RPS sent: 332.23
Request latency:
min: 1104.9
max: 1970.4
median: 1416.5
p95: 1659.1
p99: 1730.9
Codes:
200: 3369
Report @ 15:26:20(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 209
Requests completed: 3203
RPS sent: 299.4
Request latency:
min: 827.1
max: 1679.7
median: 1181.5
p95: 1464.4
p99: 1502.7
Codes:
200: 3203
Report @ 15:26:22(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 231
Requests completed: 640
RPS sent: 162.95
Request latency:
min: 118.2
max: 944.7
median: 790.7
p95: 894
p99: 928.5
Codes:
200: 640
All virtual users finished
Summary report @ 15:26:22(-0600) 2018-03-06
Scenarios launched: 500
Scenarios completed: 500
Requests completed: 50000
RPS sent: 326.71
Request latency:
min: 73.3
max: 2412.1
median: 1460.3
p95: 1776.8
p99: 1962.2
Scenario counts:
0: 500 (100%)
Codes:
200: 50000
1x CPU, 1 GiB Ram
Basic - 500 "Virtual Users", 100 GETs
λ artillery quick --count 500 -n 100 http://159.65.98.204:3000/basic
Started phase 0, duration: 10s @ 16:06:07(-0600) 2018-03-06
Report @ 16:06:17(-0600) 2018-03-06
Scenarios launched: 499
Scenarios completed: 0
Requests completed: 6656
RPS sent: 714.79
Request latency:
min: 64.3
max: 734.5
median: 357.6
p95: 558.2
p99: 615.7
Codes:
200: 6656
Report @ 16:06:27(-0600) 2018-03-06
Scenarios launched: 1
Scenarios completed: 0
Requests completed: 8778
RPS sent: 876.05
Request latency:
min: 499.7
max: 868.6
median: 575.8
p95: 605.5
p99: 626.5
Codes:
200: 8778
Report @ 16:06:37(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 8864
RPS sent: 884.73
Request latency:
min: 476.5
max: 873.3
median: 568.8
p95: 603.4
p99: 615.9
Codes:
200: 8864
Report @ 16:06:47(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 8760
RPS sent: 874.25
Request latency:
min: 473.4
max: 849.6
median: 569.5
p95: 614.6
p99: 633.4
Codes:
200: 8760
Report @ 16:06:57(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 65
Requests completed: 8857
RPS sent: 877.45
Request latency:
min: 467.9
max: 856.4
median: 542.4
p95: 590
p99: 613.9
Codes:
200: 8857
Report @ 16:07:07(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 435
Requests completed: 8085
RPS sent: 802.73
Request latency:
min: 63
max: 554.6
median: 415.7
p95: 503
p99: 517.4
Codes:
200: 8085
All virtual users finished
Summary report @ 16:07:07(-0600) 2018-03-06
Scenarios launched: 500
Scenarios completed: 500
Requests completed: 50000
RPS sent: 834.72
Request latency:
min: 63
max: 873.3
median: 552.7
p95: 600.5
p99: 622.7
Scenario counts:
0: 500 (100%)
Codes:
200: 50000
Promise - 500 "Virtual Users", 100 GETs
λ artillery quick --count 500 -n 100 http://159.65.98.204:3000/promise
Started phase 0, duration: 10s @ 16:08:07(-0600) 2018-03-06
Report @ 16:08:17(-0600) 2018-03-06
Scenarios launched: 499
Scenarios completed: 0
Requests completed: 7403
RPS sent: 788.61
Request latency:
min: 63.5
max: 700
median: 306.2
p95: 516.4
p99: 564.6
Codes:
200: 7403
Report @ 16:08:28(-0600) 2018-03-06
Scenarios launched: 1
Scenarios completed: 0
Requests completed: 9068
RPS sent: 904.99
Request latency:
min: 470.2
max: 822.4
median: 555.3
p95: 593.2
p99: 613.7
Codes:
200: 9068
Report @ 16:08:38(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 0
Requests completed: 8916
RPS sent: 893.39
Request latency:
min: 473.4
max: 872.2
median: 561.1
p95: 603.4
p99: 629.3
Codes:
200: 8916
Report @ 16:08:48(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 9
Requests completed: 8784
RPS sent: 875.75
Request latency:
min: 503.8
max: 694.4
median: 565.7
p95: 645.4
p99: 677.2
Codes:
200: 8784
Report @ 16:08:58(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 90
Requests completed: 8995
RPS sent: 888.62
Request latency:
min: 406.5
max: 804.5
median: 516.3
p95: 569
p99: 588.2
Codes:
200: 8995
Report @ 16:09:06(-0600) 2018-03-06
Scenarios launched: 0
Scenarios completed: 401
Requests completed: 6834
RPS sent: 799.25
Request latency:
min: 62.2
max: 533.6
median: 375.8
p95: 449.2
p99: 506.3
Codes:
200: 6834
All virtual users finished
Summary report @ 16:09:06(-0600) 2018-03-06
Scenarios launched: 500
Scenarios completed: 500
Requests completed: 50000
RPS sent: 855.72
Request latency:
min: 62.2
max: 872.2
median: 536.9
p95: 598.4
p99: 645.7
Scenario counts:
0: 500 (100%)
Codes:
200: 50000
Top comments (0)