DEV Community

Nuster Cache Server
Nuster Cache Server

Posted on

Web cache server HTTP/2 performance benchmark: nuster vs nginx

This is a simple benchmark comparing the HTTP/2 cache performance of nuster and nginx.

https://github.com/jiangwenyuan/nuster

I tested the RPS(Request per second) by h2load.

The test shown that nuster is almost three times faster than nginx.

Here is the result(RPS) when request /helloworld URL containing hello world data.

data size CONN nuster nginx
12(hello world) 1000 338924 110419

Testing environment

Server

Two linux servers are used, an origin web server on server129,
and cache server nuster/nginx on server130.

Server port app
10.0.0.129 wrk
10.0.0.129 8080 nginx, origin web server
10.0.0.130 8081 nuster, all cores
10.0.0.130 8083 nginx, all cores

origin web server: set server_tokens off; to make http header server same.

Hardware

  • Intel(R) Xeon(R) CPU X5650 @ 2.67GHz(12 cores)
  • RAM 32GB
  • 1Gbps ethernet card

Software

  • CentOS: 7.4.1708 (Core)
  • h2load: nghttp2/1.30.0
  • nginx: nginx/1.14.0
  • nuster: nuster/1.8.8.2

System settings

/etc/sysctl.conf

fs.file-max                    = 9999999
fs.nr_open                     = 9999999
net.core.netdev_max_backlog    = 4096
net.core.rmem_max              = 16777216
net.core.somaxconn             = 65535
net.core.wmem_max              = 16777216
net.ipv4.ip_forward            = 0
net.ipv4.ip_local_port_range   = 1025       65535
net.ipv4.tcp_fin_timeout       = 30
net.ipv4.tcp_keepalive_time    = 30
net.ipv4.tcp_max_syn_backlog   = 20480
net.ipv4.tcp_max_tw_buckets    = 400000
net.ipv4.tcp_no_metrics_save   = 1
net.ipv4.tcp_syn_retries       = 2
net.ipv4.tcp_synack_retries    = 2
net.ipv4.tcp_tw_recycle        = 1
net.ipv4.tcp_tw_reuse          = 1
net.ipv4.tcp_timestamps        = 1
vm.min_free_kbytes             = 65536
vm.overcommit_memory           = 1
Enter fullscreen mode Exit fullscreen mode

/etc/security/limits.conf

* soft nofile 1000000
* hard nofile 1000000
* soft nproc  1000000
* hard nproc  1000000
Enter fullscreen mode Exit fullscreen mode

Config files

nuster, all cores

global
    maxconn 1000000
    nuster cache on data-size 1g
    daemon
    nbproc 12
    tune.maxaccept -1
    tune.ssl.default-dh-param 2048
    tune.h2.max-concurrent-streams 1000
defaults
    retries 3
    maxconn 1000000
    option redispatch
    option dontlognull
    timeout client  300s
    timeout connect 300s
    timeout server  300s
    http-reuse always
frontend web1
    bind *:8081 ssl crt asdf.pem alpn h2,http/1.1
    mode http
    default_backend app1
backend app1
    balance roundrobin
    mode http
    nuster cache on
    nuster rule all ttl 0
    server a2 10.0.0.129:8080
Enter fullscreen mode Exit fullscreen mode

nginx, all cores

user  nginx;
worker_processes  auto;
worker_rlimit_nofile 1000000;
error_log  /var/log/nginx/errorall.log warn;
pid        /var/run/nginxall.pid;
events {
  worker_connections  1000000;
  use epoll;
  multi_accept on;
}
http {
  include                     /etc/nginx/mime.types;
  default_type                application/octet-stream;
  access_log                  off;
  sendfile                    on;
  server_tokens               off;
  keepalive_timeout           300;
  keepalive_requests          100000;
  tcp_nopush                  on;
  tcp_nodelay                 on;
  client_body_buffer_size     128k;
  client_header_buffer_size   1m;
  large_client_header_buffers 4 4k;
  output_buffers              1 32k;
  postpone_output             1460;
  open_file_cache             max=200000 inactive=20s;
  open_file_cache_valid       30s;
  open_file_cache_min_uses    2;
  open_file_cache_errors      on;
  proxy_cache_path /tmp/cache_all levels=1:2 keys_zone=STATIC:10m inactive=24h max_size=1g;
  server {
    listen 8083 ssl http2;
    ssl on;
    ssl_session_cache shared:SSL:5m;
    ssl_session_timeout 1h;
    http2_max_concurrent_streams 1000;
    http2_max_requests 10000000;
    ssl_certificate     a.pem;
    ssl_certificate_key a.key;

    location / {
      proxy_pass        http://10.0.0.129:8080/;
      proxy_cache       STATIC;
      proxy_cache_valid any 1d;
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Benchmark

nuster

$ h2load -n 10000000 -c 1000 -m 1000 -t 1000 https://10.0.0.130:8081/helloworld


finished in 29.51s, 338924.15 req/s, 48.81MB/s
requests: 10000000 total, 10000000 started, 10000000 done, 10000000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1.41GB (1510024000) total, 1.13GB (1210000000) headers (space savings 34.24%), 114.44MB (120000000) data
                     min         max         mean         sd        +/- sd
time for request:    14.51ms       3.21s       2.53s    500.02ms    74.50%
time for connect:    61.12ms       1.26s    672.73ms    332.30ms    63.50%
time to 1st byte:    95.03ms       3.74s       2.68s    777.51ms    74.00%
req/s           :     339.11      459.68      358.92       17.35    76.20%
Enter fullscreen mode Exit fullscreen mode

nginx

$ h2load -n 10000000 -c 1000 -m 1000 -t 1000 https://10.0.0.130:8083/helloworld


finished in 90.56s, 110419.16 req/s, 15.62MB/s
requests: 10000000 total, 10000000 started, 10000000 done, 10000000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 1.38GB (1482955210) total, 1.10GB (1182906210) headers (space savings 35.01%), 114.44MB (120000000) data
                     min         max         mean         sd        +/- sd
time for request:     3.98ms      14.65s       5.14s       3.67s    70.30%
time for connect:    69.96ms       6.74s       1.35s       1.08s    88.90%
time to 1st byte:   114.92ms      15.46s       5.67s       4.06s    63.40%
req/s           :     110.43     5143.28      378.40      590.47    92.50%
Enter fullscreen mode Exit fullscreen mode

Details: https://github.com/jiangwenyuan/nuster

Discussion (14)

Collapse
patricklbs profile image
patricklbs

Hello,
I would like to know how nuster works with haproxy? You do not mention it in your doc on github.
We have a haproxy for loadbalancing and we would also like to have a cache without having to install an apache. We have JavaEE middleware applications tomcat / wildfly.

Does Haproxy and Nuster work together? Do you have a doc? Haproxy listens on the: 80 and nuster on the: 8080 we have keepalived in frontend to make high availability haproxy level.

thank you in advance for the answer.

Patrick.

Collapse
nuster profile image
Nuster Cache Server Author • Edited

Hi,

You don't need haproxy and nuster work together. You can use nuster as a http loadbalancer just like haproxy as nuster is built on top of haproxy and inherits all features of HAProxy, it's 100% compatible with HAProxy.

So suppose you have haproxy in /usr/local/bin/haproxy and haproxy conf in /etc/haproxy/haproxy.conf.

What you need to do is to install nuster, say /usr/local/nuster/bin/haproxy. And if you run /usr/local/nuster/bin/haproxy -f /etc/haproxy/haproxy.conf, nuster will work just like haproxy.

In order to enable cache, update /etc/haproxy/haproxy.conf this way:

Add nuster cache on into global section, and nuster cache on and some rules nuster rule into backend section

So your /etc/haproxy/haproxy.conf looks like this:

global
    # your original conf here
    # add this:
    nuster cache on data-size 1g
#....
frontend your-original-frontend
    bind *:80 #
    mode http
    #...
backend your-original-tomcat-backend
    mode http
    # add following two lines
    nuster cache on
    nuster rule all ttl 0
    # end here
    server your-tomcat-server tomcat-ip:tomcat-port

Then kill original haproxy process, and start nuster: /usr/local/nuster/bin/haproxy -f /etc/haproxy/haproxy.conf

That's it.

you can check which haproxy version is nuster based on here: github.com/jiangwenyuan/nuster/blo...

Collapse
patricklbs profile image
patricklbs

Thank's a lot for your reply, J like what I see :)

Can you tell me which section you put the stats tag:
nuster cache on uri / nuster / cache

is it the same as the haproxy stats I guess not?
I would like to do surveys with and without nuster, to see the differences and thus convince of the good choice of nuster! Again thank you, I hope that Nuster has a nice day ahead of him :)

Thread Thread
nuster profile image
Nuster Cache Server Author

Hi, you should put it in global section,

global
    nuster cache on uri /_any/path/would/do
...

It's not haproxy stats(web page nor cli), you can get the stats using curl http://127.0.0.1[:port]/_any/path/would/do.

Thread Thread
patricklbs profile image
patricklbs
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    stats timeout 2m
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    nuster cache on data-size 1g
    # turn on stats unix socket
    stats socket /var/lib/haproxy/haproxy.sock mode 660 level admin
    stats timeout 2m
    nuster cache on uri /nuster/stats
curl http://127.0.0.1/nuster/stats 

<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

I forgot something ?

Thread Thread
patricklbs profile image
patricklbs • Edited

How to configure port for stats view, because frontend listen on port 80 :/

Thread Thread
patricklbs profile image
patricklbs

Ok I found it

nuster cache on data-size 1g uri /nuster/stats

Thread Thread
nuster profile image
Nuster Cache Server Author

Hi, you can use the port defined in frontend

Collapse
patricklbs profile image
patricklbs

Hi,

I have a problem with java EE application who start on tomcat. when I login on after I want to logout, the session stay open.

My nuster config :



backend demo
        #mode http
        cookie JSESSIONID prefix nocache
        #option redispatch
        #option httpclose
        #option forwardfor
        #http-check disable-on-404
        #option httpchk GET / HTTP/1.1\r\nHost:\ sli-iqr-11.cg13.fr
        #option http-buffer-request
        server  tomcat-demo  hostname.tomcat.demo.fr:8080 cookie check inter 2000
        server  tomcat_backup hostname.tomcat.backup.fr:82 backup
        nuster cache on
        nuster rule all ttl 0

any idee ? Thank's a lot

Patrick.

Collapse
nuster profile image
Nuster Cache Server Author

Hi,

maybe you should cache certain page instead of all nuster rule all ttl 0

Collapse
kris33 profile image
Kris33

Hi,

I'm running Haproxy on CentOS 7.6.1810, I'd give a try to nuster but I can't figure out how to install it.
Is there an rpm or something cause buiding always fails with lua versions and missing dependencies default lua version is 5.1.4.

Collapse
nuster profile image
Nuster Cache Server Author • Edited

Hi,

You can build without lua, just omit lua related parameters when you make nuster.

Or you can install lua manually, for example, you can refer to

gist.github.com/drpauldixon/b75a33...

Collapse
tictakk profile image
Matthew Kersey

What exactly does "based on HAProxy" mean? Built with the same purpose of HAProxy in mind, is it a fork? Curious to how you decided to improve upon it. Looks impressive, might have to give it a try myself.

Collapse
nuster profile image
Nuster Cache Server Author

nuster uses the source code from HAProxy and added cache functionality on top of it. nuster is not built with the same purpose of HAProxy, but with the powerful features of HAProxy, like load balacing/ACL.

nuster started from HAProxy v1.7 when there was no small cache ability in haproxy, which was introduced in haproxy v1.8.

But the cache introduced in haproxy1.8 has many limitations, for example only small response can be cached, which defaults to 16KB as defined by the global parameter tune.bufsize, while nuster can cache any size response.

Also haproxy1.8 cache only works for 200 and GET, while nuster can cache any http code and POST/PUT.

Also haproxy1.8 cache can only use host and urias key, while nuster can use header, cookie, query as key too.

And nuster has PURGE functionalities, disable/enable at runtime functionalities, stats, and so on.

Please do give it a try, any feedback is welcomed:)