haproxy tuning for TLS termination

#haproxy #openssl

I'm working on a platform which has been a little neglects from a performance tuning point of view. We had some issues recently which effectively caused denial of service, even though we were doing it to ourselves. As a result, I'm tuning some things and I've learned some things which I didn't know.

You can probably DoS your haproxy instance too, if you've not already tuned it. I'm using the wonderful hey tool to create connections to my haproxy.

My haproxy has multiple certificates with unique private keys associated with each. Some are 4096 bit keys. Most are 2048 bit keys. Which certificate and key is used depends on which hostname is used to connect to the server.

I see huge differences in the performance based on 2k or 4k PK used for signing. I've learned that this is expected and that openssl ships with a speed command so that you know what to expect. In my case, haproxy is running on nodes with 8 cores (c5.2xlarge) and is configured to use them (cpu-map 1- 0-).

I've seen the openssl speed rsa command, but I had to hunt down the rsa2048, rsa4096, and -multi commands and option.

In my case, there isn't a great solution because I'm mixing 4k and 2k PK. Don't do that. I'll be advocating to my team to use 2k as much as possible.

Here is example output from the commands and the resulting sign/s number is pretty much the value I suggest using for the maxsslrate setting in haproxy.

# openssl speed  -multi 8 rsa2048
Forked child 0
Forked child 1
Forked child 2
Forked child 3
Forked child 4
Forked child 5
Forked child 6
Forked child 7
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+DTP:2048:private:rsa:10
+R1:8096:2048:10.00
+DTP:2048:public:rsa:10
+R1:8202:2048:10.00
+DTP:2048:public:rsa:10
+R1:8279:2048:10.00
+DTP:2048:public:rsa:10
+R1:7769:2048:10.00
+DTP:2048:public:rsa:10
+R1:8329:2048:10.00
+R1:7351:2048:10.00
+DTP:2048:public:rsa:10
+DTP:2048:public:rsa:10
+R1:7355:2048:10.00
+DTP:2048:public:rsa:10
+R1:7363:2048:10.00
+DTP:2048:public:rsa:10
+R2:258668:2048:10.00
+R2:280384:2048:10.00
+R2:274594:2048:10.00
Got: +F2:2:2048:809.600000:25866.800000 from 0
+R2:262465:2048:10.00
+R2:279210:2048:10.00
+R2:255086:2048:10.00
Got: +F2:2:2048:832.900000:27921.000000 from 1
Got: +F2:2:2048:735.100000:25508.600000 from 2
Got: +F2:2:2048:827.900000:27459.400000 from 3
Got: +F2:2:2048:776.900000:26246.500000 from 4
+R2:266538:2048:10.00
Got: +F2:2:2048:735.500000:26653.800000 from 5
Got: +F2:2:2048:820.200000:28038.400000 from 6
+R2:297361:2048:10.00
Got: +F2:2:2048:736.300000:29736.100000 from 7
OpenSSL 1.1.1  11 Sep 2018
built on: Mon Jul  4 11:25:51 2022 UTC
options:bn(64,64) rc4(16x,int) des(int) aes(partial) blowfish(ptr) 
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-wL7Fqk/openssl-1.1.1=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000159s 0.000005s   6274.4 217430.6

# openssl speed  -multi 8 rsa4096
Forked child 0
Forked child 1
Forked child 2
Forked child 3
Forked child 4
Forked child 5
Forked child 6
Forked child 7
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+DTP:4096:private:rsa:10
+R1:1170:4096:10.00
+DTP:4096:public:rsa:10
+R1:1144:4096:10.00
+DTP:4096:public:rsa:10
+R1:1195:4096:10.00
+DTP:4096:public:rsa:10
+R1:1152:4096:10.00
+DTP:4096:public:rsa:10
+R1:1070:4096:10.00
+DTP:4096:public:rsa:10
+R1:1178:4096:10.01
+DTP:4096:public:rsa:10
+R1:1195:4096:10.00
+DTP:4096:public:rsa:10
+R1:1145:4096:10.02
+DTP:4096:public:rsa:10
+R2:75477:4096:10.00
+R2:71013:4096:10.00
+R2:82989:4096:10.00
+R2:84313:4096:10.00
Got: +F2:4:4096:115.200000:8431.300000 from 0
+R2:77447:4096:10.00
+R2:76991:4096:10.00
Got: +F2:4:4096:117.682318:7699.100000 from 1
Got: +F2:4:4096:119.500000:8298.900000 from 2
Got: +F2:4:4096:107.000000:7744.700000 from 3
Got: +F2:4:4096:117.000000:7547.700000 from 4
+R2:75476:4096:10.00
Got: +F2:4:4096:119.500000:7547.600000 from 5
+R2:78033:4096:10.00
Got: +F2:4:4096:114.271457:7803.300000 from 6
Got: +F2:4:4096:114.400000:7101.300000 from 7
OpenSSL 1.1.1  11 Sep 2018
built on: Mon Jul  4 11:25:51 2022 UTC
options:bn(64,64) rc4(16x,int) des(int) aes(partial) blowfish(ptr) 
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-wL7Fqk/openssl-1.1.1=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
                  sign    verify    sign/s verify/s
rsa 4096 bits 0.001082s 0.000016s    924.6  62173.9

6200 is much larger than 920.

2k keys and maxsslrate 6200 it is.

DEV Community

haproxy tuning for TLS termination

Top comments (0)

Read next

Personalizing Your Website with Random Text Displays

A Beginner’s Guide to Building APIs with Express.js

All About Axios…🥳

Software Engineer Interviews - #3 OEIS CLI