DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’» is a community of 963,274 amazing developers

We're a place where coders share, stay up-to-date and grow their careers.

Create account Log in
Yao-Ching Huang
Yao-Ching Huang

Posted on • Updated on

Python :: The Performance Benchmarks Comparing various combinations of GCC and Python

This post shows the performance benchmark under different combinations of Python and GCC.

For each combination, We launch a GCC container and build Python with the GCC. Then run benchmarks using pyperformance and export to a JSON file.

Finally, we use pyperf to compare the performance and generate Markdown tables.

Configurations

The benchmark matrix:

  • GCC
    • 8.5.0
    • 12.1.0
  • Python
    • 3.9.13
    • 3.10.6

The environment:

  • OS: Ubuntu 20.04.4 LTS with 5.15 kernel
  • CPU: AMD Ryzen 7 2700X Eight-Core Processor
  • RAM: 32GB

Benchmark Results

Python 3.10 compiled with different GCC versions

Among the 64 benchmarks, py310-gcc12 gets better results in 46 benchmarks.

Benchmark py310-gcc8 py310-gcc12
2to3 530 ms 522 ms: 1.01x faster
chameleon 16.7 ms 17.1 ms: 1.03x slower
chaos 199 ms 198 ms: 1.00x faster
crypto_pyaes 184 ms 180 ms: 1.02x faster
deltablue 12.9 ms 12.7 ms: 1.01x faster
django_template 103 ms 102 ms: 1.01x faster
dulwich_log 126 ms 123 ms: 1.03x faster
fannkuch 754 ms 776 ms: 1.03x slower
float 177 ms 169 ms: 1.04x faster
genshi_text 57.4 ms 54.8 ms: 1.05x faster
genshi_xml 120 ms 117 ms: 1.02x faster
go 398 ms 395 ms: 1.01x faster
hexiom 16.1 ms 15.3 ms: 1.05x faster
json_dumps 20.1 ms 19.8 ms: 1.02x faster
json_loads 36.8 us 34.7 us: 1.06x faster
logging_format 18.3 us 17.8 us: 1.03x faster
logging_silent 372 ns 355 ns: 1.05x faster
logging_simple 16.7 us 16.2 us: 1.03x faster
mako 26.0 ms 24.7 ms: 1.05x faster
nbody 219 ms 210 ms: 1.04x faster
nqueens 170 ms 173 ms: 1.02x slower
pathlib 32.6 ms 33.5 ms: 1.03x slower
pickle 15.6 us 15.4 us: 1.01x faster
pickle_dict 32.4 us 31.6 us: 1.02x faster
pickle_list 4.90 us 4.98 us: 1.02x slower
pickle_pure_python 923 us 876 us: 1.05x faster
pidigits 183 ms 175 ms: 1.05x faster
pyflate 1.20 sec 1.20 sec: 1.00x faster
python_startup 10.8 ms 10.6 ms: 1.02x faster
python_startup_no_site 6.90 ms 6.77 ms: 1.02x faster
raytrace 945 ms 953 ms: 1.01x slower
regex_compile 301 ms 289 ms: 1.04x faster
regex_dna 195 ms 196 ms: 1.01x slower
regex_effbot 3.41 ms 3.44 ms: 1.01x slower
regex_v8 29.6 ms 30.4 ms: 1.03x slower
richards 136 ms 138 ms: 1.01x slower
scimark_fft 617 ms 631 ms: 1.02x slower
scimark_lu 271 ms 258 ms: 1.05x faster
scimark_monte_carlo 191 ms 186 ms: 1.03x faster
scimark_sor 365 ms 355 ms: 1.03x faster
scimark_sparse_mat_mult 8.79 ms 8.14 ms: 1.08x faster
spectral_norm 265 ms 249 ms: 1.07x faster
sqlalchemy_declarative 212 ms 209 ms: 1.01x faster
sqlalchemy_imperative 32.9 ms 32.2 ms: 1.02x faster
sympy_expand 910 ms 878 ms: 1.04x faster
sympy_integrate 36.9 ms 35.3 ms: 1.05x faster
sympy_sum 277 ms 269 ms: 1.03x faster
sympy_str 531 ms 517 ms: 1.03x faster
telco 12.2 ms 11.7 ms: 1.05x faster
tornado_http 208 ms 205 ms: 1.01x faster
unpack_sequence 128 ns 129 ns: 1.01x slower
unpickle 22.4 us 22.0 us: 1.02x faster
unpickle_list 6.79 us 7.53 us: 1.11x slower
unpickle_pure_python 630 us 601 us: 1.05x faster
xml_etree_parse 246 ms 241 ms: 1.02x faster
xml_etree_iterparse 158 ms 153 ms: 1.03x faster
xml_etree_generate 167 ms 158 ms: 1.06x faster
xml_etree_process 138 ms 137 ms: 1.01x faster
Geometric mean (ref) 1.02x faster

Benchmark hidden because not significant (3): html5lib, meteor_contest, sqlite_synth

Various Python versions compiled with GCC 12

Among the 64 benchmarks, py310-gcc12 gets better results in 39 benchmarks.

Benchmark py39-gcc12 py310-gcc12
2to3 533 ms 522 ms: 1.02x faster
chameleon 17.9 ms 17.1 ms: 1.04x faster
chaos 204 ms 198 ms: 1.03x faster
crypto_pyaes 189 ms 180 ms: 1.05x faster
deltablue 12.4 ms 12.7 ms: 1.02x slower
django_template 107 ms 102 ms: 1.05x faster
dulwich_log 129 ms 123 ms: 1.05x faster
fannkuch 720 ms 776 ms: 1.08x slower
float 185 ms 169 ms: 1.09x faster
genshi_text 64.1 ms 54.8 ms: 1.17x faster
genshi_xml 123 ms 117 ms: 1.05x faster
go 444 ms 395 ms: 1.12x faster
hexiom 15.3 ms 15.3 ms: 1.01x faster
html5lib 144 ms 139 ms: 1.04x faster
json_dumps 20.4 ms 19.8 ms: 1.03x faster
json_loads 36.3 us 34.7 us: 1.05x faster
logging_format 17.4 us 17.8 us: 1.02x slower
logging_silent 329 ns 355 ns: 1.08x slower
logging_simple 16.0 us 16.2 us: 1.01x slower
mako 27.8 ms 24.7 ms: 1.12x faster
meteor_contest 152 ms 151 ms: 1.00x faster
nbody 202 ms 210 ms: 1.04x slower
nqueens 168 ms 173 ms: 1.03x slower
pathlib 32.4 ms 33.5 ms: 1.03x slower
pickle 14.8 us 15.4 us: 1.04x slower
pickle_dict 30.8 us 31.6 us: 1.03x slower
pickle_list 4.73 us 4.98 us: 1.05x slower
pickle_pure_python 909 us 876 us: 1.04x faster
pidigits 175 ms 175 ms: 1.00x faster
pyflate 1.22 sec 1.20 sec: 1.02x faster
python_startup 11.0 ms 10.6 ms: 1.04x faster
python_startup_no_site 7.21 ms 6.77 ms: 1.07x faster
raytrace 932 ms 953 ms: 1.02x slower
regex_compile 290 ms 289 ms: 1.00x faster
regex_dna 201 ms 196 ms: 1.03x faster
regex_effbot 3.53 ms 3.44 ms: 1.02x faster
regex_v8 30.2 ms 30.4 ms: 1.01x slower
richards 129 ms 138 ms: 1.07x slower
scimark_fft 609 ms 631 ms: 1.04x slower
scimark_lu 244 ms 258 ms: 1.06x slower
scimark_monte_carlo 195 ms 186 ms: 1.05x faster
scimark_sor 332 ms 355 ms: 1.07x slower
scimark_sparse_mat_mult 8.75 ms 8.14 ms: 1.08x faster
spectral_norm 245 ms 249 ms: 1.01x slower
sqlalchemy_declarative 212 ms 209 ms: 1.01x faster
sqlite_synth 4.92 us 4.50 us: 1.09x faster
sympy_expand 910 ms 878 ms: 1.04x faster
sympy_integrate 35.6 ms 35.3 ms: 1.01x faster
sympy_str 529 ms 517 ms: 1.02x faster
telco 11.8 ms 11.7 ms: 1.01x faster
tornado_http 207 ms 205 ms: 1.01x faster
unpickle 22.1 us 22.0 us: 1.00x faster
unpickle_list 7.42 us 7.53 us: 1.01x slower
unpickle_pure_python 627 us 601 us: 1.04x faster
xml_etree_parse 244 ms 241 ms: 1.01x faster
xml_etree_iterparse 152 ms 153 ms: 1.01x slower
xml_etree_generate 162 ms 158 ms: 1.02x faster
xml_etree_process 138 ms 137 ms: 1.01x faster
Geometric mean (ref) 1.01x faster

Benchmark hidden because not significant (3): sqlalchemy_imperative, sympy_sum, unpack_sequence

Overall Benchmarks

We set py39-gcc8 as baseline, let's take a look the performance of each combination against the baseline.

  • py39-gcc12 is faster in 34 benchmarks.
  • py310-gcc8 is faster in 28 benchmarks.
  • py310-gcc12 is faster in 37 benchmarks.
Benchmark py39-gcc8 py39-gcc12 py310-gcc8 py310-gcc12
2to3 526 ms 533 ms: 1.01x slower 530 ms: 1.01x slower 522 ms: 1.01x faster
chameleon 18.2 ms 17.9 ms: 1.02x faster 16.7 ms: 1.09x faster 17.1 ms: 1.07x faster
chaos 206 ms 204 ms: 1.01x faster 199 ms: 1.04x faster 198 ms: 1.04x faster
crypto_pyaes 182 ms 189 ms: 1.04x slower 184 ms: 1.01x slower 180 ms: 1.01x faster
deltablue 12.2 ms 12.4 ms: 1.01x slower 12.9 ms: 1.05x slower 12.7 ms: 1.04x slower
django_template 109 ms 107 ms: 1.02x faster 103 ms: 1.06x faster 102 ms: 1.07x faster
dulwich_log 129 ms not significant 126 ms: 1.02x faster 123 ms: 1.05x faster
fannkuch 769 ms 720 ms: 1.07x faster 754 ms: 1.02x faster 776 ms: 1.01x slower
float 190 ms 185 ms: 1.02x faster 177 ms: 1.07x faster 169 ms: 1.12x faster
genshi_text 60.8 ms 64.1 ms: 1.06x slower 57.4 ms: 1.06x faster 54.8 ms: 1.11x faster
genshi_xml 122 ms 123 ms: 1.01x slower 120 ms: 1.02x faster 117 ms: 1.04x faster
go 439 ms 444 ms: 1.01x slower 398 ms: 1.10x faster 395 ms: 1.11x faster
hexiom 15.1 ms 15.3 ms: 1.02x slower 16.1 ms: 1.07x slower 15.3 ms: 1.01x slower
html5lib 142 ms 144 ms: 1.02x slower 139 ms: 1.02x faster 139 ms: 1.02x faster
json_dumps 20.1 ms 20.4 ms: 1.01x slower 20.1 ms: 1.00x faster 19.8 ms: 1.02x faster
json_loads 38.2 us 36.3 us: 1.05x faster 36.8 us: 1.04x faster 34.7 us: 1.10x faster
logging_format 17.9 us 17.4 us: 1.03x faster 18.3 us: 1.02x slower 17.8 us: 1.01x faster
logging_silent 317 ns 329 ns: 1.04x slower 372 ns: 1.17x slower 355 ns: 1.12x slower
logging_simple 16.3 us 16.0 us: 1.02x faster 16.7 us: 1.02x slower 16.2 us: 1.00x faster
mako 27.5 ms 27.8 ms: 1.01x slower 26.0 ms: 1.06x faster 24.7 ms: 1.11x faster
meteor_contest 150 ms 152 ms: 1.01x slower 151 ms: 1.01x slower 151 ms: 1.01x slower
nbody 210 ms 202 ms: 1.04x faster 219 ms: 1.04x slower not significant
nqueens 172 ms 168 ms: 1.02x faster 170 ms: 1.01x faster 173 ms: 1.01x slower
pathlib 32.2 ms 32.4 ms: 1.01x slower 32.6 ms: 1.01x slower 33.5 ms: 1.04x slower
pickle 16.3 us 14.8 us: 1.10x faster 15.6 us: 1.04x faster 15.4 us: 1.06x faster
pickle_dict 30.8 us not significant 32.4 us: 1.05x slower 31.6 us: 1.03x slower
pickle_list 4.66 us 4.73 us: 1.02x slower 4.90 us: 1.05x slower 4.98 us: 1.07x slower
pickle_pure_python 948 us 909 us: 1.04x faster 923 us: 1.03x faster 876 us: 1.08x faster
pidigits 182 ms 175 ms: 1.04x faster 183 ms: 1.01x slower 175 ms: 1.04x faster
pyflate 1.20 sec 1.22 sec: 1.02x slower not significant 1.20 sec: 1.00x faster
python_startup 11.1 ms 11.0 ms: 1.01x faster 10.8 ms: 1.03x faster 10.6 ms: 1.05x faster
python_startup_no_site 7.64 ms 7.21 ms: 1.06x faster 6.90 ms: 1.11x faster 6.77 ms: 1.13x faster
raytrace 941 ms 932 ms: 1.01x faster 945 ms: 1.00x slower 953 ms: 1.01x slower
regex_compile 295 ms 290 ms: 1.02x faster 301 ms: 1.02x slower 289 ms: 1.02x faster
regex_dna 200 ms 201 ms: 1.01x slower 195 ms: 1.03x faster 196 ms: 1.02x faster
regex_effbot 3.36 ms 3.53 ms: 1.05x slower 3.41 ms: 1.02x slower 3.44 ms: 1.02x slower
regex_v8 29.0 ms 30.2 ms: 1.04x slower 29.6 ms: 1.02x slower 30.4 ms: 1.05x slower
richards 132 ms 129 ms: 1.03x faster 136 ms: 1.03x slower 138 ms: 1.05x slower
scimark_fft 630 ms 609 ms: 1.03x faster 617 ms: 1.02x faster not significant
scimark_lu 279 ms 244 ms: 1.14x faster 271 ms: 1.03x faster 258 ms: 1.08x faster
scimark_monte_carlo 188 ms 195 ms: 1.04x slower 191 ms: 1.02x slower 186 ms: 1.01x faster
scimark_sor 354 ms 332 ms: 1.07x faster 365 ms: 1.03x slower not significant
scimark_sparse_mat_mult 8.65 ms 8.75 ms: 1.01x slower 8.79 ms: 1.02x slower 8.14 ms: 1.06x faster
spectral_norm 273 ms 245 ms: 1.11x faster 265 ms: 1.03x faster 249 ms: 1.10x faster
sqlalchemy_declarative 211 ms not significant not significant 209 ms: 1.01x faster
sqlalchemy_imperative 32.4 ms not significant 32.9 ms: 1.02x slower not significant
sqlite_synth 4.98 us 4.92 us: 1.01x faster 4.51 us: 1.10x faster 4.50 us: 1.11x faster
sympy_expand 921 ms 910 ms: 1.01x faster 910 ms: 1.01x faster 878 ms: 1.05x faster
sympy_integrate 35.7 ms 35.6 ms: 1.00x faster 36.9 ms: 1.03x slower 35.3 ms: 1.01x faster
sympy_sum 273 ms 269 ms: 1.01x faster 277 ms: 1.01x slower 269 ms: 1.02x faster
sympy_str 534 ms 529 ms: 1.01x faster 531 ms: 1.01x faster 517 ms: 1.03x faster
telco 12.1 ms 11.8 ms: 1.02x faster 12.2 ms: 1.02x slower 11.7 ms: 1.03x faster
tornado_http 209 ms 207 ms: 1.01x faster not significant 205 ms: 1.02x faster
unpack_sequence 149 ns 129 ns: 1.16x faster 128 ns: 1.16x faster 129 ns: 1.16x faster
unpickle 22.0 us not significant 22.4 us: 1.02x slower not significant
unpickle_list 6.63 us 7.42 us: 1.12x slower 6.79 us: 1.02x slower 7.53 us: 1.14x slower
unpickle_pure_python 627 us not significant 630 us: 1.00x slower 601 us: 1.04x faster
xml_etree_parse 250 ms 244 ms: 1.02x faster 246 ms: 1.02x faster 241 ms: 1.03x faster
xml_etree_iterparse 158 ms 152 ms: 1.04x faster not significant 153 ms: 1.03x faster
xml_etree_generate 164 ms 162 ms: 1.02x faster 167 ms: 1.02x slower 158 ms: 1.04x faster
xml_etree_process 138 ms not significant not significant 137 ms: 1.01x faster
Geometric mean (ref) 1.01x faster 1.01x faster 1.02x faster

Top comments (0)

Take a look at this:

Settings

Go to your customization settings to nudge your home feed to show content more relevant to your developer experience level. πŸ›