This article is a translation of bmf-tech.com - GoのHTTP Routerを比較するベンチマーカーを実装した.

This is day 5 of Makuake Advent Calendar 2022!

Overview

Bench markers were implemented to compare the performance of Go's HTTP Router.

bmf-san/go-router-benchmark

The following HTTP Routers are currently being compared.

• bmf-san/goblin
• julienschmidt/httprouter
• go-chi/chi
• gin-gonic/gin
• uptrace/bunrouter
• dimfeld/httptreemux
• beego/mux
• gorilla/mux
• nissy/bon
• naoina/denco
• labstack/echo
• gocraft/web
• vardius/gorouter
• go-ozzo/ozzo-routing
• lkeix/techbook13-sample

In some test cases, the Go standard net/http#ServeMux is also covered.

Motivation

I have created my own HTTP Router called bmf-san/goblin.

bmf-san/goblin is a simple HTTP Router based on the Trie tree with minimal functionality.

By comparing the performance of bmf-san/goblin with other HTTP Routers, bmf-san/goblin goblin), the motivation is to get hints on how to improve bmf-san/goblin.

Another reason is that I have my own bench marker that can replace julienschmidt/go-http-routing-benchmark. Another motivation was to be able to maintain

The julienschmidt/go-http-routing-benchmark is the julienschmidt/httprouter, but maintenance seemed to have stopped in recent years, so I decided to create my own benchmarker and implement it. I decided to implement bench markers.

About the test design of the bench marker

As a premise, I would like to state that bmf-san/go-router-benchmark is not a complete comparison of HTTP Router performance .

The reasons are as follows.

• Each HTTP Router has different functions and specifications, and it is not realistic to prepare a test case that perfectly covers all of them, so only some of the specifications are taken into account in the comparison.
• Because each HTTP Router has its own data structure and algorithms, which may or may not be good or bad, and therefore the defined routing test cases may not fully evaluate the performance of each HTTP Router.

Therefore, the benchmark test will be a test case for a specific function or specification of HTTP Router only, but it can measure a certain difference in performance.

In bmf-san/go-router-benchmark, the performance is measured for the processing part of routing.
Specifically, it tests the ServeHTTP function in http#Handler.

benchmark_test.go#L21

The part that defines routing is not tested.
The process of defining routing is the process of registering data necessary for the routing process.

package main

import (
    "fmt"
    "net/http" )
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Hello World")
}

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("/", handler) // here
    ListenAndServe(":8080", mux)
}

The following two test cases are available for the routing process.

• Static routes
• Routes with path parameters

Each test case is described below.

Static routes

A static route is a route without variable parameters such as /foo/bar.

The following four input patterns are available for testing this route.

• /
• /foo
• /foo/bar/baz/qux/quux
• /foo/bar/baz/qux/quux/corge/grault/garply/waldo/fred

This test case also compares the Go standard net/http#ServeMux.

Routes with path parameters

A route with path parameters is a route with variable parameters such as /foo/:bar.

In this route test, the following three input patterns are provided.

• /foo/:bar.
• /foo/:bar/:baz/:qux/:quux/:corge.
• /foo/:bar/:baz/:qux/:quux/:corge/:grault/:garply/:waldo/:fred/:plugh

Since some HTTP Routers have different syntaxes for variable parameters, we also take into account the corresponding syntaxes for each.

ex.pathparam.go#L15

Benchmark test results

Benchmark test results are available at go-router-benchmark.

The benchmark test environment is as follows

• go version: go1.19
• goos: darwin
• goarch: amd64
• pkg: github.com/go-router-benchmark
• cpu: VirtualApple @ 2.50GHz

The benchmark results are as follows

• time
  • Number of function executions
  • The higher the number of executions, the better the performance.
• ns/op
  • Time required per function execution
  • The smaller the time, the better the performance
• B/op
  • Size of memory allocated per function execution
  • The smaller the size, the better the performance
• allocs/op
  • Number of memory allocations made per function execution
  • The fewer the number of memory allocations made per execution of the allocs/op function, the better the performance

cf. dev.to - Improving the performance of your code starting with Go

Describe the results of each test case.

Static routes

For static routes, one point of comparison seems to be whether the performance is better than or equal to the standard net/http#ServeMux.
HTTP Router, which is touted for its better performance, is still better than the standard.

time

Graph - time

nsop

Graph - nsop

bop

Graph - bop

allocs

Graph - allocs

Routes with path parameters

For routes with path parameters, we split the performance into two groups: one with a large degradation in performance as the number of parameters increases, and the other with a modest degradation.

time

Graph - time

nsop

Graph - nsop

bop

Graph - bop

allocs

Graph - allocs

Conclusion.

It can be seen that the HTTP Router with better performance shows less performance degradation in each test case.
This seems to be a clear trend indicating that the implementations are optimized.

When examining some of the better performing HTTP Router implementations, we find that they employ a more sophisticated tree structure.
For example, Echo, gin, httprouter, bon, and chi use the Radix tree (Patricia trie), while denco uses a double array.

Regarding bmf-san/goblin, we found that it is a proprietary extension of the tri-tree, which is not very optimized and has poor performance compared to other HTTP Routers. (We will do our best to improve it...)

On the other hand, it seemed possible that the multifunctionality of some of the HTTP Routers that seemed to perform poorly may have reduced performance.

We felt that we could further obtain performance trends for each HTTP Router by adding test cases, which we will try to address if we have time.