Benchmarking Ruby Code

This post originally appeared on the Scout blog.

One of the joys of using the Ruby language is the many different ways that you can solve the same problem, it’s a very expressive language with a rich set of libraries. But how do we know which is the best, most efficient, use of the language? When we are talking about algorithms which are critical to the performance of your application, understanding the most efficient approach to take is essential. Perhaps you’ve been using Scout APM to hunt down issues, and now that you have found an issue, you want to optimize it. Ruby’s Benchmark module provides a very handy way for you to compare and contrast possible optimizations, and when used in conjunction with a good APM solution it will ensure that you have all bases covered. Let’s take a look at how you can get started with it today!  

Ruby’s Benchmarking Module

Ruby provides a nice little module called Benchmark that is sufficient for most people's needs. It allows you to execute blocks of code and report their execution time, allowing you to see which method is fastest. All you have to do is include the Benchmark module like this:

require 'benchmark'

There are several different methods you can use which we will cover shortly, but first of all, regardless of which method you choose, the benchmarking results that Ruby will give you will be provided in the form of a Benchmark::Tms object and they look like this:

 

user        system        total        real

5.305745    0.000026      5.305771     5.314130

The first number shows the user’s CPU time, in other words, how long it took to execute the code, ignoring system-level calls. This is followed by the system CPU time, which shows time that was spent in the kernel, perhaps executing a system call. The third number is a total of both these values, and the last number shows the real execution time if you timed the operation from start to finish.

Measuring a single block of code

If you want to measure the performance of a single block of code, and you are not interested in comparing multiple approaches, then you can use the Benchmark::measure method.

result = Benchmark.measure do

  10000.times { x = "lorem ipsum" }

end

puts result

The measure method (like all the methods in this module) returns the result in the Benchmark::Tms format which you can print easily. Remember that the block of code can be anything, such as a method call etc.

Comparing multiple blocks of code

When you want to compare two or more different approaches to a problem to determine which is the best approach, then you can use Benchmark::bm method. This method is an easier-to-use version of the Benchmark::benchmark method.

In the code snippet below, we define two methods with slightly different approaches: one uses a for loop and one uses the times method. By using Benchmark::bm we can call the report method, passing a value for the label and the block of code that we want benchmarked.

def old_method
  for i in 1..100_000_000
    x = "lorem ipsum"
  end
end

def new_method
  100_000_000.times { x = "lorem ipsum" }
end

Benchmark.bm do |x|
  x.report("Old:") { old_method }
  x.report("New:") { new_method }
end

This is the output that is generated:

        user        system        total        real

Old:    5.305745    0.000026      5.305771     (  5.314130)

New:    5.084740    0.000000      5.084740     (  5.092787)

In situations where the results may be unfair because of garbage collection overheads, another, similar method, Benchmark::bmbm is provided. This method benchmarks twice, displaying the output for both passes. As you can see in the example below, execution was faster on the second test.

Rehearsal ----------------------------------------

Old:   5.470750   0.000000 5.470750 (  5.479510)

New:   5.131210   0.000000 5.131210 (  5.139404)

------------------------------ total: 10.601960sec

       user       system   total    real

Old:   5.432181   0.000000 5.432181 (  5.440627)

New:   5.061602   0.000000 5.061602 (  5.069408)

Conclusion

As you can see, Ruby’s Benchmark module is easy-to-use and the insights that it can give us can be incredibly useful. Once you’ve got your feet wet with the standard Benchmark module, you can take it a step further with the benchmarking-ips gem (which shows iterations per second information) or the Readygo gem (which takes a more unique approach to benchmarking). Or if you’d rather just reap the rewards of somebody else’s benchmarking of Ruby’s many features, then this page is an interesting read.