Create account

DEV Community

rhymes for The DEV Team

Posted on Jun 5, 2020

How we decreased our memory usage with jemalloc

#rails #performance #changelog

DEV is a Ruby on Rails application deployed on Heroku servers.

The other day Molly Struve, our resident SRE sorceress, noticed we were having some memory troubles. They didn't look like memory leaks, since they plateaued over time. But they were obviously worth investigating.

After investigating a bit I remembered how in the past I happened to switch a Rails app from the standard memory allocator to jemalloc.

The jemalloc library is an alternative memory allocator that can be used by apps (Redis ships with it) which works better with memory fragmentation in a multithreading environment.

We researched that, activated it and this was the result:

The slight decrease you see before the red bar is due to deployments which for obvious reasons free some occupied memory.

How do I do the same in my Rails app(s)?

The prerequisite is that you're using multiple threads with Rails, otherwise it won't help much.

If you're using Heroku you can follow these instructions.

If not, you can take inspiration from this tutorial and recompile your Ruby and activate jemalloc support.

Why does this happen?

Nate Berkopec has an extensive explanation in his post Malloc Can Double Multi-threaded Ruby Program Memory Usage but long story short: the Ruby virtual machine and the default memory allocator (malloc) aren't great at talking to each other when multithreading is involved because of the way they are both designed.

Are there any downsides?

Potentially yes, as it would make your app less portable (it doesn't work on Windows) and there are options with a smaller footprint that are being explored by Ruby core, like using malloc_trim().

My suggestion is to test it :-)

Top comments (6)

Yaser Al-Najjar • Jun 5 '20 • Edited

Great job Rhymes and Molly!

I would also suggest few things to track down the issue further:

Creating a rails app with just one endpoint and all the gems used in DEV, to determine the normal from the abnormal memory usage.
Using a memory profiler and determining which endpoints account for the majority of the memory usage.
After pinpointing the bottlenecks, you might consider switching their implementations into a faster one like C/Go/Rust (I suspect the serialization and deserialization are the culprits).
All that above matters if there is some real financial pain, otherwise it's not worth pursuing 😁

rhymes • Jun 5 '20

Thanks a lot for the suggestions!

The gem derailed_benchmarks also helps:

schneems / derailed_benchmarks

Go faster, off the Rails - Benchmarks for your whole Rails app

Derailed Benchmarks

A series of things you can use to benchmark a Rails or Ruby app.

Compatibility/Requirements

This gem has been tested and is known to work with Rails 3.2+ using Ruby 2.1+. Some commands may work on older versions of Ruby, but not all commands are supported.

For some benchmarks, not all, you'll need to verify you have a working version of curl on your OS:

$ which curl
/usr/bin/curl
$ curl -V
curl 7.37.1 #

Install

Put this in your gemfile:

gem 'derailed_benchmarks', group: :development

Then run $ bundle install.

While executing your commands you may need to use bundle exec before typing the command.

To use all profiling methods available also add:

gem 'stackprof', group: :development

You must be using Ruby 2.1+ to install these libraries. If you're on an older version of Ruby, what are you waiting for?

Use

There are…

View on GitHub

Zubair Mohsin • Jun 5 '20

the memory management API looks very much like libc's malloc implementation

Looks like PHP internally uses malloc implementation?

php.net/manual/en/internals2.memor...

rhymes • Jun 5 '20

I'm not 100% sure about it, as I'm unfamiliar with PHP. I went as far as this file github.com/php/php-src/blob/master... which contains the declaration of the list of allocators you linked.