Building a Ruby C Extension From Scratch

In this edition of Ruby Magic, we'll show you how to use code written in C from Ruby. This can be used to optimize performance sensitive parts of your code or to create an interface between a C library and Ruby. This is done by creating extensions that wrap libraries written in C.

There are a lot of mature and performant libraries written in C. Instead of reinventing the wheel by porting them we can also leverage these libraries from Ruby. In this way, we get to code in our favorite language, while using C libraries in areas where Ruby isn't traditionally strong. At AppSignal, we've used this approach in developing the rdkafka gem.

So let's see how one can approach this. If you want to follow along and experiment yourself, check out the example code. To start off, let's take this piece of Ruby code with a string, a number and a boolean (you'll C why, pun intended) and port it to a C library:

module CFromRubyExample
  class Helpers
    def self.string(value)
      "String: '#{value}'"
    end

    def self.number(value)
      value + 1
    end

    def self.boolean(value)
      !value
    end
  end
end

In order, the methods shown concatenate a string, increase a number by one and return the opposite of a boolean, respectively.

Our Library Ported to C

Below, you can see the code ported to C. The C Standard Library and the IO Library are included so that we can use string formatting. We use char* instead of a Ruby String. char* points to the location of a buffer of characters somewhere in memory.

# include <stdlib.h>
# include <stdio.h>

char* string_from_library(char* value) {
  char* out = (char*)malloc(256 * sizeof(char));
  sprintf(out, "String: '%s'", value);
  return out;
}

int number_from_library(int value) {
  return value + 1;
}

int boolean_from_library(int value) {
  if (value == 0) {
    return 1;
  } else {
    return 0;
  }
}

As you can see, you have to jump through some hoops to do simple string formatting. To concatenate a string, we first have to allocate a buffer. With this done, the sprintf function can then write the formatted result to it. Finally, we can return the buffer.

With the code above, we already introduced a possible crash or security issue. If the incoming string is longer than 245 bytes, the dreaded buffer overflow will occur. You should definitely be careful when writing C, it's easy to shoot yourself in the foot.

Next up is a header file:

char* string_from_library(char*);
int number_from_library(int);
int boolean_from_library(int);

This file describes the public API of our C library. Other programs use it to know which functions in the library can be called.

The 2018 Way: Use the ffi Gem

So, we now have a C library that we want to use from Ruby. There are two ways to wrap this C code in a gem. The modern way involves using the ffi gem. It automates many of the hoops we have to jump through. Using ffi with the C code we just wrote looks like this:

module CFromRubyExample
  class Helpers
    extend FFI::Library

    ffi_lib File.join(File.dirname(__FILE__), "../../ext/library.so")

    attach_function :string, [:string], :string
    attach_function :number, [:int], :int
    attach_function :boolean, [:int], :int
  end
end

For the purpose of this article, we're also going to explain how to wrap the C code with a C extension. This will give us much more insight into how it all works under the hood in Ruby.

Wrapping our Library in a C Extension

So we now have a C library we want to use from Ruby. The next step is to create a gem that compiles and wraps it. After creating the gem, we first add ext to the require_paths in the gemspec:

Gem::Specification.new do |spec|
  spec.name          = "c_from_ruby_example"
  # ...
  spec.require_paths = ["lib", "ext"]
end

This informs Rubygems that there is a native extension that needs to be built. It will look for a file called extconf.rb or a Rakefile. In this case, we added extconf.rb:

require "mkmf"

create_makefile "extension"

We require mkmf, which stands for "Make Makefile". It's a set of helpers included with Ruby that eliminates the finicky part of getting a C build set up. We call create_makefile and set a name for the extension. This creates a Makefile which contains all the configuration and commands to build the C code.

Next, we need to write some C code to connect the library to Ruby. We'll create some functions that convert C types such as char* to Ruby types such as String. Then we'll create a Ruby class with C code.

First off, we include some header files from Ruby. These will import the functions we need to do type conversion. We also include the library.h header file that we created earlier so that we can call our library.

#include "ruby/ruby.h"
#include "ruby/encoding.h"
#include "library.h"

We then create a function to wrap each function in our library. This is the one for string:

static VALUE string(VALUE self, VALUE value) {
  Check_Type(value, T_STRING);

  char* pointer_in = RSTRING_PTR(value);
  char* pointer_out = string_from_library(pointer_in);
  return rb_str_new2(pointer_out);
}

We first check if the Ruby value coming in is a string, since processing a non-string value might cause all sorts of bugs. We then convert the Ruby String to a char* with the RSTRING_PTR helper macro that Ruby provides. We can now call our C library. To convert the returned char*, we use the includes rb_str_new2 function. We'll add similar wrapping functions for number and boolean.

For numbers, we do something similar using the NUM2INT and INT2NUM helpers:

static VALUE number(VALUE self, VALUE value) {
  Check_Type(value, T_FIXNUM);

  int number_in = NUM2INT(value);
  int number_out = number_from_library(number_in);
  return INT2NUM(number_out);
}

The boolean version is also similar. Note that C doesn't actually have a boolean type. The convention is to instead use 0 and 1.

static VALUE boolean(VALUE self, VALUE value) {
  int boolean_in = RTEST(value);
  int boolean_out = boolean_from_library(boolean_in);
  if (boolean_out == 1) {
    return Qtrue;
  } else {
    return Qfalse;
  }
}

Finally, we can wire up everything so that we can call it from Ruby:

void Init_extension(void) {
  VALUE CFromRubyExample = rb_define_module("CFromRubyExample");
  VALUE NativeHelpers = rb_define_class_under(CFromRubyExample, "NativeHelpers", rb_cObject);

  rb_define_singleton_method(NativeHelpers, "string", string, 1);
  rb_define_singleton_method(NativeHelpers, "number", number, 1);
  rb_define_singleton_method(NativeHelpers, "boolean", boolean, 1);
}

Yes, you read that right: we can create Ruby modules, classes and methods in C. We set up our class here. We then add Ruby methods to the class. We have to provide the name of the Ruby method, the name of the C wrapper function that will be called and indicate the number of arguments.

After all that work, we can finally call our C code:

CFromRubyExample::NativeHelpers.string("a string")

Conclusion

We jumped through hoops, didn't crash and got our C extension to work. Writing C extensions is not for the faint of heart. Even when using the ffi gem you can still quite easily crash your Ruby process. But it is doable and can open up a world of performant and stable C libraries for you!