loading...

Ruby GUI progress: Leaking worlds, naming is hard

roryo profile image Rory O'Connell ・6 min read

First and most importantly this project progressed enough that it now needs a catchy, searchable name. I have a couple candidates that are fine though they don't feel right. There are Latin words pyrope which is a kind of red or rust colored gem though inferior to a ruby gem; and punicus which sounds better though it has multiple meanings. Along with the color bright red it may also relate to old Carthage and the Roman bloodlust for Carthaginian genocide which isn't what I want to evoke.

Now the less important stuff

(dev.to embedded gifs aren't functioning at the moment)

https://dev-to-uploads.s3.amazonaws.com/i/7afo799od7nrmgegw90v.gif

I've made a tremendous amount of progress in a few days it's hard to talk about where things are at. I'll detail the most interesting thing I came across the past few days, a few quick hits and then where I'm going next.

I'm using the Dear ImGUI library for creating the GUI.

GitHub logo ocornut / imgui

Dear ImGui: Bloat-free Immediate Mode Graphical User interface for C++ with minimal dependencies

dear imgui

Build Status

(This library is available under a free and permissive license, but needs financial support to sustain its continued improvements. In addition to maintenance and stability there are many desirable features yet to be added. If your company is using dear imgui, please consider reaching out.)

Businesses: support continued development via invoiced technical support, maintenance, sponsoring contracts
  E-mail: contact @ dearimgui dot org

Individuals: support continued maintenance and development with PayPal.


Dear ImGui is a bloat-free graphical user interface library for C++. It outputs optimized vertex buffers that you can render anytime in your 3D-pipeline enabled application. It is fast, portable, renderer agnostic and self-contained (no external dependencies).

Dear ImGui is designed to enable fast iterations and to empower programmers to create content creation tools and visualization / debug tools (as opposed to UI for the average end-user). It favors simplicity and productivity toward this goal, and…

It's a tiny easily embedded interface library written in an orthodox C++, or C+, style. That is, almost entirely functions and structs without any of the C++ nightmares. Im means immediate mode GUI, in contrast to retained mode. The primary difference between the two is what controls the state of the interface. Most people are used to working with something that looks like a retained mode GUI. The user of a retained mode interface notifies an underlying system of changes. When notifying GUI system of changes the GUI library changes internal data structures and rebuilds lists of commands for drawing the interface. Working with the browser DOM is a great example of retained mode style. We tell the browser presentation engine when things change by modifying the DOM elements, not by telling the browser how to render a page directly.

Immediate mode style GUI does away with that state and hands the user of the API complete control of how to draw all the elements. Instead of notifying a system when state changes, with an immediate mode design the GUI code you write is instead executed linear sequence every frame. It's best shown with an example, here from the Dear ImGUI demonstration

if (ImGui::BeginMenuBar())
{
    if (ImGui::BeginMenu("Menu"))
    {
        ShowExampleMenuFile();
        ImGui::EndMenu();
    }
    if (ImGui::BeginMenu("Examples"))
    {
        ImGui::MenuItem("Main menu bar", NULL, &show_app_main_menu_bar);
        ImGui::MenuItem("Console", NULL, &show_app_console);
        ImGui::MenuItem("Log", NULL, &show_app_log);
        ImGui::MenuItem("Simple layout", NULL, &show_app_layout);
        ImGui::MenuItem("Property editor", NULL, &show_app_property_editor);
        ImGui::MenuItem("Long text display", NULL, &show_app_long_text);
        ImGui::MenuItem("Auto-resizing window", NULL, &show_app_auto_resize);
        ImGui::MenuItem("Constrained-resizing window", NULL, &show_app_constrained_resize);
        ImGui::MenuItem("Simple overlay", NULL, &show_app_simple_overlay);
        ImGui::MenuItem("Manipulating window titles", NULL, &show_app_window_titles);
        ImGui::MenuItem("Custom rendering", NULL, &show_app_custom_rendering);
        ImGui::MenuItem("Documents", NULL, &show_app_documents);
        ImGui::EndMenu();
    }
    if (ImGui::BeginMenu("Tools"))
    {
        ImGui::MenuItem("Metrics", NULL, &show_app_metrics);
        ImGui::MenuItem("Style Editor", NULL, &show_app_style_editor);
        ImGui::MenuItem("About Dear ImGui", NULL, &show_app_about);
        ImGui::EndMenu();
    }
    ImGui::EndMenuBar();
}

Reading this top to bottom you know exactly what's going on. Two points of interest. One, the if() calls around ImGUI functions generally mean 'if this element is visible'. If that element is completely obscured in some way it doesn't execute that code path. Amazingly simple! The other interesting point is Dear ImGUI elements usually take a boolean parameter. This is a reference to an address of memory representing if the element should be displayed or disabled. The interface for creating a new window is ImGui::Begin("Ruby Eval", &ruby_eval_open). The second parameter becomes false when the user closes the window. We can re-open the window whenever we want by setting ruby_eval_open to true.

(This pattern works fantastically in C and comes back to bite us when we're in Ruby land. Discussions for another day)

Casey Muratori describes the differences and benefits of an immediate mode in great detail here

Setting aside HN style nonsense on the computational efficiency or wastefulness of this pattern, the benefit to this is we have full control over what we want to draw when and the clear simplicity of it.

Back to making the GUI work with Ruby. It's a straightforward process making an mruby class in C in which calling the methods on the class in Ruby execute C code. For now I'm generally mapping the ImGUI functions to a GUI module one to one.

In a World...

The main program loop is in C. It looks something like this, abbreviated

while(!done) {
  process_window_messages();
  begin_new_frame();
  mrb_load_string(mrb, "World.render");
  render_frame();
}

The important bit here is mrb_load_string. It's essentially an eval into the mruby state. Surrounding the call processes incoming Windows messages, sets up the next frame, then renders the frame. Presently World.render is the entry point for each frame from the Ruby side. World is another simple construction. It maintains a list of all the open windows. When we call World.render it iterates over all of the open windows calling render on each.

module World
  @open_windows = []

  def self.add_window window
    raise unless window.respond_to? :render

    @open_windows.push window unless @open_windows.include? window
  end

  def self.remove_window window
    @open_windows.delete window
  end

  def self.render
    # TODO see if current active window respond_to? menu_bar,
    # otherwise render this
    GUI.main_menu do
      GUI.menu("Tools") do
        GUI.menu_item "Class Browser", "Ctrl+I" do
          ClassBrowser.open
        end

        GUI.menu_item "Open Transcript", "Ctrl+T" do
          Transcript.open
        end

        GUI.menu_item "Open Stats", "" do
          Stats.open
        end
      end
    end

    @open_windows.each { |window| window.render }
  rescue => ex
    puts ex
  end
end

When you want to open a window, call World.add_window thing. Closing it is World.remove_window thing. Simple and straight forward.

Implementing a render method is a simple set of ImGUI calls. For example, this is the full Transcript, a global console-like thing where you can print stuff to

module Transcript
  @lines = []

  def self.add obj
    @lines.push obj.to_s
  end

  def self.clear
    @lines = []
  end

  def self.open
    World.add_window self
  end

  def self.close
    World.remove_window self
  end

  def self.render
    GUI.new_window 'Transcript' do
      @lines.each { |line| GUI.text line }
    end
  end
end

This all works fine and I was reasonably satisfied with it so far. Until one time I left the program open for several hours I noticed the program acquired multiple gigabytes of working memory. It usually allocates about 300kb total. Classic memory leak somewhere. Getting to the heart of why is a long saga detailed in this short Github exchange.

Effective ways on running Ruby code from within C without leaking Proc objects? #5001

RoryO avatar
RoryO posted on

Short summary

mrb_load_string, related functions and the typical run mruby code from C patterns create an RProc object on every call which is never marked for GC. There's an easy reproduction case with a simple C program

#include "stdio.h"
#include "ext/mruby.h"
#include "ext/mruby/compile.h"
#include "ext/mruby/gc.h"

int main() {
    mrb_state *mrb = mrb_open();
    for(int i = 0; i < 100000; ++i) {
        mrb_load_string(mrb, "nil");
        if(i % 100 == 0) {
            printf("%zi\n", mrb->gc.live);
        }
    }
    
    return 0;
}

Longer explanation

I'm working on embedding mruby in a GUI. The C portion of program sets up all the initial rendering, then passes control to the mruby VM, then renders the frame after the VM finishes. I call the ruby VM exactly once each frame from C with one entry point. My main program loop looks roughly like

while(!done) {
  setup_frame();
  mrb_load_string(mrb, "World.render");
  render_frame();
}

And this works wonderfully! The fast C code path does the intensive graphics calculations and rendering, and I have one entry point which kicks off processing the VM every 'tick' with World.render Leaving out a lot of unnecessary surrounding context here to get at the heart of the matter.

After being so happy for a couple days on how easy this was going I left my program overnight, coming back the program taking up multiple gigabytes of memory where it usually only takes a few hundred kb. Tell tale memory leak of some kind. I did a lot of digging within my own C code as that's the most likely answer and came up with nothing.

Looking at the live object count every frame I noticed the live object count never decreases. I started digging in the Ruby code thinking I created object islands and came up with nothing. Finally I changed my loop to do mrb_load_string(mrb, "nil"); and much to my surprise the object count increased linearly infinitely just as before when the mruby VM shouldn't do anything at all!

Digging some more into ObjectSpace while simply calling nil every frame I found that T_PROC objects grow indefinitely. After a few minutes of running

{:TOTAL=>73728, 
:FREE=>206, 
:T_OBJECT=>2, 
:T_CLASS=>59, 
:T_MODULE=>10, 
:T_ICLASS=>14, 
:T_SCLASS=>75, 
:T_PROC=>73285, 
:T_ARRAY=>2, 
:T_HASH=>2, 
:T_STRING=>23, 
:T_EXCEPTION=>2, 
:T_ENV=>44, 
:T_DATA=>3, 23=>1}

Looking at the definition of mrb_load_string and others related it it's usually the same sequence of function calls: creating a new parser context, parsing the string, generating a proc object, run the code in the vm, free the parser and context. I replicated this replacing mrb_load_string in my loop

mrbc_context* ctx = mrbc_context_new(state->ruby_state);
mrb_parser_state *parser = mrb_parse_string(state->ruby_state, "World.render", ctx);
RProc *proc = mrb_generate_code(state->ruby_state, parser);
mrb_value retval = mrb_vm_run(state->ruby_state, proc, mrb_top_self(state->ruby_state), 0);

mrb_pool_close(parser->pool);
mrbc_context_free(state->ruby_state, ctx);

And realized there are the infinitely growing Proc objects created. Digging through old issues I had the idea of calling mrb_gc_arena_save/mrb_gc_arena_restore before and after the loop... and this works! The object count stabilizes over time. Hurray, problem solved!

However I'm not completely satisfied. Going back to the old 'principle of least surprise' I found it highly surprising that the typical patterns used in functions like mrb_load_string leak objects.

Some questions

  • It feels like a mistake that functions like mrb_load_string and it's relatives leak Proc objects on every call. Should we do a gc save/restore within these functions?
  • Is there a better way to invoke a Ruby method from C without creating a new RProc object?
  • Alternately, is there a way we could mark the RProc used for execution as black immediately after running it in the VM so it becomes GC'd next cycle? This feels like it's crossing too many boundaries however.

The short answer is this innocent looking C call

mrb_load_string(mrb, "World.render");

Which it turns out any C function named mrb_load_* creates an RProc object in the current mruby state. However, that RProc is not garbage collected at any point! For reasons I sort understand this document details the situation. However it feels surprising that the mrb_load_* function calls don't automatically protect against that situation.

Going back to the main loop, we call mrb_load_string every frame, yielding to the World object for rendering all the windows. That generates an RProc object in the VM which the garbage collector never collects. We do the at least 60 times a second, so we're leaking 60 RProc objects a second. A basic Ruby object is tiny, 36 bytes or so which is why this behavior was not noticeable.

I've since added basic GC stats into the interface so I can observe such things

https://dev-to-uploads.s3.amazonaws.com/i/jam8uak08klab6mwbvum.gif

Next up

As it stands currently this is primarily a C program driving the mruby state as shown by the main program loop is in C. I want to flip this around where the main program loop is in Ruby which calls back to C code when necessary. I want the C main function to look something like

int main() {
  setup_state();
  mrb_load_string(mrb, "GUI.start");
  cleanup();
  return 0;
}

And then the Ruby side looking something like

class GUI
  def self.start
    while is_running do
      process_window_events
      begin_next_frame
      render
      present_frame
    end
  end
end

I think this will take a significant amount of work and I know it will pay off later. Reading through past mruby issues and recommendations by matz the general solution to most of the integration problems is 'push what you're doing into the mruby VM instead of C'.

I didn't think of moving the loop into ruby this way until I saw this demonstration of a madman using mruby for creating a Sega Dreamcast game. This combination of esoteric things, homebrew Dreamcast development and mruby, make me quite happy it exists.

mrbtris: Sample game for Sega Dreamcast written in Ruby

mrbtris is a simple game for the Sega Dreamcast which is written in Ruby as an proof-of-concept of using the mruby implementation on the Sega Dreamcast. This project was written by @yujiyokoo.

This project is built on the top of KallistiOS (KOS), which is the low-level library used to interact with the Sega Dreamcast hardware. Usually, programs written for the Sega Dreamcast are in C/C++, this project aims to demonstrate the use of Ruby source code targeting the Sega Dreamcast.

This project aims to provide a simple example of how to use KallistiOS (KOS) API and mruby together.

Demonstration

Below you may find a video of this game running on the real hardware #mrbtris running on Sega Dreamcast

TODO

  • Make an mrbgem for the Dreamcast specific things
  • Create unit tests
  • Use more Sega Dreamcast features

Building

mrbtris uses KallistiOS (KOS) and mruby as dependencies…

Posted on by:

Discussion

pic
Editor guide