Building a Better Library Loader in C# - Part 1

Interop with native libraries is really important in game dev. There is always some driver or low level system library with no .NET support. C# offers up a rudimentary way to map and call functions in native libraries. Observe:

static class Sqlite3
{
    [DllImport("sqlite3.dll", EntryPoint = "sqlite3_open", CallingConvention = CallingConvention.Cdecl)]
    public static extern int Open(string filename, out IntPtr database);
}

The EntryPoint field is optional, but I like using it because I like mapping beautiful Pascal case C# code over whatever wonky style the C library chose. I really wish I could simply define the style conversion in a method rather than type out every single mapping.

Another issue is that the library must be specified at compile time. That's... OK, I guess... sometimes. What if I have multiple variants/architectures and want to select it at run-time? I could do something gross like rename sqlite3_x86.dll to sqlite3.dll before calling any of the native methods, but that's brittle and hacky. What if I'm not allowed to move the library? What if I need to determine the location of the library as part of my application startup? What if there are four possible choices I need to differentiate?

Manual Loader

Sadly, .NET does not really offer up a smooth way to load libraries at run-time. We have to manually load the library ourselves. These are the necessary methods:

public static class WinLoader
{
    private const string Library = "kernel32";

    [DllImport(Library)]
    public static extern IntPtr LoadLibrary(string dllToLoad);

    [DllImport(Library)]
    public static extern IntPtr GetProcAddress(IntPtr hModule, string procedureName);

    [DllImport(Library)]
    public static extern int FreeLibrary(IntPtr hModule);
}

public static class UnixLoader
{
    private const string Library = "dl";

    [DllImport(Library, EntryPoint = "dlopen")]
    public static extern IntPtr Open(string path, int mode);

    [DllImport(Library, EntryPoint = "dlsym")]
    public static extern IntPtr LoadSymbol(IntPtr handle, string name);

    [DllImport(Library, EntryPoint = "dlclose")]
    public static extern int Close(IntPtr handle);
}

A quick note: .NET Core 3 is introducing an abstraction over these functions called NativeLibrary. It will reduce the complexity of selecting functions based on operating system. Until that time, this is the code we need to manually load up one native function and call it. First we need to define a delegate that matches the structure of the native function we want to call.

public delegate int OpenDelegate(string file, out IntPtr database);

Then we need to actually load the delegate.

string library = "path/to/custom/sqlite3.dll";
bool isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
IntPtr libraryHandle = isWindows ?
    WinLoader.LoadLibrary(library) :
    UnixLoader.Open(library, 1);
IntPtr procAddress = isWindows ?
    WinLoader.GetProcAddress(libraryHandle, "sqlite3_open") :
    UnixLoader.LoadSymbol(libraryHandle, "sqlite3_open");

OpenDelegate open = Marshal.GetDelegateForFunctionPointer<OpenDelegate>(procAddress);
open("data.sqlite", out IntPtr database);
// Do stuff.

// When we are all done, we can free up the library.
if (isWindows)
    WinLoader.FreeLibrary(libraryHandle);
else
    UnixLoader.Close(libraryHandle);

This is a lot of work to load up and use one single native function. We don't even have error handling yet! For every single native function, I have to make a new delegate type (unless two functions happen to have identical signatures... good luck naming that delegate anything sane), load the function pointer, make a delegate instance to hold the function pointer, and put that delegate somewhere useful. Even in the small example above, how is anyone going to realistically use that open delegate???

I wish you luck in your journey.

Dream Loader

There has to be a better way. Before diving into the technical details, let's look at how we want to load libraries. Personally, I want to just map the library like this.

interface ISqlite3
{
    int Open(string file, out IntPtr database);
    int Close(IntPtr database);
}

Then, elsewhere, I want to just load and use it like this:

ISqlite3 sqlite3 = LoadLibrary<ISqlite3>("path/to/custom/sqlite3.dll");

sqlite3.Open("data.sqlite", out IntPtr db);
sqlite3.Close(db);

Wouldn't that be great? That is my kind of loader. Can it be done? Thanks to the magic of C#, the answer is yes. It requires venturing deep into the depths of System.Reflection. It's a scary place for the uninitiated, but I promise the journey is worth it. Stay with me. :)

Iterating Methods

Let's start at the beginning and define our loader method. Unfortunately, there is no way to constrain generics to require an interface, so we'll just check it at run-time.

public static T LoadLibrary<T>(string library) where T : class
{
    if (!typeof(T).IsInterface)
        throw new Exception($"Type T must be an interface. {typeof(T)} is not an interface.");

    foreach (var method in typeof(T).GetMethods())
        Console.WriteLine(method.Name); // Let's just see what we find.

    return null; // Patience. :)
}

See that call to GetMethods()? That lets us analyze the interface at run-time and take the first steps to implementing it. As you know, an interface needs a concrete type behind it in order to function. No such type exists... so we're going to make one... at run-time. System.Reflection lets us look at type information; System.Reflection.Emit lets us create types and dynamically write code.

One step at a time.

First, we need to solve a small problem: we need to associate C# method names with C function names. We could map them directly using attributes.

interface ISqlite3
{
    [FunctionName("sqlite3_open")]
    int Open(string file, out IntPtr database);

    [FunctionName("sqlite3_close")]
    int Close(IntPtr database);
}

I mean, that works, but, uh... again... have fun with that.

I shouldn't have to specify every function individually. There is a very clear pattern here: MethodName becomes sqlite3_method_name. So, let's augment our solution to allow a delegate for transforming method names to function names.

public static T LoadLibrary<T>(
    string library,
    Func<string, string> methodNameToFunctionName)
    where T : class
{
    if (!typeof(T).IsInterface)
        throw new Exception($"Type T must be an interface. {typeof(T)} is not an interface.");

    foreach (var method in typeof(T).GetMethods())
        Console.WriteLine(method.Name + " -> " + methodNameToFunctionName(method.Name));

    return null; // Patience. :)
}

Now we're talkin'. Now I don't have to map every single one. Laziness is king. Now, let's write a method to specifically handle the SQLite naming convention. It's fairly straightforward when you think about: the function always starts with sqlite3, and every capital letter needs to be replaced with an underscore and the lowercase equivalent.

public static string GetSqliteFunctionName(string methodName)
{
    var prefix = "sqlite3";
    var chars = new char[prefix.Length + methodName.Length * 2];
    prefix.AsSpan().CopyTo(chars);

    int n = prefix.Length;

    for (int i = 0; i < methodName.Length; ++i)
    {
        char c = methodName[i];

        if (char.IsUpper(c))
        {
            chars[n++] = '_';
            chars[n++] = char.ToLowerInvariant(c);
        }
        else
        {
            chars[n++] = c;
        }
    }

    // Real game devs avoid StringBuilder.
    return new string(chars, 0, n);
}

Now we're ready. Now we can load our library like this.

ISqlite3 sqlite3 = LoadLibrary<ISqlite3>("sqlite3.dll", GetSqliteFunctionName);

In part 2, we'll take the first steps to actually implementing a class that implements ISqlite3.