Using libffi

While working on an addon to add universe support to node, I ran into the problem of trying to pass an array of arguments to a function that takes variadic arguments. This was a bit hard to google as I didn't know what exactly I was trying to do but eventually found a stackoverflow question that was asking the same thing. The answer was it couldn't be done but someone did mention that you can do with libffi.

ic_subcall takes in 4 parameters that are known and then a variable set of parameters at the end. These are the arguments that get passed to universe as subroutine arguments.

void ic_subcall ic_proto((LPSTR, LPLONG, LPLONG, LPLONG, ...));

Now for javascript to be able to have the same ability, I would need to be able to pass in as many arguments as I want. Using the node addon api this is pretty straightforward as you can access node variables in C via the CallbackInfo array.

universe.CallSubroutine("NIVT", arg1, arg2, arg3);

This will the populate an info variable in C. The hard part is getting this into a form that we can call ic_subcall. One quick solution would be to simply count how many variables javascript passed over and then write a different ic_subcall for each length.

I didn't like this idea as that just leads to duplicate code and is limited by my imagination.

With libffi, I can convert all of the arguments to live in a list and then call the function and pass in the parameters. libffi from what I understand will then generate the function for me with the variable number of arguments.

This worked perfectly though it was a bit of a pain to figure out how to get everything to work together. The big thing with using libffi for this was getting a proper handle on types and referencing the data.

This code is from my node-pick-universe project and so it could be helpful to see it in that context.

https://github.com/Krowemoh/node-pick-universe

All the code is located in src/universe.cc.

double call_subroutine(char *subname, long numargs, ICSTRING *icList) {
    int pad = 4;
    int arg_len = pad + numargs;

    ffi_cif     call_interface;
    ffi_type    *ret_type;
    ffi_type    *arg_types[arg_len];

    ret_type = &ffi_type_double;

    ffi_type icstring_type;
    ffi_type *icstring_type_elements[3];

    icstring_type.size = icstring_type.alignment = 0;
    icstring_type.type = FFI_TYPE_STRUCT;
    icstring_type.elements = icstring_type_elements;

    icstring_type_elements[0] = &ffi_type_slong;
    icstring_type_elements[1] = &ffi_type_pointer;
    icstring_type_elements[2] = NULL;

    arg_types[0] = &ffi_type_pointer;
    arg_types[1] = &ffi_type_pointer;
    arg_types[2] = &ffi_type_pointer;
    arg_types[3] = &ffi_type_pointer;

    for (int i=0;i <numargs; i++) {
        arg_types[pad+i] = &ffi_type_pointer;
    }

    if (ffi_prep_cif(&call_interface, FFI_DEFAULT_ABI, arg_len, ret_type, arg_types) == FFI_OK) {
        void *arg_values[arg_len];

        char **subname_pointer = &subname;
        arg_values[0] = subname_pointer;

        long size = strlen(subname);
        long * size_pointer = &size;
        arg_values[1] = &size_pointer;

        long status = 0;
        long * status_pointer = &status;
        arg_values[2] = &status_pointer;

        long * numargs_pointer = &numargs;
        arg_values[3] = &numargs_pointer;

        ICSTRING *ptrs[numargs];

        for (int i=0;i <numargs; i++) {
            ptrs[i] = &icList[i];
            arg_values[pad+i] = &ptrs[i];
        }

        double z = 0;
        ffi_call(&call_interface, FFI_FN(ic_subcall), &z, arg_values);
        return z;
    }
    return -1;
}

The first thing we do is set up the ffi stuff. We create an array of types and begin filling in the structure of the parameters we want to pass in. ic_subcall takes in 4 parameters at the beginning and so those are hardcoded. The variable number of parameters are then at the end.

Next thing I do is create an ffi_type for ICSTRING. ICSTRING is an InterCall type and so doesn't exist from libffi's point of view. Luckily creating types is very straightforward.

I then use loops to add however many icstring_types I need to the list of arg_types.

If the typing is valid, I can now begin building up a list of values. The list of values is a void* array which means that it contains references to all of the parameters I want to pass to ic_subcall.

Once again, I hardcoded the first 4 parameters and then use a loop to add the arguments to the list.

Once the list is done, I can call use ffi_call to run the function and pass in my list of arguments.

Voila! We have now added the ability to call ic_subcall with as many arguments as we want from C and by extension node!

This was quite a bit of learning as dealing with typing and properly setting up the arg_values was the most error prone. I had trouble wrapping my head around what exactly libffi was looking for. The documentation for it is also a bit lacking, there are some really good examples but they are all pretty simplistic. The thing that helped the most was stripping down the code and actually dummying my own ic_subcall with the same function signature and playing around with trying to call it dynamically. Quit fun!

I read that libffi is often used by interpreters to provide this sort of function and I'm curious to see some real life examples so hopefully I'll dig into it at some point.

This was quite the fun project!