Jorrit

Posted on Aug 14, 2019

Rust -> wasm -> browser & nodejs

#rust #javascript #browser #node

When learning rust I found de rust-wasm "Hello World" example a little overwhelming (+ 350 npm packages) and with many extra tools (wasm-pack, cargo-generate, typescript and webpack). This is my effort in describing a trial and error process in how to build and use wasm with rust, why and how to use wasm-bindgen, with only the minimal requirements from scratch.

Starting

Creating a rust lib

cargo new libexample --lib

cd libexample

Building this lib for wasm is done by adding the target to cargo build command

cargo build --target wasm32-unknown-unknown --release

However it does not do anything yet, it is a starting point.
Open src/lib.rs and lets add 2 functions.

#[no_mangle]
pub fn add(a: i32, b: i32) -> i32 {
    a + b
}

// this function changes a value by reference (borrowing and change)
#[no_mangle]
pub fn alter(a: &mut [u8]) {
    a[1] = 12;
}

Now lets build the lib again. We somehow need to generate .wasm files.
Execute the build command and inspect

ls target/wasm32-unknown-unknown/release/deps

We need to indicate that we want a dynamic lib crate-type. Open Cargo.toml and add the following.

[lib]
crate-type = ["cdylib"]

Now re-build and copy the *.wasm file

cargo build --target wasm32-unknown-unknown --release
mkdir example
cp target/wasm32-unknown-unknown/release/deps/libexample.wasm example/.

We 'r gonna use this wasm file first in nodejs. Now create this script: example/nodeTest.js

const fs = require('fs');

const wasmBin = fs.readFileSync(__dirname + '/libexample.wasm');
const wasmModule = new WebAssembly.Module(wasmBin);
const libexample = new WebAssembly.Instance(wasmModule, []);

// Call wasm method 'add' typical stack method
let result = libexample.exports.add(10, 2);
console.log('add result:' + result);
// Now let's access heap memory /reference
// RuntimeError: unreachable
var a = new Uint8Array(100);
a[0] = 225;
a[1] = 10;
console.log('pre remote call a[1] === ' + a[1]);
libexample.exports.alter(a);
console.log('post remote call a[1] === ' + a[1]);

Run this script with node:

node example/nodeTest.js

Now it is getting interesting: the simple 'stack' type vars used in the 'add' method work like a charm ('add result'). It is not possible to change the value in the Uint8Array (heap memory) in this scenario. Therefore we need some extra steps this is where wasm_bindgen comes to play. Open Cargo.toml and add the following line:

[dependencies]
wasm-bindgen = "0.2"

Open src/lib.rs and change it like this:

extern crate wasm_bindgen;
use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn add(a: i32, b: i32) -> i32 {
    a + b
}

// this function changes a value by reference (borrowing and change)
#[wasm_bindgen]
pub fn alter(a: &mut [u8]) {
    a[1] = 12;
}

Now rebuild and copy the wasm-file

cargo build --target wasm32-unknown-unknown --release && cp target/wasm32-unknown-unknown/release/deps/libexample.wasm example/.

Exciting does nodejs run as expected?

node example/nodeTest.js

Probably not, wasm_bindgen changes the workflow quite a bit:

TypeError: WebAssembly.Instance(): Import #0 module="__wbindgen_placeholder__" error: module is not an object or function

Now, we need some extra steps: wasm-bindgen-cli

cargo install wasm-bindgen-cli

Takes some time with a lot of Compiling (some 181 crates)
Installed packagewasm-bindgen-cli v0.2.48(executableswasm-bindgen,wasm-bindgen-test-runner,wasm2es6js)
Time to see what is does:

wasm-bindgen target/wasm32-unknown-unknown/release/deps/libexample.wasm  --out-dir ./example

Wow, when looking in the example folder there are typescript @types and a ready to use libexample.js file. So this creates the javascript wrapper for us (ready to use?). Now lets change our example/nodeTest.js accordingly.

const libexample = require('./libexample.js');

// Call wasm method 'add' typical stack method
let result = libexample.add(10, 2);
console.log('add result:' + result);
// Now let's access heap memory /reference
var a = new Uint8Array(100);
a[0] = 225;
a[1] = 10;
console.log('pre remote call a[1] === ' + a[1]);
libexample.alter(a);
console.log('post remote call a[1] === ' + a[1]);

Lets run:

node example/nodeTest.js

Bad luck : nodejs doesn't allow import * from .. yet. We need to do something extra for nodejs. Add the --nodejs flag.

wasm-bindgen target/wasm32-unknown-unknown/release/deps/libexample.wasm --nodejs  --out-dir ./example

Now run :

node example/nodeTest.js

Yeah, the two methods are working as expected. Now lets create a browser integration. Create a folder www.

mkdir www

Run the following command the create te javascript wrapper for browser-based integration.

wasm-bindgen target/wasm32-unknown-unknown/release/deps/libexample.wasm --out-dir ./www

Create an www/index.html file like this.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Trial and error</title>
  </head>
  <body>
    <script src="./index.js"></script>
  </body>
</html>

Create the www/index.js file like this:

import * as libexample from "libexample"
// Call wasm method 'add' typical stack method
let result = libexample.add(10, 2);
console.log('add result:' + result);
// Now let's access heap memory /reference
var a = new Uint8Array(100);
a[0] = 225;
a[1] = 10;
console.log('pre remote call a[1] === ' + a[1]);
libexample.alter(a);
console.log('post remote call a[1] === ' + a[1]);

Now serve dewww folder via a/any http-server. and inspect the console output.
Stil no luck. Somehow there is a network error message disallowed MIME type (“application/wasm”). Wasm does not run this way in a browser via import * as example from 'file.wasm'. So I landed on webassembly.org and there it's very clear. You cannot load wasm via an import statement. We need to:

Get the .wasm bytes in a ArrayBuffer.
Compile the bytes into a WebAssembly.Module
Instantiate the WebAssembly.Module with imports to get the callable exports

Change the generatedwww/libexample.js file to this:

var libexample;
fetch('libexample_bg.wasm').then(response => {
  response.arrayBuffer().then(bytes => {
    var wasmModule = new WebAssembly.Module(bytes);
    libexample = new WebAssembly.Instance(wasmModule, []).exports;
  })
});
/**
* @param {number} a
* @param {number} b
* @returns {number}
*/
export function add(a, b) {
    const ret = libexample.add(a, b);
    return ret;
}

let cachegetUint8Memory = null;
function getUint8Memory() {
    if (cachegetUint8Memory === null || cachegetUint8Memory.buffer !== libexample.memory.buffer) {
        cachegetUint8Memory = new Uint8Array(libexample.memory.buffer);
    }
    return cachegetUint8Memory;
}

let WASM_VECTOR_LEN = 0;

function passArray8ToWasm(arg) {
    const ptr = libexample.__wbindgen_malloc(arg.length * 1);
    getUint8Memory().set(arg, ptr / 1);
    WASM_VECTOR_LEN = arg.length;
    return ptr;
}
/**
* @param {Uint8Array} a
*/
export function alter(a) {
    const ptr0 = passArray8ToWasm(a);
    const len0 = WASM_VECTOR_LEN;
    try {
        libexample.alter(ptr0, len0);
    } finally {
        a.set(getUint8Memory().subarray(ptr0 / 1, ptr0 / 1 + len0));
        libexample.__wbindgen_free(ptr0, len0 * 1);
    }
}

The wasm file is now loaded as suggested and we needed to change the bindings here and there. There is one drawback. it is possible that libexample module methods are called while the wasm module is not loaded. In production we do need to throw some event when the module is ready to use. For now we use a timeout, Change www/index.js to this:

import * as libexample from "./libexample.js"
setTimeout( () => {
  // Call wasm method 'add' typical stack method
  let result = libexample.add(10, 2);
  console.log('add result:' + result);
  // Now let's access heap memory /reference
  var a = new Uint8Array(100);
  a[0] = 225;
  a[1] = 10;
  console.log('pre remote call a[1] === ' + a[1]);
  libexample.alter(a);
  console.log('post remote call a[1] === ' + a[1]);
}, 1000);

Now this runs and that's okay.
But of course things can be simpler by just use the proper commands.
wasm-bindgen with --target web

wasm-bindgen target/wasm32-unknown-unknown/release/deps/libexample.wasm --target web  --out-dir ./www

This will change the libexample.js to a ready to use script.

Conclusion

When sharing heap memory (or access js from rust) wasm-bindgen is very handy/essential. It also generates the required embed code (for nodejs and the web). However, when developing a library I would use an extra wasm-bindgen glue library in the last stage of the project and keep the actual code/lib as standard and simple as possible.