Rust-scales Python: Basic Experiment

Some bits

Python bindings from/to C and C++ are probably one of the hottest topic for an intermediate/advanced Python engineer trying to get some handle on profiling and optimisation, and the inner capabilities of Python’s Virtual Machine. These low-level system languages are probably the fellows to walk with towards understanding of Python’s stack and the wider C-family of language.

The latest most notable addition to this family is probably Rust, a Mozilla-backed language that provides a stable API since 2015: there is a lot of enthusiasm around its adoption and in this article I try an experiment that may suggest one reason for; beside the many other advantages you can already read about in many other articles around. Main reasons I find motivating to learn Rust are:

• it is multi-paradigm by design:

(C-like structures but also Functional tools beside Static-typing and more) the most popular languages around are designed taking one paradigm as reference, and have tried to cope by adding support for other paradigms via additions to the respective standard libraries. This unfortunately ends up in a mix that makes a lot of engineers unsatisfied; the outcome of a single-paradigm design are usually limitations in possibilities of expression.

• it is designed considering user-experience for the entire workflow/life-cycle:

(from installation to release, through testing, documentation, packages managing) most of the languages have grown in their core capabilities before reaching the critical mass to justify the development of automation/productivity tools; this realised into gaps, conflicts and quality differences between tools used at different steps of the developing experience.

• it provides the best level of integration with WebAssembly (Wasm) at the moment and possibly in perspective:

WebAssembly has reached a stable interface and it is now a standard; it will be the platform on-top of which Web software is going to be built. Rust provides straightforward access to Wasm, as simple as we are going to show in this experiment.

Let’s see how simple it is to start a project, create a function in Rust, compile the library in WebAssembly and finally call this function from Python (yea! no work on C-bindings with all the memory-safety pitfalls and compiling flags! Python will call native WASM modules).

Simple that matters

You can try this kind of experiment by installing:

• rustup: tool-chains manager to navigate the manifold of Rust’s ecosystem
• cargo: the package manager
• rustc: the Compiler, working along with the rest of a stable Rust toolchain for your machine.

rustup install all of that for you!

Let’s start:

.1. Add the Wasm tool-chain:

 rustup target add wasm32-unknown-unknown

.2. Create a new project and change directory:

cargo new medium-xp && cd medium-xp 

.3. Create a lib.rs file in src directory. All code goes in src, lib.rs is where Rust keep the root library code.

.4. Cargo.toml is your package configuration file, add this section to the Cargo.toml file:

[lib]
crate-type = [“cdylib”]

This is to define the project as a dynamic library

.5. main.rs is your entry-point module, we are not going to use it this time as we are creating a library, not a binary. In lib.rs add this code for our function:

#[no_mangle]
pub extern fn simple_add(a: i32, b: i32) -> i32 { a + b}

.6. Compile the library for Wasm:

cargo build --release --target wasm32-unknown-unknown

Now you will find your library in target/wasm32-unknown-unknown/release compiled as WASM module!

.7. On the Python side:

• install wasmer extension for Python (tested on Python 3.6):

 python3.6 -m pip install wasmer

• create a run-wasm-from-python.py file with this script:

from wasmer import Instance
path = '<relative path to your Wasm file>'
with open(path, ‘rb’) as bytecode:
    wasm_bytes = bytecode.read()
    instance = Instance(wasm_bytes)
    result = instance.exports.simple_add(12, 12)
    print('Modules exported from Rust: ')
    print(instance.exports)  # this will print function's name
    print('call simple_add(12, 12): ')
    print(result)  # 24

• run this script with: python3.6 run-wasm-from-python.py

If you have done everything correctly…

Modules exported from Rust: 
["simple_add"]
call simple_add(12, 12): 
24

This demonstrates how it is technically possible to make Python inter-operating with compiled Rust using Wasm. It is incredibly straightforward thanks to the powerful design of Rust and the incredible work done at every step of the tool-chain. As you could see everything runs smoothly with high-levels of productivity involved. I leave to You the considerations about advantages and disadvantages of trying to optimise heavy loads of computing with Rust while keeping the dynamic approach of Python at the higher level. This was possible only recently as all the interfaces and tools involved have stabilised, thanks to the great work done by Open Source developers all over the World.

References

• Full repository on Github
• Definitely check out this Reddit
• Wasmer here

In the same series

• New Episode ---> some-examples-1
• New episode ---> Flask-experiment