How to load dynamic libraries in Rust?

What is Dynamic Library Loading?

Dynamic library loading, also known as dynamic loading or runtime dynamic linking, is a programming technique that allows a program to load external libraries (also known as dynamic link libraries or shared libraries) into memory and use their functionalities at runtime, rather than at compile time. These libraries contain compiled code that can be executed by the program, enabling modularization, code reuse, and flexibility.

Dynamic library loading is in contrast to static linking, where all the necessary code is included in the program's executable binary during the compilation process. With dynamic library loading, the program remains smaller in size, and libraries can be updated or replaced without requiring changes to the main program.

Dynamic Library Loading in Rust

Rust provides a safe and ergonomic interface for loading dynamic libraries at runtime. The libloading crate provides a cross-platform API for loading dynamic libraries and calling functions defined in those libraries. It also provides a safe wrapper around the dlopen and dlsym functions on Unix-like systems and the LoadLibrary and GetProcAddress functions on Windows.

The complete source code is available on GitHub.

Setup the workspace

Let's start by creating a new Rust project named lib-loading:

cargo new lib-loading

# remove the src directory as we will be creating a workspace

Now, update the Cargo.toml file to create a workspace and add two members to it:

[workspace]

members = [
    "hello-world",
    "executor",
]

Create a dynamic library

Let's now create a library named hello-world with the following Cargo.toml file:

cargo new hello-world --lib

[package]
name = "hello-world"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["dylib"]

Note the crate-type field in the [lib] section. This tells the compiler to build a dynamic library instead of the default static library. This output type will create *.so files on Linux, *.dylib files on macOS, and *.dll files on Windows.

You can read more about different crate-types in the Rust Reference.

Next, update the src/lib.rs file to define a function named execute that prints a message to the console:

#[no_mangle]
pub fn execute() {
    println!("Hello, world!")
}

Name mangling is a compiler technique that encodes function names with additional information like the function's parameters and return type. However, this makes it difficult to call functions from other languages or dynamically loaded libraries.

The no_mangle attribute turns off Rust's name mangling, so that it has a well defined symbol to link to. This allows us to call the function from the dynamically loaded library.

Create a binary crate

Next, let's create a binary crate named executor that will load the hello-world library and call the execute function defined in it.

cargo new executor

Add a dependency on the libloading crate to the Cargo.toml file:

[package]
name = "executor"
version = "0.1.0"
edition = "2021"

[dependencies]
libloading = "0.8"

Now, update the src/main.rs file to load the hello-world library and call the execute function:

use libloading::{Library, library_filename, Symbol};

fn main() {
    unsafe {
        // Load the "hello_world" library
        let lib = Library::new(library_filename("hello_world")).unwrap();

         // Get the function pointer
        let func: Symbol<fn()> = lib.get(b"execute").unwrap();

        func() // Call the function
    }
}

It's a good practice to use the library_filename function to get the platform-specific filename of the library. This function returns libhello_world.so on Linux, libhello_world.dylib on macOS, and hello_world.dll on Windows.

Build and run the project

First, let's build the project with cargo build:

lib-loading on master [?] via 🦀 v1.69.0
➜ cargo build --release
   Compiling cfg-if v1.0.0
   Compiling hello-world v0.1.0 (/Users/gaurav.gahlot/workspace/playground/lib-loading/hello-world)
   Compiling libloading v0.8.0
   Compiling executor v0.1.0 (/Users/gaurav.gahlot/workspace/playground/lib-loading/executor)
    Finished release [optimized] target(s) in 1.42s

Next, run the executor using cargo run:

lib-loading on master [?] via 🦀 v1.69.0 took 2s
➜ cargo run -p executor --release
    Finished release [optimized] target(s) in 0.02s
     Running `target/release/executor`

Hello, world!

Congratulations! You've successfully loaded a dynamic library and called a function defined in it.

When to use Dynamic Library Loading?

Dynamic library loading is useful in several scenarios where you want to achieve modularity, flexibility, and runtime extensibility in your applications. Here are some situations where dynamic library loading can be advantageous:

1. Plugin Systems: If you're building an application that supports plugins or extensions, dynamic library loading can allow you to load and unload these plugins at runtime without modifying or restarting the main application. This can be useful for applications like text editors, web browsers, or media players that support third-party plugins.

2. Modular Applications: When you're developing a large application, dynamic library loading can help you break down the functionality into smaller, manageable components. This can lead to faster development cycles, easier maintenance, and improved code organization.

3. Hot Reload: For certain types of applications like game development tools or graphical design software, dynamic library loading can enable hot reloading. This means you can modify parts of the application's code, compile them into a dynamic library, and load that library without restarting the entire application. This greatly speeds up development iterations.

4. Platform-Specific Implementations: If your application needs to interact with platform-specific features, dynamic library loading allows you to provide different implementations for different platforms. You can load the appropriate library based on the runtime environment, avoiding unnecessary code in the main application binary.

5. Versioning and Updates: When you want to provide updates or bug fixes to a specific part of your application, you can distribute and load only the updated dynamic library without needing users to update the entire application.

6. Resource Sharing: Dynamic libraries can be used to share resources like database connections, network sockets, or other system resources across multiple parts of your application. This can help optimize resource usage and improve performance.

7. Language Interoperability: If you need to interface with code written in other programming languages (e.g., C or C++), dynamic libraries provide a way to call functions written in those languages from your Rust code.

8. Security and Isolation: In some cases, you might want to isolate certain components of your application for security reasons. Dynamic libraries can help you encapsulate sensitive code, limiting its exposure to the main application.

Challenges of Dynamic Library Loading

Dynamic library loading also introduces complexity and potential challenges. Here are a few things to consider when using dynamic libraries:

• Memory Management: You need to manage memory and resource allocation properly, ensuring that you release resources and unload libraries when they're no longer needed to avoid memory leaks.

• Compatibility: Ensuring that dynamically loaded libraries are compatible with your application's version and other libraries can be challenging. Version mismatches can lead to crashes or unexpected behavior.

• Unsafe Code: Working with dynamically loaded libraries often involves using the unsafe keyword due to the inherent risks associated with runtime operations that Rust's safety mechanisms cannot fully verify.

• Debugging: Debugging issues in dynamically loaded libraries can be more challenging compared to monolithic applications.

• Performance Overhead: There can be a slight performance overhead when using dynamic libraries compared to statically linking code directly into your application binary.

Conclusion

In summary, dynamic library loading is most beneficial when you need to create modular, flexible, and extensible applications, but it requires careful design, proper resource management, and an understanding of the potential trade-offs. Rust's safety mechanisms can help you avoid many of the pitfalls associated with dynamic library loading, but you still need to be aware of the risks and challenges involved.