Borsh to JSON in Rust using only schemas

tl;dr

Use this utility to generate a binary schema file from a Rust struct that implements the BorshSchema trait. Then using the schema file, easily convert to and from JSON and Borsh. No code generation needed.

Source

https://github.com/wkennedy/borsh-schema-utils

[dependencies]
borsh-schema-writer = "0.1.0"
borsh-serde-adapter = "0.1.0"

Overview

Serialization is an essential process for transmitting and storing data, offering various approaches and technologies to suit different use cases. In many instances, the data that needs to be serialized is predetermined, allowing for the use of libraries that generate code for serialization and deserialization (known as non-descriptive). This approach aligns with Borsh's methodology. However, there are situations where the data to be serialized is not known in advance or when the recipient of the serialized data cannot utilize generated code effectively. This is where borsh-schema-utils libraries come into play.

Data often traverses freely and finds its way to unknown consumers. Depending on the circumstances, maintaining generated code from a schema can prove challenging - this might occur when working with a language lacking a readily available library for simple serialization/deserialization or when dynamic implementation becomes necessary. Nevertheless, almost every programming language possesses JSON parsing capabilities; if not already integrated within it, implementing such functionality tends to be relatively straightforward. The borsh-schema-utils offers utilities that facilitate schema generation from Rust structs through its BorshSchema trait while concurrently enabling seamless transformation between JSON format and Rust struct representation.

borsh-schema-writer

The library includes a feature that allows for the extraction of a Struct with the BorshSchema trait and saving its schema to a designated file. This file can then be stored in various locations such as registries, file systems, databases, web storage, and more. This enables consumers to access and utilize the schema as needed. Here's an example of how it works:

use std::fs::File;
use std::io::BufReader;
use borsh::{BorshDeserialize, BorshSerialize, BorshSchema};
use borsh::schema::{BorshSchemaContainer, Definition, Fields};
use borsh_schema_writer::schema_writer::write_schema;

#[derive(Debug, Default, BorshSerialize, BorshDeserialize, BorshSchema)]
pub struct Person {
    first_name: String,
    last_name: String
}

fn write_schema_example() {
    write_schema(Person::default(), "./tests/schema/person_schema.dat".to_string());
    let file = File::open("./tests/schema/person_schema.dat").unwrap();
    let mut reader = BufReader::new(file);
    let container_from_file = BorshSchemaContainer::deserialize_reader(&mut reader).expect("Deserialization for BorshSchemaContainer failed");
}

Neat. Now you have a schema file that can be used by consumers to deserialize data. Let's see how that can be done.

borsh-serde-adapter

The library offers functions for converting serialized data from Borsh into a serde_json value using the binary file produced by the borsh-schema-writer library. By utilizing this library, you can convert a serde_json value into its corresponding borsh serialization. To illustrate, here is an instance of deserialization in action:

fn deserialize_from_borsh_schema_from_file() {
    let person = Person {
        first_name: "John".to_string(),
        last_name: "Doe".to_string(),
    };

    let person_ser = person.try_to_vec().expect("Error trying to seralize Person");

    let file = File::open("./tests/schema/person_schema.dat").unwrap();
    let mut reader = BufReader::new(file);
    let container_from_file = BorshSchemaContainer::deserialize_reader(&mut reader).expect("Deserializing BorshSchemaContainer failed.");

    let result = deserialize_from_schema(&mut person_ser.as_slice(), &container_from_file).expect("Deserializing from schema failed.");
    println!("{}", result.to_string());
}

Here is the result:

{"first_name": "John", "last_name": "Doe"}

In this example you can see that we aren't deserializing a struct with the BorshDeserialize trait. Typically, you would do something like this:

let person_serialized = person.try_to_vec().expect("Error trying to serialize Person");
let person_deserialized = Person::deserialize(&mut person_serialized.as_slice());

But with the schema, we can go from borsh serialized, directly to serde_json and then we can easily get JSON as a string. This makes it much easier for other consumers to handle.

We can do something similar with serializing from serde_json to borsh.

fn serialize_from_borsh_schema_with_string() {
    let file = File::open("./tests/schema/person_schema.dat").unwrap();
    let mut reader = BufReader::new(file);
    let person_schema = BorshSchemaContainer::deserialize_reader(&mut reader).expect("Deserializing BorshSchemaContainer failed.");

    let person_value = json!({"first_name": "John", "last_name": "Doe"});
    let mut person_writer = Vec::new();

    let _ = serialize_serde_json_to_borsh(&mut person_writer, &person_value, &person_schema).expect("Serialization failed");

    let result = deserialize_from_schema(&mut person_writer.as_slice(), &person_schema).expect("Deserialization failed");
}

Here you can see instead of serializing from the Person struct with the BorshSerialize trait, we were able to go from serde_json directly to borsh serialization, with person_writer containing the serialized struct as a vec of bytes.

Borsh Schema to JSON

It's possible to get the schema as JSON. This is useful for consumers that utilize the binary file. This can be done using the write_schema_as_json function. Here is an example:

fn schema_to_json_test() {
    write_schema_as_json(Person::default(), "./tests/schema/person_schema.json".to_string());
    let file = File::open("./tests/schema/person_schema.json").unwrap();
    let reader = BufReader::new(file);
    let result: Value = serde_json::from_reader(reader).expect("Deserialization failed");
    println!("{}", result.to_string());
}

This will result in the following JSON:

{
  "declaration": "Person",
  "definitions": [
    [
      "Person",
      {
        "Struct": {
          "fields": {
            "NamedFields": [
              [
                [
                  "first_name",
                  "string"
                ],
                [
                  "last_name",
                  "string"
                ]
              ]
            ]
          }
        }
      }
    ]
  ]
}

Conclusion

I hope you've enjoyed having fun with serialization! If you have any questions, please feel free to reach out to me.