DEV Community

Discussion on: Protobuf code generation in Rust

Collapse
 
ahwatts profile image
Andrew Watts

An alternative to this would be to copy the generated rust module from $OUT_DIR to $CARGO_MANIFEST_DIR/src. In the past, I've used github.com/danburkert/prost for protobuf things, and used something along these lines:

let proto_path = (the path to the protobuf definitions)
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
let src_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap()).join("src");
let out_path = out_dir.join("messages.rs");
let module_path = src_dir.join("messages").join("generated.rs");

prost_build::Config::new()
    .compile_protos(
        &[proto_path.to_str().unwrap()],
        &[proto_path.parent().unwrap().to_str().unwrap()],
    )
    .expect("Failed to compile protobuf definitions");

fs::copy(out_path, module_path).unwrap();

My experience has generally been that the actual protobuf definitions don't change very often, and the generated code isn't build-dependent, so committing the generated file isn't a huge deal (or you could set it to be ignored). You can then shave a little bit of time off the compilation by only regenerating the code if the definitions are newer than the generated module.

Collapse
 
elshize profile image
Michał Siedlaczek

Thanks for sharing. I didn't know about prost and would love to know how it compares with protobuf-codegen-pure if you know. For one, it seems to write to OUT_DIR by default.

Do I understand correctly that src_dir will point to your project/src directory? If so, we will experience the same problem of not being able to write to it on Docs.rs, right? I guess your suggestion would be to not run the generation as a build script but rather do it manually and commit the generated files, which certainly seems like a reasonable option for protobuf, which shouldn't change between builds.

Collapse
 
ahwatts profile image
Andrew Watts

As I recall, I used prost because at the time it offered me a way to customize derived traits for the generated types (I needed the serde traits), while the other protobuf crate I looked at did not. That might not be the case now.

CARGO_MANIFEST_DIR is the directory where Cargo.toml lives, so $CARGO_MANIFEST_DIR/src is the project's src directory.

In my case, the definitions were in another repository, and might not always be available if that repo had not been checked out in the right place on the machine doing the build, but I still wanted to be able to build in that case.

Thus, I do run the generator as a build script, but I also commit the output to source control, so rustdoc ought to see it. There's some logic in the build script (that I didn't post) that only runs the protobuf generator if the module is older than the definitions. If I need to force it to regenerate, I can always locally delete the generated module and build, and then commit any differences.

Thread Thread
 
elshize profile image
Michał Siedlaczek

There's some logic in the build script (that I didn't post) that only runs the protobuf generator if the module is older than the definitions.

Ok, that explains a lot, thanks for clarifying. Sounds like a sensible approach.