Rust #7: Command-Line interfaces

I have discovered that Rust has great support for writing Command-Line Interface (CLI) tools, or tools that you interact with on the command-line.

The basic requirement of CLI tools is to be able to read the arguments that the user types after the command on the terminal and process them.

You may have noticed that most CLI tools are written in Rust look the same when interacting with them. For example, invalid use of them shows some text on how you should use them. Typing --help after the command will list all sub-commands (if any) and flags (commands that start with a hyphen or double hyphen) that you can use. Even typing --version after the command will show version information. I guarantee that none of the CLI tool developers worked hard to provide this functionality.

So to provide an introduction on how to write CLI tools, let me introduce you to my demo CLI app called greeter:

// in src/main.rs
use std::env;

fn main() {
    let args: Vec<String> = env::args().collect();

    enum Greet {
        Unknown,
        Hello,
        Goodbye,
    }

    let mut greet = Greet::Unknown;

    let mut name = String::from("World");
    let mut i = 1;
    while i < args.len() {
        if (args[i] == "-n" || args[i] == "--name") && i < args.len() - 1 {
            name = args[i + 1].clone();
            i += 1;
        } else if args[i] == "hello" {
            greet = Greet::Hello;
        } else if args[i] == "bye" {
            greet = Greet::Goodbye;
        }
        i += 1;
    }

    println!(
        "{}",
        match greet {
            Greet::Unknown => String::from("No idea what to do!"),
            Greet::Hello => format!("Hello {}", name),
            Greet::Goodbye => format!("Goodbye {}", name),
        }
    );
}

This program takes a sub-command (a command that follows after the program's name) and an optional flag to provide a name to use in the greeting. Let's put it through its paces:

$ greet
No idea what to do!
$ greet hello
Hello World
$ greet bye
Goodbye World
$ greet --name Matt
No idea what to do!
$ greet hello --name Matt
Hello Matt
$ greet bye -n Matt
Goodbye Matt

But there are problems with this program. For example, greet hello bye --name Matt --name Bob is valid and will result in Goodbye Bob. We'd probably want to restrict that kind of confusing use. Processing command-line parameters is tricky and full of edge cases.

There are even more issues. There is no help, no support for version information and as mentioned before, no good error handling.

So the code above is not idiomatic at all and is implemented similar to how a C programmer might do so. Processing command-lines arguments is error-prone and is code that you usually incrementally increase as you come back to it time and time again to implement more features. C programmers can use a library called Getopt to manage this and is very common in Unix CLI tools.

Rust provides us with the arguments using std::env::args() that provides an iterator (in true Rust fashion) that can iterate through all the commands that were given. The very first iteration provides the name of the program, which is why the loop starts at 1 and not 0.

Enter the Clap!

clap crate

Clap is a crate that was written to manage command-line options and provide a familiar interface via --help and --version. Clap uses the builder pattern to declaratively describe how your tool should interact with the command-line. So let's start rewriting our amazing CLI tool using Clap.

For starters, it can provide application information:

use clap::App;

fn main() {
    let matches = App::new("The Amazing Greeter")
        .version("1.0")
        .author("Matt Davies")
        .about("The best greeter in town!")
        .get_matches();
}

Don't forget to add clap = "2.33" to Cargo.toml in the [dependencies] section.

This short program already gives us plenty of functionality. For example:

$ cargo run -q -- --help
The Amazing Greeter 1.0
Matt Davies
The best greeter in town!

USAGE:
    greet

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

The standard help information! The -- in the command cargo run -- --help separates the arguments that go to Cargo from the arguments that go to our program. So --help is sent to our program and Clap dutifully follows it out. The -q flag sent to Cargo stops it from outputting build information ('q' is for quiet). Let's try the version information:

$ cargo run -q -- -V
The Amazing Greeter 1.0

Now we need to add support for the name flag:

use clap::{App, Arg};

fn main() {
    let matches = App::new("The Amazing Greeter")
        .version("1.0")
        .author("Matt Davies")
        .about("The best greeter in town!")
        .arg(
            Arg::with_name("name")
                .short("n")
                .long("name")
                .value_name("NAME")
                .help("Provides a name to use in the greeting")
                .takes_value(true)
                .default_value("World"),
        )
        .get_matches();

    let name = matches.value_of("name").unwrap();
    println!("NAME: {}", name);
}

The .arg() takes a builder that is created with Arg::with_name(). From this builder, we can set all the properties for the option. First, we give it a short name (a single character) and a long name. This allows the option to be provided with -n or --name. value_name() and help() provide extra information for the help information:

$ cargo run -q -- --help
The Amazing Greeter 1.0
Matt Davies
The best greeter in town!

USAGE:
    greet [OPTIONS]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -n, --name <NAME>    Provides a name to use in the greeting [default: World]

value_name() gives the name between the angled brackets and help() provides the text afterwards. Of course, the information from short() and long() are used too.

takes_value() tells clap that --name requires a value to follow after it and default_value() provides a value if the option is omitted. You may also have noticed that hyphened names are called flags unless they have values following them. In that case, they are called options. You may have additionally noticed that the value you passed to default_value() is also shown in the help information.

The intent for how this option is interacted with is much, much clearer than the original naive C-like code we used before. Let's test it further:

$ cargo run -q
NAME: World
$ cargo run -q --name Matt
NAME: Matt

Let's continue and add our sub-commands hello and bye:

use clap::{App, Arg, SubCommand};

fn main() {
    let matches = App::new("The Amazing Greeter")
        .version("1.0")
        .author("Matt Davies")
        .about("The best greeter in town!")
        .arg(
            Arg::with_name("name")
                .short("n")
                .long("name")
                .value_name("NAME")
                .help("Provides a name to use in the greeting")
                .takes_value(true)
                .default_value("World"),
        )
        .subcommand(
            SubCommand::with_name("hello")
                .about("Let's meet!")
                .version("1.0")
                .author("Matt Davies (again!)"),
        )
        .subcommand(
            SubCommand::with_name("bye")
                .about("We part ways")
                .version("1.0")
                .author("Matt Davies (again!)"),
        )
        .get_matches();

    let name = matches.value_of("name").unwrap();

    match matches.subcommand_name() {
        Some("hello") => println!("Hello {}", name),
        Some("bye") => println!("Goodbye {}", name),
        _ => println!("No idea what to do!"),
    }
}

interestingly, the sub-commands have their own about(), version() and author() calls in the builder generated by SubCommand::with_name(). So let's try some help:

$ cargo r --q -- -h
The Amazing Greeter 1.0
Matt Davies
The best greeter in town!

USAGE:
    greet [OPTIONS] [SUBCOMMAND]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -n, --name <NAME>    Provides a name to use in the greeting [default: World]

SUBCOMMANDS:
    bye      We part ways
    hello    Let's meet!
    help     Prints this message or the help of the given subcommand(s)

Now, we have a subcommands section with the added subcommand help added:

cargo r -q -- help hello
greet-hello 1.0
Matt Davies (again!)
Let's meet!

USAGE:
    greet hello

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

Now the name of the application is the concatenation of the application name and the subcommand: greet-hello. I find this interesting as this implies you can add plugins by creating programs that are called greet-<command> providing that greet generates the filename if the subcommand is unknown, and calls it. This is exactly how Cargo works. I wonder if Cargo uses clap? Looking at its dependencies we can see that yes, indeed, it does. Will Clap execute plugins automatically? Let's try it:

$ cargo r -q -- test
error: Found argument 'test' which wasn't expected, or isn't valid in this context

USAGE:
    greet [OPTIONS] [SUBCOMMAND]

For more information try --help

No, it doesn't. But it's great to see that the usage description is generated according to the fact we have added options and subcommands.

Ok, now let's try our program out:

$ cargo r -q
No idea what to do!
$ cargo r -q -- --name Matt
No idea what to do!
$ cargo r -q -- --name Matt hello
Hello Matt
$ cargo r -q -- --name Bob bye
Goodbye Bob
$ cargo r -q -- hello --name Matt
error: Found argument '--name' which wasn't expected, or isn't valid in this context

USAGE:
    greet hello

For more information try --help

Well, that's not good. It seems that with Clap, the options before a subcommand are considered different than the options passed after a subcommand. We can add arguments to the subcommands:

use clap::{App, Arg, SubCommand};

fn main() {
    let matches = App::new("The Amazing Greeter")
        .version("1.0")
        .author("Matt Davies")
        .about("The best greeter in town!")
        .arg(
            Arg::with_name("name")
                .short("n")
                .long("name")
                .value_name("NAME")
                .help("Provides a name to use in the greeting")
                .takes_value(true)
                .default_value("World"),
        )
        .subcommand(
            SubCommand::with_name("hello")
                .about("Let's meet!")
                .version("1.0")
                .author("Matt Davies (again!)")
                .arg(
                    Arg::with_name("name")
                        .short("n")
                        .long("name")
                        .value_name("NAME")
                        .help("Provides a name to use in the greeting")
                        .takes_value(true),
                ),
        )
        .subcommand(
            SubCommand::with_name("bye")
                .about("We part ways")
                .version("1.0")
                .author("Matt Davies (again!)")
                .arg(
                    Arg::with_name("name")
                        .short("n")
                        .long("name")
                        .value_name("NAME")
                        .help("Provides a name to use in the greeting")
                        .takes_value(true),
                ),
        )
        .get_matches();

    let name = matches.value_of("name").unwrap();

    match matches.subcommand_name() {
        Some("hello") => {
            let submatches = matches.subcommand_matches("hello").unwrap();
            let name = submatches.value_of("name").unwrap_or(name);
            println!("Hello {}", name);
        }
        Some("bye") => {
            let submatches = matches.subcommand_matches("bye").unwrap();
            let name = submatches.value_of("name").unwrap_or(name);
            println!("Hello {}", name);
        }
        _ => println!("No idea what to do!"),
    }
}

A few things to take note of. Firstly, we don't provide default values for the subcommand options as the global option can provide that.

Secondly, to access the local options of a subcommand we need to grab its matches with subcommand_matches() that may or may not exist. Since we've already matched the name, we can unwrap it here with safety. Then we fetch the value if it exists, or use the global name value. The global name should always have a value because it has a default one.

This works quite well now:

$ cargo r -q -- hello --name Matt
Hello Matt
$ cargo r -q -- bye --name Matt
Goodbye Matt
$ cargo r -q -- hello
Hello World
$ cargo r -q -- --name Matt hello
Hello Matt
$ cargo r -q -- --name Matt hello --name Bob
Hello Bob

OK, it's not perfect since it doesn't detect two --name options, but perhaps we can remove the global option and ensure that the two subcommand options have default values.

There is a lot more you can do with Clap and I encourage you to explore the documentation.

But there's a better, more concise way to interact with Clap.

Structopt crate

What problem with Clap is that it is very verbose to describe the options and to use them. A better way to access the command line data would be to place it all in a structure hierarchy. Something like this:

struct CommandLineData {
    subcommand: SubCommand,
}

enum SubCommand {
    Hello(HelloData),
    Bye(ByeData),
}

struct HelloData {
    name: String,
}

struct ByeData {
    name: String,
}

But we still have to extract the information from Clap and insert it into that data structure. And, you've guessed it, is exactly what Structopt strives to do.

Through the use of macros and a function call, Structopt can generate the calls to Clap to declare the configuration and to extract the data into your structures. It is an extremely useful crate.

First, replace the clap entry in Cargo.toml with structopt = "0.3". And replace main.rs with:

use structopt::StructOpt;

#[derive(Debug, StructOpt)]
#[structopt(
    name = "The Amazing Greeter",
    about = "The best greeter in town",
    author = "Matt Davies"
)]
struct CommandLineData {
    #[structopt(subcommand)]
    subcommand: Option<SubCommand>,
}

#[derive(Debug, StructOpt)]
enum SubCommand {
    /// Let's meet!
    Hello(HelloData),

    /// Let's part ways
    Bye(ByeData),
}

#[derive(Debug, StructOpt)]
#[structopt(author = "Matt Davies")]
struct HelloData {
    /// Provides a name to use in the greeting.
    #[structopt(short = "n", long = "name", default_value = "World")]
    name: String,
}

#[derive(Debug, StructOpt)]
#[structopt(author = "Matt Davies (again!)")]
struct ByeData {
    /// Provides a name to use in the greeting.
    #[structopt(short = "n", long = "name", default_value = "World")]
    name: String,
}

fn main() {
    let opt = CommandLineData::from_args();

    match opt.subcommand {
        Some(SubCommand::Hello(data)) => println!("Hello {}", data.name),
        Some(SubCommand::Bye(data)) => println!("Goodbye {}", data.name),
        None => println!("Not sure what to do!"),
    }
}

Every structure and enum used to house command-line data must start with #[derive(Debug, StructOpt)]. The Debug trait needs to be there to provide error messages when things go wrong. The StructOpt trait is there to do the magic. Later in the structures and enums, we have #[structopt()] attribute macros and documentation comments to help with the meta-data. For example, the document comments for the Hello variant in SubCommand is used to generate the string that's passed to about() in Clap and is used for documentation. Two birds with one stone (that expression seems very cruel to me!).

One difference I noticed was that version information throughout the declaration is lifted directly from Cargo.toml and so different subcommands cannot have different versions. For me, this makes much more sense!

Finally, a quick call to from_args() method on your top-level data structure will construct an instance. The code that uses the command line data ends up being very concise.

As with Clap, StructOpt has lots of features that I encourage you to discover in the documentation. For example, you can declare an option as being required if another option is used. Also, you can collect all non-flag and non-option arguments into an array by just declaring a Vec field in your structure. Flags are easily produced using a bool type field. You may have noticed that name didn't require explicit information that it takes an argument. StructOpt was able to figure that out all by itself because it was of type String.

But there's one more thing we can do...

Paw crate

Paw allows us to treat the command line data structure as an argument to main(). In C and C++, you access the command line arguments via arguments passed to the main function but Rust handles it differently. You have to call a function to fetch them using std::env::args(). But with Paw, you can have the same functionality.

In Cargo.toml, change the dependencies to:

[dependencies]
structopt = { version = "0.3", features = ["paw"] }
paw = "1.0"

and now main can be:

#[paw::main]
fn main(opt: CommandLineData) {
    match opt.subcommand {
        Some(SubCommand::Hello(data)) => println!("Hello {}", data.name),
        Some(SubCommand::Bye(data)) => println!("Goodbye {}", data.name),
        None => println!("Not sure what to do!"),
    }
}

The paw::main macro wraps our main and transforms it into one that can take a single argument containing our command line data. No more calls to from_args(). You're not saving much but I thought this crate was quite cute. Very rust-like.

Conclusion

I showed you a terrible naive way of writing a program that processed the command-line, and then I showed you how to improve the situation drastically using Clap. Following that, I showed you that you didn't require much code at all and StructOpt allowed you to pack all your command line data into your own data structures whose very structure described the command-line use. Finally, I showed you Paw that allowed the command-line structure to be passed directly to your main function.

Hopefully, I have shown you that Rust, along with a few crates, is the best language to use for processing command-line arguments.

So, journey on and write some amazing CLI programs!