Bazel allows several ways to define rules in a workspace. In this article, we are focusing on genrule, the easiest way to create a custom rule (with some limitations).
# full signature of genrule
genrule(name, srcs, outs, cmd, cmd_bash, cmd_bat, cmd_ps, compatible_with, deprecation, distribs, exec_compatible_with, exec_properties, exec_tools, executable, features, licenses, local, message, output_licenses, output_to_bindir, restricted_to, tags, target_compatible_with, testonly, toolchains, tools, visibility)
The goal of a rule is to produce outs as form of one or several files. So the minimal arguments to set are :
-
nameused to identified theoutsthen you can refer them as a group into input for otherruleor what to build. -
outsa non empty list of files generated by the rule - the command to generate the
outs, via at least ofcmd,cmd_bash,cmd_bat,cmd_ps
The following example create a file hello20s.txt after 20s. When outs is a list of one file, then $@ can be used as a shortcut to identify the output file.
Go back to hello20s
Let's start with the rule hello20s defined in previous article.
# BUILD.bazel
genrule(
name = "hello20s",
outs = ["hello20s.txt"],
cmd_bash = "sleep 20 && echo 'hello20s' >$@",
)
❯ bazel build //:hello20s
INFO: Analyzed target //:hello20s (5 packages loaded, 7 targets configured).
INFO: Found 1 target...
Target //:hello20s up-to-date:
bazel-bin/hello20s.txt
INFO: Elapsed time: 20.334s, Critical Path: 20.02s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
The target was run but the output was not as expected in the same as folder BUILD.bazel, it is at bazel-bin\hello20s.txt, because bazel keeps separated what is generated and what is not.
Also note, that instead of requesting to build every rules, we request to build hello20s. There are several syntaxes for the target, in this case //<package_name>:<name> is a full qualified target name (start by //) with package_name empty because BUILD.bazel is at the root of the workspace.
What happens if we moved this to a sub-level package named exp_genrule ?
# create a package is made by creating a folder with a BUILD file
mkdir exp_genrule
mv BUILD.bazel exp_genrule
❯ bazel build //:hello20s
ERROR: Skipping '//:hello20s': no such package '': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
- /home/david/src/github.com/davidB/sandbox_bazel
WARNING: Target pattern parsing failed.
ERROR: no such package '': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
- /home/david/src/github.com/davidB/sandbox_bazel
INFO: Elapsed time: 0.030s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
It failed as expected, because the full qualified name need to include the new package'name.
❯ bazel build //exp_genrule:hello20s
INFO: Analyzed target //exp_genrule:hello20s (1 packages loaded, 1 target configured).
INFO: Found 1 target...
Target //exp_genrule:hello20s up-to-date:
bazel-bin/exp_genrule/hello20s.txt
INFO: Elapsed time: 20.058s, Critical Path: 20.01s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
Note that the new package name is also applied into the path of the output bazel-bin/exp_genrule/hello20s.txt
Build environment
When a rule is built the cmd part is executed into a sandboxed environment instead of "in-place" like lot of build tool. It could be interesting to explore this sandbox a little beat to better understand it for our future custom build.
Let's start by grabbing information about the environment. Add into exp_genrule/BUILD.bazel the rule envinfo:
genrule(
name = "envinfo",
outs = ["envinfo.txt"],
cmd_bash = """( env; echo "-----"; pwd; ls; echo "-----"; whoami; echo "outs = $@" ) | tee $@""",
)
In this rule, we pipe the output into tee instead of redirect to also have the output displayed on terminal. It is easier for debug (in CI or in interactive).
❯ bazel build //exp_genrule:envinfo
INFO: Analyzed target //exp_genrule:envinfo (1 packages loaded, 1 target configured).
INFO: Found 1 target...
INFO: From Executing genrule //exp_genrule:envinfo:
PWD=/home/david/.cache/bazel/_bazel_david/76e87152cc51687aee6e05b5bdcf89aa/sandbox/linux-sandbox/9/execroot/__main__
TMPDIR=/tmp
SHLVL=1
PATH=/home/david/.cache/bazelisk/downloads/bazelbuild/bazel-4.0.0-linux-x86_64/bin:/home/david/.nix-profile/bin:/home/david/.nix-profile/bin:/home/david/.poetry/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/david/.cargo/bin:/home/david/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/var/lib/snapd/snap/bin
_=/usr/bin/env
-----
/home/david/.cache/bazel/_bazel_david/76e87152cc51687aee6e05b5bdcf89aa/sandbox/linux-sandbox/9/execroot/__main__
bazel-out
external
-----
david
outs = bazel-out/k8-fastbuild/bin/exp_genrule/envinfo.txt
Target //exp_genrule:envinfo up-to-date:
bazel-bin/exp_genrule/envinfo.txt
INFO: Elapsed time: 0.043s, Critical Path: 0.01s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
We didn't trunk the output, so as you can see :
- the rule is executed with our account
- the environment variables and shell is not the one from where the command was called, notice that
-
$PATHis the same as in the caller shell -
$HOMEis not defined (like other environment)
-
- the current working directory is under
$HOME/.cache/... - the value of
$@is a relative path that generate intobazel-outnot intobazel-bin
But what change if we need input files ?
# create 2 input files
echo "A" > exp_genrule/a.txt
echo "B" > exp_genrule/b.txt
Modify envinfo to depend of the input files:
genrule(
name = "envinfo",
srcs = [
"a.txt",
"b.txt",
],
outs = ["envinfo.txt"],
cmd_bash = """( env; echo "-----"; pwd; ls -l; echo "-----"; whoami; echo "outs = $@" ) | tee $@""",
)
❯ bazel build //exp_genrule:envinfo
INFO: Analyzed target //exp_genrule:envinfo (1 packages loaded, 3 targets configured).
INFO: Found 1 target...
INFO: From Executing genrule //exp_genrule:envinfo:
PWD=/home/david/.cache/bazel/_bazel_david/76e87152cc51687aee6e05b5bdcf89aa/sandbox/linux-sandbox/13/execroot/__main__
TMPDIR=/tmp
SHLVL=1
PATH=/home/david/.cache/bazelisk/downloads/bazelbuild/bazel-4.0.0-linux-x86_64/bin:/home/david/.nix-profile/bin:/home/david/.nix-profile/bin:/home/david/.poetry/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/david/.cargo/bin:/home/david/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/var/lib/snapd/snap/bin
_=/usr/bin/env
-----
/home/david/.cache/bazel/_bazel_david/76e87152cc51687aee6e05b5bdcf89aa/sandbox/linux-sandbox/13/execroot/__main__
total 12
drwxr-xr-x 3 david david 4096 Mar 21 17:46 bazel-out
drwxr-xr-x 2 david david 4096 Mar 21 17:46 exp_genrule
drwxr-xr-x 3 david david 4096 Mar 21 17:46 external
-----
david
outs = bazel-out/k8-fastbuild/bin/exp_genrule/envinfo.txt
Target //exp_genrule:envinfo up-to-date:
bazel-bin/exp_genrule/envinfo.txt
INFO: Elapsed time: 0.044s, Critical Path: 0.01s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
Bazel add a folder exp_genrule and if you modify the rule to use ls -lR, you will see that bazel also added symlink to a.txt and b.txt. It means that if we want to generate a file that take both file as input, we'll need to change directory to go inside exp_genrule or to accept path instead of filenames. To help us bazel provide function and Make Variables (like $@). Try to display some of them:
genrule(
name = "envinfo",
srcs = [
"a.txt",
"b.txt",
],
outs = ["envinfo.txt"],
cmd_bash = """(
env
echo "-----"
pwd
ls -l
echo "-----"
echo "whoami = $$(whoami)"
echo "TARGET_CPU = $(TARGET_CPU)"
echo "BINDIR = $(BINDIR)"
echo "GENDIR = $(GENDIR)"
echo "OUTS = $(OUTS)"
echo "SRCS = $(SRCS)"
echo "RULEDIR = $(RULEDIR)"
echo "a.txt = $(location :a.txt)"
echo "b.txt = $(location :b.txt)"
) | tee $@""",
)
...
TARGET_CPU = k8
BINDIR = bazel-out/k8-fastbuild/bin
GENDIR = bazel-out/k8-fastbuild/bin
OUTS = bazel-out/k8-fastbuild/bin/exp_genrule/envinfo.txt
SRCS = exp_genrule/a.txt exp_genrule/b.txt
RULEDIR = bazel-out/k8-fastbuild/bin/exp_genrule
a.txt = exp_genrule/a.txt
b.txt = exp_genrule/b.txt
...
NOTE the
:at the begining of filename, it's becauselocationtakes a target as parameter, and every files can be identified as a target
So we can use those variables and the location function to compute the path that will be needed to run our tool inside bazel. But it will not be as simpler than with other builder. This complexity/sandbox is also what enforce that :
- every inputs (
srcs,tools,cmd, environment variables, ...) are declared - every outputs are generated
Try remove "a.txt" from srcs or to not generate content into $@, bazel will failed.
Enough experimentation about genrule for today,...
Tips
We can also retrieve some of information with a built-in bazel command, give a try to:
bazel info --show_make_env
Conclusion
We only overlook part of genrule, we'll continue to discover when we'll used it in the following articles. If you want to know more, then follow the links from this article to the official doc.
You can also take a look at
The sandbox_bazel is hosted on github (not with the same history, due to errors), use tag to have the expected view at end of article: article/2_genrule.
Top comments (0)