What I found when I pointed a CLI recorder at a second Rust rewrite

I have been using behave to record how GNU coreutils 9.12 behaves, then replay the same cases against uutils, the Rust rewrite. That work has already led to a run of upstream bug reports and merged fix PRs. This week I pointed the same recorder at a second Rust rewrite: uutils findutils, starting with xargs, and compared it to GNU findutils 4.10.

The idea is the same. Record stdout, stderr, and the exit status from the reference tool, then replay the cases against the rewrite. A failure is a concrete difference, not a guess about what the tool "should" do.

The new suite lives at examples/xargs. It has 44 cases, recorded from GNU findutils 4.10.0, and they run in behave's CI. Three of those differences were real bugs. Each one is now an issue plus a fix PR with a test.

1. -I with the default command printed the input items

When -I, -i, or --replace is used without an explicit command, xargs should run echo once per input item. The default command has no initial arguments, so the replace string never appears, and each invocation should print a blank line (the same as echo with no arguments).

uutils printed the input items instead.

printf 'hello\nworld\n' | xargs -I R

GNU findutils 4.10.0 prints two blank lines. uutils findutils 0.11.0 (current main, when I recorded it) printed:

hello
world

The same happened with xargs -i and xargs --replace. Passing an explicit command was already fine on both sides:

printf 'hello\nworld\n' | xargs -I R echo
# two blank lines

printf 'hello\nworld\n' | xargs -I R echo R
# hello
# world

So the happy path looked correct, and the default-command case was the one that drifted. I filed uutils/findutils#984 and the fix in PR #986.

2. -a rejected a filename that starts with a dash

xargs -a FILE reads items from FILE instead of stdin. If that name starts with a dash, GNU still treats it as a filename. uutils treated it as an unknown option and refused to run.

The same gap showed up on the other options that take a value: -n, -s, -L, -d, and -P. A name or number that looks like a flag should still be accepted as the option's value.

# GNU findutils 4.10.0: reads items from the file named -foo
# uutils: rejected the filename
xargs -a -foo

I filed issue #985 and PR #987.

This is a small class of bugs I keep seeing when a rewrite parses flags with a stricter library default than the original tool. The reference tool often accepts a value that starts with a dash. A probe that only tries tidy names will never see it.

3. --eof swallowed the next option

--eof and -e set the logical end-of-file string. The argument is optional. When it is omitted, the next token should still be parsed as a normal option. uutils consumed that token as the EOF string, so a following flag disappeared.

The short form I used while recording was:

xargs --eof -l

GNU findutils still honours -l. uutils treated -l as the EOF string and ignored the line-limit flag. I filed issue #992 and PR #993.

Optional arguments are a reliable place to look. A recorder that walks flag pairs will put --eof next to -l without anyone writing that case by hand.

A lesson I had already paid for: pin the reference version

While this suite was going in, behave's CI was red on a pre-existing base32 case, and it never reached xargs. The runner ships GNU coreutils 9.4. I record from 9.12, because that is the release uutils aims at. The failing cases were about decoding unpadded input: GNU 9.5 and later decode it, 9.4 rejects it.

The fix was not to weaken the cases. I tagged those three specs gnu-9.5 (two in examples/base32, one in examples/base64) and skipped that tag in CI, so they still run against a current GNU locally. CI went green again, including the 44 xargs cases.

The same lesson showed up when I was testing coreutils, and it is worth repeating. If you claim compatibility with a specific reference release, record from that release. A case that is correct on 9.12 can fail on 9.4, and the noise looks like a product bug. Version-sensitive specs belong behind a tag, or they teach people to ignore failures.

A third rewrite: uutils sed

I also pointed behave at uutils sed, compared to GNU sed 4.9, using non-interactive cases. Two differences stood out, and both are now filed with a test and a fix.

The GNU v command is a version assertion. It does nothing at runtime if the running sed is new enough, and it is a common guard at the top of GNU sed scripts. uutils already compiled v and rejected versions newer than 4.9, but a successful check still panicked at runtime with invalid command code as soon as there was any input. Existing tests called sed v with no stdin, so they never reached the processor. Feeding a line was enough:

printf 'a\n' | sed 'v'
printf 'a\n' | sed 'v4.9'

GNU sed 4.9 prints a and exits 0. uutils aborted. The argument parser also read the rest of the line as the version, so v;s/a/A/ was rejected, and addresses like 1v were refused. I filed this as uutils/sed#669 and the fix in PR #671.

The second gap was comments. GNU sed treats # after a command, including after s/// flags, as the start of a comment. uutils treated # as an invalid substitute flag or as extra characters at the end of the command:

printf 'a\n' | sed 's/a/A/#comment'
# GNU: A
# uutils: invalid substitute flag: '#'

The same happened for p#comment, y/a/A/#c, and {p}#c. A related terminator was missing too: {s/a/A/} was rejected because flag parsing did not stop at }. I filed issue #670 and PR #672.

What I would take to the next rewrite

1. Reuse the recorder. The xargs bugs were not exotic. Default command plus -I, a filename starting with a dash, and an optional argument followed by another flag are ordinary corners. A suite of 44 recorded cases was enough to surface all three.
2. Pin the reference release, and tag the rest. The 9.4 versus 9.5 unpadded-decode mismatch would have buried the new xargs specs in a red CI job. Skipping a version tag is honest; deleting the cases is not.
3. Feed the tool some input when you test a no-op. The v panic was hidden by tests that compiled the script and exited before a line arrived.
4. File a fix with a test. Upstream reviewers can see the case fail without the patch and pass with it.

If you want to try the same specs:

brew install emirmrkaljevic/tap/behave
# or: go install github.com/emirmrkaljevic/behave/cmd/behave`latest

behave run examples/xargs
behave diff examples/xargs --impl /path/to/uutils/xargs

The recorder, the suites, and the upstream links are in the behave repository.