Over the past few years the Salesforce CLI became widely adopted as the preferred tool for developers to interact with Salesforce. As we constantly improve our tooling this also brings some changes, like how you should parse CLI output. This blog introduces changes that are coming with Spring ’20, and how you can adopt them.
Salesforce CLI started out as a simple tool to help introduce source-driven development, and has grown to become the standard way to interact with Salesforce via the command line. The CLI exceeds millions of command executions every week. As the CLI grows in popularity, we work hard to standardize the commands, flags, and output. For example, we implement --json on all commands so you can easily build tools and scripts on top of the Salesforce CLI.
This blog post aims to clarify several things:
- Output Salesforce produces.
- Explain the difference between output streams.
- Describe the impact on Salesforce CLI and upcoming changes.
- Show best practices when using output in scripts, tools, and continuous integration.
If you are new to the concept of Salesforce DX tools or Salesforce CLI, here are a few Trailhead projects and modules you might want to take.
If you’re already using or thinking about using Salesforce CLI for scripting and CI/CD, read on to learn more about the CLI and your day-to-day work.
Human Readable and Machine Output
When we run a CLI command, we want to produce output that’s easy to read. The type of output we produce depends on the intended audience. This will be very different for a human than for a machine. Let’s look at an example:
-> ls -l
total 20
-rw-r--r-- 1 username staff 12 Feb 5 21:08 example.txt
-rw-r--r-- 1 username staff 8 Feb 5 21:10 example2.txt
As you can see, this is fairly easy to read. It is a table that contains all the files in the current working directory, including who owns them, the file size, when they were created, and the file name. For a computer, this becomes much more difficult, as it’s not in a parsable format. The first line is not in parsable format, so that would need to be removed. Then, to read the table, a computer would iterate over each line parsing the text by whitespace, and indexing the information based on the directions from a script or tool. This works, as long as the output NEVER changes. Once the information changes in any way, it will break the tool or script that is trying to parse information.
Because we want your scripts to work, and we want to be able to improve the human readable output without fear of breaking your scripts or tools, we enforce --json on all commands. The JSON output is protected by the Salesforce CLI Deprecation Policy.
Let’s reiterate that.
We do not support parsing or relying on the human readable output of the Salesforce CLI! JSON output is built for tools and scripts to read.
Note that if we must make changes to JSON output, we will warn you and offer you a period of time to react.
Standard out (stdout) and Standard error (stderr)
There are two terminal streams that programs can send output to: stdout and stderr. Based on their names, it makes sense that usually errors and warnings are sent to stderr and everything else is sent to stdout. However, there is plenty of other useful information to send to stderr, like progress status or diagnostic/debugging information. In general, stderr is thought of as a random stream of output that shouldn’t be parsed or relied on because anyone and anything can send any information to it at any time. For example, there is a warning when a new CLI version is available that is thrown to stderr by a library we don’t control.
There are a lot of opinions on these streams, but in general all CLIs will follow the guideline that stderr should not be parsed.
Using stderr provides many benefits. A common use is that users can separate the results of the command from other noise, like warnings or progress.
# stdout.txt will contain the echo, stderr.txt will be empty
echo "print to stdout" > stdout.txt 2> stderr.txt
Let’s look at how Salesforce uses these streams for different output and what is changing in Spring ‘20.
What about Salesforce CLI?
Salesforce CLI strictly followed those stdout and stderr guidelines, or so we thought. We took the approach that all errors went to stderr, including when --json was provided. That meant that tools had to parse stdout when the status is 0 and stderr when the status is greater than 0. We tried to disable ALL output other than the final JSON output. This worked well except in cases when “anyone and anything” sent something to stderr in a way that we didn’t expect. In these cases, JSON parsing broke.
sfdx force:org:display -s --json > stdout.txt 2> stderr.out
# if there is a random warning or debug information, the following will fail
more stderr.txt | jq # parse the json error with jq
We could have continued to try and capture all messages to stderr except the one we control, but this continues to go against the guideline that stderr should not be parsed or relied on. Instead the JSON object, even for errors, should always go to stdout. Since this would break existing tools and scripts, we introduced an environment variable to move all JSON output to stdout. That way anything can still be sent to stderr without it ruining tools trying to parse the JSON output.
# ignoring stderr so the jq parsing always works
SFDX_JSON_TO_STDOUT=true sfdx force:org:display -s --json 2> /dev/null | jq
We released SFDX_JSON_TO_STDOUT back in v44 and said it would be the default behavior in v45. That did not happen, but it is now going live in v48 which will be released 2/15/2020. If you rely on parsing errors from stderr when using --json then please update your tools and scripts, or set SFDX_JSON_TO_STDOUT to false. You can expect more warnings and errors to be sent to stderr even with --json set.
Sometimes a piece of information goes to the wrong place. For example, in version 47.18.0 of the salesforcedx plugin for the Salesforce CLI, we moved the progress bar output to stderr where it belongs. If you think output is going to the wrong place, please feel free to file an issue.
Scripting Best Practices
Scripts can be used for a wide variety of things. The power of scripting is that you create a repeatable process for automated testing or to simplify your day-to-day work. For example, setting up your project before you start working, or creating a scratch org with all the metadata, data, permissions sets, and whatever else you might need in that org.
When we talk about CI/CD, we are talking about scripting. It is about doing a bunch of repeatable actions to test or deploy your code without human intervention. For example, the CI tool could use the same scratch org creation script so the development team is creating the same org setup as your CI.
Anyone who has worked on CI/CD scripts knows that they can fail quite often based on the tools or environments that you use. The goal of JSON output and the Salesforce CLI Deprecation Policy are to prevent breaking scripts and to give you plenty of time to update them if we need to break them in a future version.
As we learned above, JSON output is the supported way for scripts to consume data from CLI output. This means you should never, ever, parse non-JSON output and expect it to last long. Changes like the progress bar or improvements to the human readable output will quickly break your scripts.
Let’s look at an example. Say we want to enqueue a long running deploy, do some other stuff, then report on that job and wait until it is done. We can extract the ID from the human readable output using sed.
# Grab and store the enuqueued job ID using sed
ID=$(sfdx force:source:deploy --metadata=StaticResource -w=0 | sed 's/^Id: \(.*\)/\1/')
# Report on that job later in the script
sfdx force:source:deploy:report --jobid=$ID
Then due to overwhelming community response, we change “Id: ” to “Enqueued Deploy Job Id: ”. This would break the above script. Instead, it should be using the --json flag to get the Id.
# Grab and store the enuqueued job ID using jq
ID=$(sfdx force:source:deploy --metadata=StaticResource -w=0 2> /dev/null | jq -r .result.id)
# Report on that job later in the script
sfdx force:source:deploy:report --jobid=$ID
Not only is it more clear because you don’t have to use regular expressions, but it will also protect against future CLI updates. You will also notice 2> /dev/null that will throw away stderr to ignore any unwanted messages that would break our JSON parsing.
Another benefit to the JSON output is that it usually contains a lot more information, including special information about the results or additional information on errors in case you want to take special action for particular errors. We will cover this in an upcoming blog post.
Recap
Four years ago Salesforce made a commitment to improve Salesforce DX, our developer experience. To support this promise, we’ve focused on and improved developer tools, including Salesforce CLI- and we are nowhere close to slowing down. We are committed to continually improving the developer experience.
I highly recommend learning more by doing the Trailhead projects and modules mentioned above:
We also have CI templates for a variety of CI tools which help you get set up with automation faster:
- Salesforce DX Developer Guide: Continuous Integration
- Sample Repositories
- CircleCI Orb
- Github Actions
- GitLab Templates
Hopefully this gives you an idea on how the Salesforce CLI handles output and some ways to make sure they don’t break moving forward. The big takeaways are to use --json and always parse from stdout.
I personally want to thank everyone who has taken the time to use our tools and report any bugs or feature requests. We know there is a long way to go and we are all very excited to see the Salesforce Customer Success Platform become the best developer-focused platform on the market.
Top comments (0)