Open Sourcing Salesforce CLI

Salesforce CLI continues to evolve as we add more features and commands that support Salesforce development. We’re excited to announce the next step in that evolution. Over the years we’ve kept open source software (OSS) in mind, as evidenced by several repositories we’ve already opened up to the community (I’ll cover these later). However, we’ve decided to commit to moving towards 100% OSS! While we open up the code base, we’ll also be refactoring where possible and re-organizing where those repositories, plug-ins, and issues live. Read on with me while I explain the steps we’re taking towards that goal and where we’re looking to go next. In this post I’m going to cover:

• Why we’re moving towards OSS
• What you have access to now
• Steps we’re currently taking

Why go open source?

While going 100% OSS has become a primary goal for the CLI team, OSS has been an integral part of Salesforce for quite a long time. You can read more about Salesforce’s commitment to OSS here. Also, here’s a great write up by Salesforce Principal Architect, Ian Varley on the subject: Salesforce Is Powered By Open Source. Open source has always been a part of the CLI conversation. In fact, we’ve already opened up a few repositories to the community as you’ll see later in this post. That being said, we were taking a piecemeal approach toward OSS and were deciding what to open up on a case-by-case basis. As the CLI continues to grow, we’re seeing more and more that going full OSS will actually help us solve or prevent some looming challenges:

• Keeping open and consistent lines of communication with our users
• Communicating to users what we’ve been working on and where we’re going
• Monitoring issues between internal and external channels can cause some bugs to be overlooked
• Finding enough bandwidth for the CLI team to work on “low priority” features that we recognize as being useful

Shifting our focus towards open sourcing the CLI rather than taking a piecemeal approach will help us more quickly mitigate some of these challenges. Here’s a more specific list of “Whys”:

• Incorporate customer feedback at earlier steps in our process
• Build transparency and trust with our users
• Give the community an opportunity to dig into the code and understand how it works
• Open Salesforce CLI to pull requests and bug fixes from the community
• Leverage those contributions to free up the CLI team to focus on larger bugs and features
• Surface bugs and edge case issues much more rapidly than our current internal QA process allows
• Create a direct line of communication with our customers
• Ensure that our growing feature sets will continue moving in a direction that parallels the needs of our users

What goes into the CLI?

Let’s take a quick look under the hood. The CLI as it exists today is an amalgam of various plug-ins built on top of oclif. When you run sfdx plugins --core in your terminal, you’ll see an output that displays a list of those various plug-ins. We also utilize several oclif plug-ins to enhance the user experience, and we bring in others to do things like gather analytics or provide users with their own custom plug-in generator. One key Salesforce plug-in to take note of is salesforcedx.

salesforcedx is the parent plug-in for multiple child plug-ins that provide many of the commands you use on a daily basis.

Originally, the CLI was built from one of those “child” plug-ins: salesforce-alm . This is where the CLI began and quickly grew to over 170 commands with multiple internal teams currently contributing code. The bulk of the force commands still come from this one plug-in. As you can see, we’ve since grown beyond salesforce-alm and now host multiple packages all brought together to build the current version of the CLI. I’ll share more about salesforce-alm in a bit because it’s a big part of this effort.

What you’ve got access to now

While it may seem as though we’re unveiling a brand new OSS concept for the CLI, this really has been on our radar for some time. The real update is that we’re making OSS a top priority this year. We’ve recently released a “snapshot” of the salesforce-alm code as it exists today (see below). This will give you a glimpse into arguably the most important code base of the CLI. Not only that, we’ve already opened up several packages and libraries that you can start utilizing in your own projects:

• https://github.com/forcedotcom/cli-packages
• https://github.com/forcedotcom/sfdx-core
• https://github.com/forcedotcom/salesforcedx-templates
• https://github.com/forcedotcom/sfdx-dev-packages
• https://github.com/forcedotcom/sfdx-plugin-generate
• https://github.com/forcedotcom/cli/issues – (Report issues with the CLI here)

You can bring many of these into your own projects now. Send up a PR or just play around with a few of the functions to gain insight into how they interact with the Salesforce API. You can also use them to help build your own CLI plug-in!

The Snapshot

We wanted to make it clear that open sourcing the CLI really is a priority for us. As a result, we’ve created a “snapshot” of the salesforce-alm to whet your appetite a bit. It’s a read-only picture of the code as it exists right now. No PRs or issues will be accepted on this repository. Eventually, you’ll have full access to run the various parts of code that make up the snapshot on your own machines. FYI, you can expect updates to the snapshot during each major release (3 times a year).

A GitHub organization for Salesforce CLI

Part of the re-organizing effort will be to create our own GitHub organization for the CLI. Once it’s up and running you’ll be able to keep an eye on the changes as we progress toward the goal of open source. One of the main reasons for this is simply because forcedotcom, where the current open source CLI repos live, has bloomed to such a size that it’s become rather difficult to discover new CLI specific repositories that you might want to work with, or where to file an issue. Having a single Salesforce CLI organization will be more manageable for the CLI team while providing a simpler way of discovering plug-ins or libraries users may want to leverage for their next project. However, we won’t be able to move over ALL the things just yet. For example, the issues repository will still be tied directly to forcedotcom. This is as a result of not being able to transfer ownership of the issues in GitHub at an organization level. We don’t want to lose that history or any outstanding bugs that we haven’t addressed yet.

A single source for issues

We’re consolidating our issues into one repository you may already be familiar with: https://github.com/forcedotcom/cli/. Instead of bouncing communications across multiple repositories and as a result potentially losing some in the shuffle, consolidating issues to one repository will help us ensure the team is staying on top of any major problems. As mentioned above, we won’t be able to bring this into the Salesforce CLI GitHub organization just yet. Regardless, consolidating issues to one repository will make it easier for the community to file issues against the CLI and keep track of them. Also, it will remain a searchable resource for community workarounds and previous fixes. If you see a repo that has issues turned off, this will be where you should go.

Public roadmap, release notes and Github issues. Oh my!

Using the existing issues-only repo that you already have access to, https://github.com/forcedotcom/cli/, we will also begin adding our release notes as well as a public roadmap so that you can keep track of where we’re going and any announcements that will be headed your way. This will increase our transparency and communication with our users.

Next steps to open source

Open source by default

Moving forward we’ll be working with anyone who contributes to the code base to ensure that any new plug-ins will have open source as part of their Generally Availability (GA) timeline. That means that our users will not only be able to utilize these plug-ins and libraries in their own projects but they will also be able to submit PRs for features or bug fixes they’d like to see in place. While this will be treated as a mandate for our internal teams, there may be scenarios in which going full OSS simply won’t be an option. I’ll go over a few examples in the next section. Fortunately, you can be certain that these will be few and far between.

Breaking up salesforce-alm

As mentioned earlier, salesforce-alm is where it all began. Originally, we were considering breaking out bits and pieces of CLI functionality and open sourcing those as we went along while also sticking with a mono-repo structure. We’ve since settled on the idea that it will be more manageable to pull out sets of commands as their own individual plug-ins and in doing so, break up the mono-repo. Where before you would have seen salesforce-alm, after you’ll see multiple new plug-ins for each group of related commands:

Before:

After:

Unfortunately, this isn’t going to happen overnight. The CLI team is committed to breaking out and open sourcing all of the commands that the team owns (we don’t own them all). Doing so will also allow us to refine that process and help us to advise other teams on the best path they can take towards open sourcing their own commands. Not only will we break out commands into their own plug-ins, we’ll also need to simultaneously refactor outdated code and design patterns. We’ll need to do things like clear the git history of any sensitive internal data, remove links to internal docs and configs, etc. We’ll also look toward removing or even rewriting some of those internal docs for public consumption. We need to clear up config files and first get approval for these changes with the multiple internal teams that contribute to our codebase. Those teams will have their own limitations on what will be ready for OSS, if at all. The list goes on and on. Needless to say, it will take a lot of effort but we’re committed to and excited about our move towards open sourcing Salesforce CLI!

Recap

• Salesforce CLI has set a goal to go 100% OSS.
• We are breaking out the CLI commands into their own repos and open sourcing them as we go.
• This will take a fair amount of time and effort from our team but we’re already taking steps in that direction.
• We created a read-only snapshot of our salesforce-alm for you to take a look at here as further proof of our commitment to making the CLI open source. The snapshot will be updated 3 times a year.
• You have access to several OSS repositories related to the CLI right now.
• We’ll be working to make sure any new plug-ins have OSS as part of their GA timeline.
• Release notes, Roadmaps and Issues will remain here: https://github.com/forcedotcom/cli/

About the Author

Ricardo Viera is an engineer on the Salesforce CLI team. Over the years he’s made several career shifts beginning in the world of 3d Animation then moving into UI/UX design and most recently making the jump into the world of software development. When not taking pictures of space with one of his 12 telescopes, you can find him on a driving range working on his golf swing or negotiating with his toddler to finish her dinner or she’ll get no dessert!