Build an Open Source Project: Behind the Scenes

Starting Point

One of my favorite hobbies is working on Open Source projects. Usually, it starts with solving my own problem. At some point in time, I can assume other people may find my experiments useful, and it's time to open-source the project. I see a lot of people think it’s enough to push the code to GitHub to open-source their project. Technically, yes, it's an essential step. Is it enough? Definitely, not.

If you decide to open-source your project, the first preparation steps can be found at https://opensource.guide/starting-a-project/ guideline. If you didn’t read it, I highly recommend doing that. It’s still not enough, but a good starting point.

Some time ago, I started a project called "xq", which is a command-line XML and HTML beautifier and content extractor written in Go. Using this project as an example, I want to show what I did to make it a little bit more discoverable and usable by other people.

Motivation

Well, you already pushed the source code of your project to GitHub. Why do you need to read next? Very soon, you may realize nobody is using your cool stuff that you were building by night last year. It can be disappointing.

The community around my projects motivates me to continue working on them and improving them, bringing some interesting ideas and even code contributions. It doesn’t matter if it's an Open Source project or a commercial one. It is always inspiring to see if your tool is used by hundreds, thousands, or millions of people. The more, the better. Meanwhile, as far as I always start such projects to solve my own problems, I benefit a lot from the growing amount of feedback.

Another thing that motivates me a lot is the ability to experiment with technologies and try new things. I have a regular job and work on the commercial product. It's always hard to bring something new and unreliable because commercial products are not playgrounds. So my open source initiatives always helped me to get some knowledge of technologies I can't use or experiment with on the main job.

Target Audience

It's quite important to analyze the potential target audience of your project in detail for the project's success. I often see how creators and maintainers expect the same level of skills from their project users. In most cases, it’s a misunderstanding that becomes a problem for both parties.

For example, for my "xq" utility which deals with XML in CLI and is written in Go, I expect the users to know what XML is for and how to use command-line utilities. But I don’t expect the knowledge of Go, the corresponding toolchain, or even any coding skills at all.

Usability

The next important thing is to think about your project as a product that is at least easy to install and start using. Ideally, the value of the product is absolutely clear for the end-user from the first seconds of your product usage.

We like to push our Open Source project to GitHub, but it's code-centric. We should help our users with clear and easy installation instructions. And here is a trap. I use Mac and want my "xq" utility installable using Homebrew:

brew install xq

I checked the guidelines on how to contribute to Homebrew and found that if your repo has zero "stars", you have zero chances to be added. One needs to find alternatives to start with. Ok, Go provides a facility for that:

go install github.com/sibprogrammer/xq@latest

At first glance, it looks good, but it limits the audience to people who already have the Go toolchain installed. It's quite a strict limitation. So in my case, a good starting point was the bash installer for the CLI utility:

curl -sSL https://bit.ly/install-xq | sudo bash

After gaining the first 50-70 "stars", we will be ready for new endeavors like brew or apt install.

But even before thinking about how to make the installation process easy, we need to attract a potential user and show the value of our product. I really like the projects which have a screenshot or animated video that demonstrates the main features right at the beginning of the README.md file. If we are talking about CLI utilities, the comprehensive set of usage examples is also an essential part. If it’s some kind of web product, a link to preinstalled demo server is a must. Life is short, and we all are very busy. You probably have no sales team to convenience the potential users, and people may evaluate your product not longer than just 1-2 minutes.

Marketing

Well, for the majority of us, the word "marketing" has no association with something good, interesting, and what we want to do. But it's not enough to have a good product, we need to tell other people about it somehow.

It is not so hard to start with friends and local communities. Twitter, Reddit, Facebook, LinkedIn, and other social networks may help you to gain the first feedback and attract the first users. I want to tell you the story about "xq" marketing efforts to show how it works in real life.

The company I work for has an initiative called "Research Days," and there is a special event named "Research Days Demos." So, the first presentation I did was at this event for my colleagues. There were not so many people, but some of them liked the utility. I also made a short post in the internal Slack channel related to similar initiatives.

The next two attempts were to tell about the utility on Reddit. One was successful (in terms that there was a discussion and the project attracted a few more users), and another one was blocked by the moderator (and I still have no clue why). Eventually, I got enough "stars" to be able to prepare a pull request to join Homebrew.

Later there was a period of slow organic growth, fixing of bugs, and implementing new features. I researched the ways how to make the installation on Linux more simple, and the assistance of distro maintainers helped a lot with adoption.

At some point in time, I decided to try the power of Hacker News, and it was so impressive. Tons of feedback and feature requests alongside the increasing number of "stars." I'm not a very active Twitter user and have a few followers, but after the post to Hacker News, I found there are a lot of bots on Twitter trying to repost every piece of news. There were even a couple of discussions after such reposts.

There are some more ideas, like joining "awesome" lists or publishing a dedicated article with a detailed feature set description and use cases. In general, I think it's a good idea, after several months of development, to spend some efforts on marketing to improve the product's discoverability.

Save Your Time

Working on a project can easily become boring due to different chores. Sorting out badly written bug reports, manual testing of every change and etc. may become a main reason for dissatisfaction. Because you are working on an Open Source project, spend your time and do it for free. One of the ways to keep yourself motivated is to be focused on answering the question of how to avoid chores and save time.

GitHub allows providing custom templates for new issue creation. Do not underestimate. It significantly helps to streamline the reports and forces the people to answer the questions necessary to the maintainer.

The last thing I want to deal with in the scope of my Open Source projects is the regression bugs. That's why a comprehensive set of tests is a significant time saver and brings the joy of working on the project improvements. Without tests, the maintenance of more or less complex projects after 5-6 release cycles will easily become a nightmare. It's interesting to see how often this truth is ignored.

Writing a set of tests is not enough. GitHub Actions is an excellent facility to organize the CI process not only for pushes to the master branch but for the pull requests as well. Otherwise, it is quite disappointing to find out after the merge of someone's pull request that code style or even tests are broken. It's not so hard to setup the actions. If you check your favorite Open Source project, I guess all of them will have established CI.

With "xq", I went even further and automated the release process using GoReleaser. To publish a new release, the only thing I need is to create and push the Git tag. The corresponding GitHub Action will trigger a release process, and GoReleaser prepares the binaries and changelog based on declared conventions. The result has a high level of predictability, and no manual work is required.

Perks

Open Source projects are usually not about making money. At least in a direct way. If you want to create a product and are not sure how to monetize it, starting this product by open-sourcing it can be not a good idea at all from my point of view.

But there are a lot of other exciting benefits. One of them I already mentioned briefly in the "Motivation" section: learning by doing and working with cutting-edge technologies can be easier in the scope of Open Source projects. Many times after such experiments, I re-used the knowledge in the commercial products as well.

If we a talking about a career, I believe almost every developer with 10+ years of experience should be able to show some code he or she wrote. NDA is not an excuse. Almost every modern software highly depends on Open Source components. So it should be just a matter of time and experience to contribute the fix or improvement to some project. I could be completely wrong, but as a man who did hundreds of tech interviews during the last two decades, it was much easier to build the candidate profile if I could check the code he or she wrote. So while working on Open Source projects, you are definitely investing in the building of your public profile as a developer.

Usually, you can't build a product without using various tools. Some of them can be free, and some of them can be commercial. The great benefit of working on Open Source projects is that a lot of companies with commercial products have special offers for non-commercial development. In the case of the "xq" utility, which is written in Go, I use GoLand IDE by JetBrains. I paid for it for several months but later tried to apply to their Open Source Program. They provided me with a license not only for GoLand but for the whole product pack! Another example is the CodeCov service. I want to track code coverage in an easy way to control the quality and ensure all major scenarios are covered by tests. CodeCov is quite expensive for commercial products (especially the hosted version, which costs ~50K USD), but it's absolutely free for Open Source projects, and this is awesome! If you need project hosting (e.g., for demo purposes), you may try to apply to the DigitalOcean Open Source initiative, and there are other alternatives available. This is just a few examples.

Connecting the Dots

To summarize the article, here are some takeaways I hope you may find helpful.

If you want to build an Open Source project, think about it in terms of the product, not the project, even if we are talking about the library. It should be easy to install and start using. The value for the end-user should be obvious from the first minutes of usage.

Without marketing, people have almost zero chances to know about your cool project. It is surprisingly not so hard to start. And it's connected with your long-term motivation in more ways than you can initially think about.

Your Open Source project should not become a tedious job. One of the ways to achieve that is to automate all routine work and enforce the policies. In the end, the release should be a joy, and reading user feature requests and bug reports should not make you cry due to the lack of essential details.

Last but not least, working on Open Source projects is rewarding. Sometimes it's not so obvious, but if we are talking about the long-term perspective, it definitely is.