This article was published on Thursday, June 1, 2023 by Yassin Eldeeb @ The Guild Blog
The best and most obvious way to judge an open-source project is to look at the code but this can be
kind of tedious and sometimes you don't like what you see there, so an alternative that we have all
naturally developed on our own or have been advised to, is to see how many people have starred a
project, and then pick the one with the most stars."For example, React.js has 207K stars compared to Angular's measly 88K stars, so we can conclude
that React.js is a better framework" — Ben AwadWhile the above is a joke (not that React isn't a better frontend framework), it is not a good idea
to judge the quality of an open-source project by how many stars it has.How many times have you gazed upon an open source project, and wondered fabulously about the
correlation between the number of stars garnered and the actual value of the project?If it's zero, then ok, you're a better person than me.But if not, then allow me to introduce you to my recent experiments and research. It's likely that
this will be your first encounter with this bizarre market.## Buying GitHub StarsYep! That's a thing. Just as purchasing Instagram likes is a known used marketing technique, buying
GitHub stars is equally tangible.
**But I won't just tell you that buying GitHub stars is real. That would be so disappointing, right?So I did it! I have tried two platforms. I won't say any names or links, as I would never advocate
for such a thing.I thought it would be a fascinating and very educational experiment (not for my wallet, though!). So
let's dive into it!#### Premium StarsBear with me. I know that you're thinking: "what the hell?! There are different star types??!" Yes,
there are!Premium Stars are the kind that is much more expensive, the most legit-looking, and most notably,
those stars have a higher chance of staying over the long term.So I put on Incognito mode and started hunting using my second identity "Boogeyman".
And so it costed me 20 Euros for 25 stars which is €0.8/star, it's expensive (not if you deduct it
from your taxes), but I'm gonna show you how they look like in a minute.It took six hours for my order to complete, and the accounts look legit; each has a profile picture,
different companies that they work for, a couple of repositories, and a contribution to one or more
open-source projects, next to being a GitHub member for over a year.
#### Cheap Stars"Whenever you see premium goods, remember, the bargain bin isn't far behind!" — Probably meThere are also cheaper options like this one:
This package costed me 8.19 Euros for 100 stars which is €0.08/star.As you can see from the receipt, my order is number #57189, so it's definitely not something that's
only used every once in a while 🙂.Now, what is the difference?Well, the biggest one is that those are brand new accounts — they were created at the time of my
order. They don't have any fake personal information or repositories or contributions.And after a month, they are all gone. GitHub detected and banned them.
What's crazy is that you can reach out to them , and they are going to redo your order for free 🙂### Detecting Fake StarsThen I wondered if any tools can expose those repositories 🤔And I discovered Astronomer, a tool to detect illegitimate
stars from bot accounts on GitHub projects.It works by fetching data from every GitHub user who starred a repository and computing how likely
it is that those users are real humans. It works using their "Trust algorithm".The trust score is computed based on different factors: The average amount of lifetime contributions among stargazers
- The average amount of private contributions
- The average amount of public-created issues
- The average amount of publicly authored commits
- The average amount of publicly opened pull requests
- The average amount of public code reviews
- The average weighted contribution score (weighted by making older contributions more trustworthy)
- Every 5th percentile, from 5 to 95, of the weighted contribution score
- The average account age — older ones are more trustworthySo, I've tried Astronomer on three repositories to see the results.Premium Stars: ``` Fetching contributions for 69 users up to year 2013 Building trust report...ok
Averages Score Trust
Weighted contributions: 18268 B
Private contributions: 370 A
Created issues: 91 A
Commits authored: 259 C
Repositories: 41 A
Pull requests: 36 A
Code reviews: 6 C
Account age (days): 1619 B
5th percentile: 12 A
10th percentile: 16 C
15th percentile: 20 D
20th percentile: 20 E
25th percentile: 24 E
30th percentile: 24 E
35th percentile: 24 E
40th percentile: 28 E
45th percentile: 332 D
50th percentile: 1106 B
55th percentile: 1692 B
60th percentile: 2770 A
65th percentile: 6422 A
70th percentile: 10958 A
75th percentile: 15598 A
80th percentile: 26596 A
85th percentile: 34116 A
90th percentile: 69967 A
95th percentile: 85147 A
Overall trust: B
**Cheap Stars:**
Fetching contributions for 24 users up to year 2013
Building trust report...ok
Averages Score Trust
Weighted contributions: 856 E
Private contributions: 21 E
Created issues: 3 E
Commits authored: 87 E
Repositories: 4 E
Pull requests: 3 E
Code reviews: 1 E
Account age (days): 36 E
5th percentile: 3 D
10th percentile: 3 E
15th percentile: 3 E
20th percentile: 3 E
25th percentile: 3 E
30th percentile: 3 E
35th percentile: 3 E
40th percentile: 3 E
45th percentile: 3 E
50th percentile: 3 E
55th percentile: 3 E
60th percentile: 3 E
65th percentile: 3 E
70th percentile: 3 E
75th percentile: 3 E
80th percentile: 4 E
85th percentile: 4 E
90th percentile: 4 E
95th percentile: 4 E
Overall trust: E
**Legit Stars (as a control check):**
Fetching contributions for 139 users up to year 2013
Building trust report...ok
Averages Score Trust
Weighted contributions: 118821 A
Private contributions: 2407 A
Created issues: 72 A
Commits authored: 1300 A
Repositories: 74 A
Pull requests: 101 A
Code reviews: 69 A
Account age (days): 2538 A
5th percentile: 145 A
10th percentile: 711 A
15th percentile: 2726 A
20th percentile: 4144 A
25th percentile: 8535 A
30th percentile: 13650 A
35th percentile: 20728 A
40th percentile: 23322 A
45th percentile: 28244 A
50th percentile: 37636 A
55th percentile: 42536 A
60th percentile: 55776 A
65th percentile: 78956 A
70th percentile: 93676 A
75th percentile: 129164 A
80th percentile: 194463 A
85th percentile: 231380 A
90th percentile: 286798 A
95th percentile: 559934 A
Overall trust: A
So, that's scary. The premium stars scored decently as `B`, but the cheap ones resulted in the
lowest score — `E`. Who knows, maybe there's an ultra-premium stars service that's scoring an `A`?## Lessons LearnedAs you shouldn't judge a book by its cover, you shouldn't judge an open-source project by the number
of stargazers. As we saw in this article, this is an option that they are not legitimate. It's like
any other social media platform where fake accounts and fake likes exist.Instead, ask your colleagues or your community on Twitter why you should pick this project over
another. You can also start a new discussion or create an issue on GitHub asking for other people's
experiences. And if that's not enough — you can review the project's code. Though, we all know,
that's not something we have time and energy for on a daily basis.Choosing a high-quality open-source project can be a challenging task. However, there are key
factors to consider when making such decisions. As [The Guild](https://github.com/the-guild-org)
whole structure
[was created to ship long term open source projects](https://the-guild.dev/about-us), we've also
have developed a sense on how to identify good open source projects, based on long-term
sustainability and collaborative ideas from everyone involved.We recommend considering the following:1. Long-term sustainability: Look for projects that demonstrate a commitment to continuous
development and maintenance. A good place to start is the contributors tab on GitHub, as it shows
you the whole timeline span of a project. For example
[GraphQL Mesh](https://github.com/Urigo/graphql-mesh/graphs/contributors) and
[GraphQL Codegen](https://github.com/dotansimha/graphql-code-generator/graphs/contributors) are
consistantly being worked on for years, Hive is
[increasing work gradually and consistanly](https://github.com/kamilkisiela/graphql-hive/graphs/contributors),
on [GraphQL Yoga](https://github.com/dotansimha/graphql-yoga/graphs/contributors) you can see how
it started, then abandoned, and later returned strongly after
[The Guild took over maintaince from previous maintainers](https://the-guild.dev/blog/announcing-graphql-yoga-v2)
and the [Nextra project](https://github.com/shuding/nextra/graphs/contributors) significantly
increased work after [we've joined as maintainers for v2](https://the-guild.dev/blog/nextra-2).
2. Community engagement: Assess the level of community involvement and collaboration within the
project. Here you can check out the
[discussions tab on GitHub](https://github.com/dotansimha/graphql-yoga/discussions) or the
[GitHub community graph](https://github.com/dotansimha/graphql-yoga/graphs/community).
3. Code quality: Evaluate the quality of the codebase, including readability, maintainability, and
adherence to best practices.
4. Responsiveness: Check how responsive the project maintainers are to issues, pull requests, and
community feedback. The [pulse tab on GitHub](https://github.com/dotansimha/graphql-yoga/pulse)
can help with that.
5. Popularity and reputation: Consider the project's reputation within the community, including the
number of contributors, stars, and positive reviews.By following these guidelines, you can increase your chances of selecting a high-quality open-source
project. At The Guild, we strive to [embody these principles](https://the-guild.dev/about-us) in our
own work, creating sustainable and impactful solutions for the long term.
And, there you have it!Got some seed money for your next startup, and you wonder how to spend them? Throw it into building
truly worthwhile projects. That's an investment you won't regret. 😉
Top comments (0)