This is how I built an analytic tool for Digital Products

#javascript #mongodb #buildinpublic

Hey guys,

Recently, I’ve fallen in love with data analytics of all kinds.

As I was looking for a side hustle, I explored different options and stumbled upon the world of digital products. Platforms like Gumroad, Ko-fi, and others dominate this market.

But I hit a roadblock right at the beginning: there are hundreds of products.

So, which one should I sell?

After researching various articles and platforms, I realized that there wasn't enough solid data to make a well-informed decision.

This led me to start tracking trends on one of the biggest digital product marketplaces: Gumroad.

v0.1

I started with a simple JavaScript script that performs basic API requests to the publicly available Gumroad API. You can easily see these requests in the browser's Networking tab:
https://gumroad.com/products/search?&tags%5B%5D=vrchat&from=10

It was a good start. I decided to track the 10 most popular tags and fetch data daily.

Tech stack I used:

Node.js
Axios + axios-retry + axios-rate-limit
FS to write to the file system
AI tools to speed up coding

It worked out well. I collected data for several days in a row and found it interesting enough to expand further. I also received support from the community on X, so I decided to take the next step.

v0.2

I realized that knowing the most popular tags wouldn't be enough — the real potential lies in identifying the tags that are growing.

In my next iteration, I collected sub-tags from the most popular niches and gathered over 5,000 tags this way.

This generated more than 20,000 rows of data daily, so I needed a storage solution.

Since it's still possible to host a free cluster on MongoDB Atlas and MongoDB pairs well with JavaScript, I chose it as my storage option. Now, the data is stored, and I can run various analytics.

Another challenge was the time required to collect the data.
It initially took five hours. That's when I learned about proxies. After setting up several proxy servers to process the data in parallel, the collection time dropped from five hours to just 30 minutes.

v0.2.1

I was running all the scripts manually, so I decided to automate the process.

I hosted the source code on a remote server on Hetzner and set up cron jobs via crontab.

0 18 * * * /usr/bin/node index.js collect-data
0 21 * * * /usr/bin/node index.js collect-totals

The first job collects the data, and the second one aggregates it for a summary. It removes duplicates and calculates the growth compared to the previous day.

const [mode = 'collect-data'] = process.argv.slice(2);
if (mode === 'collect-data') {
    return collectData();
}
if (mode === 'collect-totals') {
    return collectTotals();
}

v0.3

I decided to create a simple UI for this project and share it publicly. This is how gumroadtrends.com was launched.

I used:

VueJS for the UI
Bootstrap for the CSS framework
Chart.js for visualizing the data
GA4 for usage analytics
I also used Hetzner and PM2 for hosting since I already had a server for hosting all my products.

Here’s what the UI looks like:

v0.4

It’s hard to understand trends with only seven days of data, so I plan to continue collecting data on autopilot and revisit it after a few weeks for deeper insights.