DEV Community

Mubashar Iqbal
Mubashar Iqbal

Posted on

From Idea to 4M Page Views in 4 Weeks

Originally published on Hacker Noon.

Last Tuesday saw the official launch of Will Robots Take My Job? and 5 days later we have passed 500K visitors and 4M page views.

To say this surpassed any expectations we had would be a major understatement. This is a recap of how the project got started and the first 5 days after launch.

The Back Story

Just over a month ago Dann Petty launched Freelance TV. I’ve freelanced on and off for over 10 years, so I was happy to back the documentary, and joined the Slack community.

Dimitar Raykov has also joined and messaged me, and we started talking about the possibility of working together on a project. We decided to start with something small and quick to see how we might like working together.

Dimitar mentioned his idea of building a website around a report published in 2013 about the probability that jobs would be automated thru Machine Learning and Mobile Robotics.

I’ve been interested in the AI/ML space for a while, even co-founding BotList.

We’ve already seen many chatbots assist people in their jobs, even replacing them in some instances, the HR and Travel categories are good examples of this.

We decided to move forward with the project.

Just showing the percentage automation risk, would not make for a very interesting user experience, so we looked for additional data that would help. We found additional information about the jobs in the report via the U.S. Bureau of Labor Statistics website. We were able to download salary data and estimates for number of people employed.


The Build

Dimitar would handle the design & front end and I would handle the backend and hosting duties.

We wanted to make it easy for people to find jobs, since the report only covered 702 jobs, finding similar jobs would be an important feature. We decided to go with Algolia to achieve this. Their search matching is great (even with misspelled words) and the synonym feature would make it easy to find related jobs. We signed up for the trial and thought the free tier would mean we’d get able to run the site with no cost – more on that later!

We started working and I built a quick prototype on LaravelPHP using Bootstrap, while Dimitar worked on the design.

Once the design was complete, Dimitar built the pages out in static html pages, and passed them over to me to integrate.

Some fun features we added as we worked:

  • Algolia Instantsearch to power the job search autocomplete, why reinvent the wheel.
  • Easy job page link copying and sharing to Twitter and Facebook.
  • Custom open graph share images for each job.
  • Random color palette for each visit. Go ahead and try it! Each visit you’ll be assigned one of ten palettes.

All in the build took about 3 weeks, of us working on evenings and weekends.

I setup the website on a Digital Ocean droplet I had for other projects, and the site was live, just nobody knew it. We share the site with a few friends and got some early feedback.


The Launch

Given my affinity for Product Hunt that seemed to be the obvious place to launch our project. I reached out to Ben Tossell and discussed the project with him. He agreed to hunt it for us. Dimitar created the thumbnail image we wanted to use for the product on the site, and prepared some images to show some of the features, and we passed those on to Ben.

The site appeared on Product Hunt about 3:15am (et) on Tuesday (May 30th). I like to be around when something is first hunted, to add a comment and respond to any immediate questions. Since I was up, I decided to also post the site to Hacker News, and Dimitar posted it to Designer News.

We immediately got traction on all three sites. We quickly moved to the 2nd spot on Product Hunt, and my post moved to the front page of Designer News and Hacker News. As I was going to head to sleep, I noticed the visitors start to tick up. Google Analytics showed we have 48 people on the site, that jumped to 88, and then to 327. No going to bed now.

It wasn’t until about 5am when I finally got to bed, the visitors peaked out at around 440 users.

After my short nap, had to get up and get the kids ready for school, I saw that we’d been written about in The Next Web. That had pushed our visitors higher, we had over 500 visitors on the website.

We quickly passed the Agolia free tier limit and upgraded to their Starter package. 1M operations a month, we should be good with that.

Dimitar started to reach out to journalist to help spread the word about the website. Having the initial success, helped frame the outreach, which would lead to further success.

Visitors continued to rise through the day, the article from TNW was syndicated to a number of sites, including MSN and AOL. We jumped to 822 visitors and then peaked out during the day at over 1400.

Day 2

We continued to get have articles written on all sorts of website, most interestingly not just in the tech space. General news sites in Arizona, Texas and Washington state wrote about us, and a post on BarStoolSports lead to a massive jump in visitors.

We peaked out with about 2000 visitors on the site!

We were rapidly approaching the Starter tier limit on Aloglia. I couldn’t justify the $300 to upgrade to the next tier, and started investigating alternatives. I tweeted about the situation, and Algolia responded and offered to help us out by raising our usage limit. That saved us a bunch of time and worry, the support from Algolia is much appreciated!

The full search alternatives to Algolia wouldn’t have been much cheaper and a lot more work. Running our own server with Lucene or Elasticsearch would have been one option, but the cost of hosting and the time sunken to get that operational wouldn’t have been a good use of my time.

Our best alternative would have been to switch to TNTSearch for Laravel Scout, no additional server would be required and writing a JSON API around the search would have been relatively straight forward in Laravel. Thankfully none of this ended up being required.

Day 3

We had a very steady stream of visitors all day, ended by topping the InternetIsBeautiful sub-reddit.

Although we finished the day with over 5000 votes, we unfortunately didn’t make the Reddit popular page, perhaps next time!

Day 4

I decided it was best to upgrade the server, we were getting pretty steady visitors and although the server was handling the load pretty well, I wanted to have some spare capacity, and ensure the pages loaded very quickly for our visitors.

Laravel Forge makes it ridiculously easy to setup servers, my only concern was adding SSL to the new server. I used LetsEncrypt on the original server, but you can’t do this until you’ve pointed your DNS to the server. Luckily I remembered that I had some unused SSL certificates in my Namecheap account. I ordered a SSL cert, and in a few minutes the new server was up and running.

I crossed my fingers as I updated the DNS to the new server. Luckily we managed to get things migrated with zero downtime.

The extra capacity came in handy. Although we didn’t get another big surge in visitors, the average was higher thru the day. We did end the day with a peak of 1600 visitors which appeared to be coming from Google search. We’re still unsure what caused that surge in visitors.

Day 5

Finally some time to breath. The weekend lead to lower visitors levels, but with about 300–400 visitors on the site through the day, we’re still pretty happy!

The Results

Although we topped the Product Hunt homepage for a while on launch day, we ultimately ended up in the 2nd spot. Falling behind Andy Rubin’s Essential phone.

We have close to 950 votes, I’m hoping to reach the 1000 vote club for the 4th time before soon :)

We stayed on the Hacker News homepage for a few hours, we topped a popular sub-reddit for 3 days, and appeared in many popular sites, including: TheNextWeb, LifeHacker, Mashable, MSN, FoxNews, AOL, BarStoolSports, BusinessInsider, Geek.com, Vice, Gizmodo, BoingBoing, and Fortune.

By the end of Saturday, just 5 days, over 500K people have visited the website and generated over 4.2M page views, an average of over 6 pageviews per session. We expect the visitors to slow down, even a significant drop will leave us with a lot more traffic than we expected.


The Lessons

Just because you build something small and quick, doesn’t mean it won’t have a big impact. Will Robots Take My Job? is easily the most successful personal product launch I’ve been involved with.

Yes I’ve been involved more high profile, high traffic launches, but those involved big marketing teams and big budgets. We are a developer from Upstate New York, a designer from Bulgaria, with no budget.

Make it fun

Dimitar and I had fun building the website, and working together for the first time, but that’s not the only thing what I’m referring too. We made the website fun for our visitors. Instead of being dry and just displaying numbers, we injected some fun into the website. Any job highly likely to be automated was presented with a “Your are doomed message”.

Make it Shareable

Although it wasn’t easy we made it so that every job had a custom share image, so when people shared their job on social media it presented their specific automation likelihood rather than a generic image. Facebook and Twitter accounted for over 100K visitors, which we think is large part due to our custom share images.

Be Available

Via email, via social media, via comments. We tried to respond to anyone and everyone who had a question or comment. We believe this lead to more articles, more shares and more votes. It takes time, but if you don’t dedicate your time to your project why should any one else.

Be Grateful

Both Dimitar and I are very grateful to everyone who wrote about, shared or just visited the site. None of this would have been possible without you.

We’re especially grateful to Algolia for the support, we may need to discuss raising operations limit again 😁.


If you haven’t had a chance to visit Will Robots Take My Job? yet, go ahead and try it out now. Come back and leave a comment with what you think.

Originally published on Hacker Noon.

Top comments (11)

Collapse
 
walker profile image
Walker Harrison

this is awesome (and scary...). Just to confirm: the numbers all originate from the 702 jobs in the initial report with connections to similar roles that you guys made?

Collapse
 
mubashariqbal profile image
Mubashar Iqbal

Yep the original report focused on 702 jobs, and we just report the same, but we've added synonyms so if you search something that isn't the same job title as in the report we link you to the job that was listed. Lots of fuzzy matching on the job titles too.

Collapse
 
walker profile image
Walker Harrison

Interesting. Determining whether job titles are interchangeable is an data science project unto itself!

Collapse
 
jess profile image
Jess Lee

Really fun read and so cool that many sites picked it up. Great job!

Collapse
 
mubashariqbal profile image
Mubashar Iqbal

Thanks! It definitely was fun to see all the surges in traffic.

Now back to the boring work :)

Collapse
 
fhferreira profile image
Flávio H. Ferreira

Awesome :), congrats.

Collapse
 
mubashariqbal profile image
Mubashar Iqbal

Thanks, has been a fun ride! Not sure where it will lead, but it's already opened a few doors we weren't expecting, perhaps a further post to discuss that later.

Collapse
 
eonist profile image
Eon

Great story. Awesome that you posted it on dev.to Seems like a great community 😻

Collapse
 
ben profile image
Ben Halpern

Just got around to reading this after you posted it the other day. Great advice, thanks a lot. Mind if I pick your brain more about this sometime?

Collapse
 
mubashariqbal profile image
Mubashar Iqbal

Absolutely, would love to discuss this more, just let me know when/how :)

Collapse
 
ben profile image
Ben Halpern

Cool, I'll be in touch next week.