Migrating The TechIsHiring Twitter Bot to Twitter API v2

Introduction

TechIsHiring is a pretty lightweight organization and it doesn’t have much infrastructure. It does have one main thing though that is crucial to its operation, the TechIsHiring Twitter bot. The TechIsHiring Twitter bot retweets any tweet containing the string ‘#TechIsHiring’, which in turn makes the TechIsHiring Twitter account the main source of information on the hashtag. Our newsletter is also indirectly powered by the Twitter bot because it allows me to easily go through and find tweets using the Twitter account to put in the newsletter.

The TechIsHiring Twitter bot is easily the main reason for TechIsHiring going from nothing to over 10,000 followers on Twitter and having over 700 subscribers on our newsletter.

Outline

• Why migrate the Twitter bot?
• Updating the Twitter bot
• Migration and Deployment
• Conclusion

Why migrate the Twitter bot?

The TechIsHiring Twitter bot was initially deployed on Heroku Dynos on their free tier and has been in production for over a year and a half. In that time, not much has really changed from its original inception, really only needing to be updated as a few new problems came up as more eyes saw the hashtag and the Twitter account.

Unfortunately, Heroku has decided to end its free tier for Dynos and this pushed me to consider migrating the Twitter bot. Also with the consideration of migration, I felt updating the bot was also a good idea. I had grown significantly as a developer since its original development and there were some features that I wanted to add to handle some minor administration issues with the Twitter account.

So with that, I set out to update the Twitter bot and move it to a new home.

Updating the Twitter Bot

There are a few main technology changes made to the Twitter bot in it’s update:

• Adding TypeScript
• Moving from twit to twitter-api-v2
• Updating how banning accounts work

The change of updating to TypeScript was one of the major changes the bot went through. At the time I had developed the original Twitter bot, it was written in JavaScript because I grabbed another project (this is the original project I leveraged: https://github.com/mayeedwin/twitterbot) and I honestly didn’t know any TypeScript at the time. While TypeScript didn’t provide too many advantages over JavaScript for this project, it made the migration from twit to twitter-api-v2 significantly easier.

The original project I used leveraged twit to bootstrap leveraging the Twitter API. While twit is a pretty good package, my main concern was that it hadn’t been updated in over 4 years as of the time of this writing and I was having a difficulty trying to figure out how to upgrade the bot’s ability to keep certain accounts from being retweeted using it so I felt the need to migrate away. After a bit of searching, I came across twitter-api-v2 which was the second go-to Twitter API package on npm so I felt this would work out. While it didn’t work exactly how I thought it would, it worked in the way I needed and I really appreciate using this package.

Let’s quickly circle back to a time during the original Twitter’s bot’s deployment. A few months into the bot being brought online, the TechIsHiring hashtag started to gain traction with several notable people starting to use it. The one of the more notable at the time (and whom I am eternally grateful for) is @Tech__UK. @Tech__UK was run at the time by @real_temz and @Sham_jab and they started hosting Twitter Spaces that would last several hours long. They would be so gratuitous to add the hashtag to their Twitter Spaces and several people would tweet the space to share it with their community. At the time, I had set notifications for the TechIsHiring account and for several hours I’d get notifications of people sharing the space, having 20 or 30 notifications by the time their space ended. There were so many people posting to the hashtag that it was trending on Twitter a few times. So you’d think the issue would be the bot tweeting so many notifications and having a problem with that? While there were minor issues there, the main issue and the thing I tried to actively solve for was spam.

With people seeing #TechIsHiring, a few people decided that it would be a good hashtag to start spamming. There were a few spam accounts that would send tweets about winning or entering a particular contest or something and they’d send that tweet several times, almost 20 or 30 times. This was a problem for a few weeks as I’d sit there and have to painstakingly unlike and unretweet each tweet by hand only to have them send even more a few minutes later. I initially tried to use some Twitter features to either block or report them but nothing seemed to help. They’d just keep on spamming the hashtag. I eventually decided that I’d have to update the bot to ignore these spam accounts. What I ended up doing was hardcoding a list of Twitter IDs that I would ignore to keep the account from being spammed. It's funny because in trying to implement that solution, I actually learned that I had to figure out if the tweet was just a regular tweet or a retweet so that I could properly check each ID and see if they were on the ban list. The implementation wasn’t the best but it worked for now. Its main failing is that to update the list, I would have to manually add a new Twitter ID to the list and redeploy the bot.

Now coming back to updating the Twitter bot, I wanted to also update the banning mechanism. After poking around, I realized that Twitter had an endpoint that would return a list of Twitter IDs based on accounts that were muted. Originally, I wanted to block the accounts and get a list of IDs from there but this turns out to be a better solution. They can still view the account but when they tweet, it wouldn’t show up. Twitter-api-v2 also made it easy to implement because they had a function that handled sending the request and getting the data back so there wasn’t much work needed from my end. The only other implementation detail that needed to be made was the actual check itself. Originally, I used the includes function in JavaScript arrays to check the IDs before posting. While the list is fairly small, I felt that was slow and could potentially be a performance issue if the list got much larger (though I don’t expect it to). I decided this time around I would put the array in a JavaScript set and use the has function for JavaScript sets which was a much faster lookup.

And with that I had updated the Twitter bot and had added the functionality of using the mute function of Twitter as a ban list I could leverage from Twitter’s UI.

Migration and Deployment

Now with the Twitter bot actually updated, I now needed to find a new place to run the bot. Heroku Dynos was free and I enjoyed not having to pay or really need to think about the bot for a year and a half so my initial requirements was that it was free. I also wanted to keep deployment and maintenance fairly simple as my infrastructure knowledge wasn’t particularly strong so my search started with ‘Heroku alternatives’. There were a few main options I explored:

• Render
• DigitalOcean
• Fly.io
• Railway
• Staying on Heroku
• AWS

I experimented with some of the above options, even getting as far as deploying on them before running into problems. This is when I learned the difference between ‘Web Service’ and ‘Background Worker’. You see I had deployed the bot as a worker on Heroku but didn’t realize what that meant. So several of the free tiers for the above services allowed you to run static sites or web services for free but they wouldn’t allow you to run a background worker for free which is what I needed to run the bot. I learned this fact trying to deploy to Render, seeing the app run and seeing it fail after a few moments of running. After some quick research, I learned the difference and what my needs were. I resigned myself to needing to pay for a service to run a background worker to run the Twitter bot.

I ultimately decided on DigitalOcean as out of all the options because they were my initial choice before finding out that I couldn’t run a service for free. I figured if I was going to pay for a bot, I’d do it with a more well-known service so it could pump up my resume a bit. I also had some previous experience with them and thought they would be a good fit. After a bit of messing around (the process was surprisingly simple), I deployed the bot and today it’s running on a DigitalOcean Droplet.

Conclusion

And that’s my journey to updating and Migrating the TechIsHiring Twitter bot. If you’re interested in viewing my work, check out the repo here:

https://github.com/TechIsHiring/techishiring-twitter-bot-v2

Feel free to critique my work or suggest changes. While the bot is currently meeting my needs, I wouldn’t mind playing around with things to update the bot though I don’t have any ideas. If you want to suggest ideas, please feel free to do so in this article or in the issues of the repo.

• If you found this article interesting, please feel free to heart this article!
• If you’re interested in learning more about Front-End Engineering, follow me here on Dev.to and Twitter.
• If you’re looking for jobs, I’d highly recommend checking out @TechIsHiring on Twitter or LinkedIn for posted jobs and other resources!