Further Developing Joyful Tweets

This is some further information extending my original post for my #gftwhackathon entry:

Joyful Tweets

Matt Hamilton ・ Jun 3 '20 ・ 4 min read

#gftwhackathon #node #machinelearning #ibmwatson

In the original project I was using the IBM Watson Tone Analyzer service. This was nice and simple to use, and already existed. However it was not optimal for what I needed. The main issue is that it only accepts a single 'document' at a time. And when I'm trying to pass it a batch of tweets in realtime, it would either take a while or I'd hit the rate limit on the API. Also, you only get a certain number of calls free per month, and I'd hit that pretty quickly.

So I decided to try and create my own tone analyser service. The main requirements:

• Only needed to be able to detect 'joy' and 'anger'
• Needs to be able to handle multiple 'documents' in a single call
• Able to run on IBM Cloud Foundry in 512MB of RAM

Training

I used IBM Watson Studio to do the general analytics and training. You can get a GPU instance, which sped up the training considerably compared to running it locally on my Macbook.

I wrote a Python script to connect to Twitter's websocket API and capture a bunch of tweets. I searched for tweets containing the terms: 'joy', 'anger', 'happy', 'sad' and left if running for a couple of hours. It captured around 800k tweets, which was a suitable sized corpus for training in this instance.

I used Keras to create a neural network model and pre-process and train on the tweets. Full details of the training process are in the following video post on Cinnamon:

https://www.cinnamon.video/watch?v=339756216043439894

Note, at the end of the video I was having trouble as I couldn't get the model to run on the GPU, it kepts saying 'resource unavailable'. As it turns out, I had an existing Jupyter kernel running and that had already allocated the GPU resources.

I was also not getting very good results, as I wasn't letting it train long enough. Even with a GPU it would have taken a lot longer than we had time for in the livestream to wait. I let it run a lot longer after the livestream and got much better results.

Serving

Once I had created and trained the model. I downloaded it from Watson Studio and then tried to write a wrapper about it to serve up a simple API. However I hit a stumbling block. I was trying to get it to run in under 512MB of RAM, which is the maximum on IBM Cloud Foundry's free tier. It wasn't even getting as far as failing to run, it was failing on the pip install tensorflow command. Ie. seems you need more the 512MB of RAM just to even install tensorflow!

I soon discovered Tensorflow lite and was able to convert my Keras model to a Tensorflow Lite model and then just install the tflite-runtime package which is far smaller and just contains what you need to perform predictions using a tensorflow model. The conversion worked pretty well, but the convertor does not support embedding layers, so I had to manually process that part in python before passing to the Tensorflow model.

End result

In the end, I got an API running:

matt@Matts-MBP JoyDetector % cat input.txt 
{
  "texts": [
    "I love the world",
    "I hate the world",
    "I m not happy about riots",
    "I like ice cream"
  ]
}
matt@Matts-MBP JoyDetector % curl -s -H "Content-Type: application/json" --request POST --data @input.txt http://joydetector-smart-kob-ma.eu-gb.mybluemix.net/tones | jq .
{
  "tones": [
    {
      "anger": 0.2507,
      "joy": 0.7493,
      "text": "I love the world"
    },
    {
      "anger": 0.9185,
      "joy": 0.0815,
      "text": "I hate the world"
    },
    {
      "anger": 0.6479,
      "joy": 0.3521,
      "text": "I m not happy about riots"
    },
    {
      "anger": 0.1967,
      "joy": 0.8033,
      "text": "I like ice cream"
    }
  ]
}

This API is what the code running at https://joyfultweets-daring-wildebeest-eq.eu-gb.mybluemix.net/ now uses to analyse tweets from your twitter feed.

Live Stream of the Development

I'm going to be doing a live coding session on Twitch on Tuesday (16th June) at 2pm BST (9am EDT) in which I'll be running through the whole process I used to convert the model to Tensorflow Lite.

See you there!