Train a Deep Neural Network to recognize handwritten digits with Dannjs.

A famous task performed by Deep models is handwritten digit recognition.
Here is how you can create your model & train it with some images of digits.

Requirements

• Nodejs
• Some knowledge about neural networks

Using

• easy-mnist
• Dannjs
• fs (optional)

MNIST

We're going to use a dataset called MNIST. You can read more about it here.

MNIST is a dataset containing a total of 70 000 28x28 images of handwritten digits.

It contains an image and a label in order to identify the digit.

MNIST to a Deep Model

To show these images to our Deep model, we're going to need to feed every pixel with values ranging in-between 0 and 1. A 0 would represent a black pixel and a 1 would represent a white pixel.

The image below demonstrates this very well.

We could see a 28 by 28 image as an array of 784 values. This is how we are going to feed the images to our neural network.

When feeding our model with image data, we also need to give the desired output in order to train it. In the image above, the neuron labeled 6 is circled to demonstrate this.

Now that we understand how we are going to train our MNIST model, let's get started!

Project setup

Start by initiating an npm project with

npm init -y

In order to access the dataset, install the MNIST npm package

npm i easy-mnist

We are also going to install our Neural Network library, Dannjs.

npm i dannjs

Import the dataset

In our index.js file, were going to require the dataset. We are creating a training set of 60 000 images and a testing set of 10 000 images.

const dataset = require('easy-mnist').makeData(60000,10000);

This is going to create two sets, the training set & the testing set.

Luckily, our dataset of images already comes in 784-length array format and the labels come as a 10-length array.
We can access those like so

// For the training set
dataset.traindata[index].image
dataset.traindata[index].label

// For the testing set
dataset.testdata[index].image
dataset.testdata[index].label

Creating our Model

We first need to create a model, for this, we are going to import dannjs

const Dann = require('dannjs').dann;

We can then create a Dann model with 784 inputs and 10 outputs. We're also going to add two hidden neuron layers with leakyReLU activations and then set the learning rate to 0.0001.

const nn = new Dann(784, 10);
nn.addHiddenLayer(128 ,'leakyReLU');
nn.addHiddenLayer(64 ,'leakyReLU');
nn.makeWeights();
nn.lr = 0.0001;

We can then log the model to confirm the model creation was successful

nn.log();

Training the model

For our model to be trained, we need to iterate through the whole training set. Having completed the entirety of the training set is what we call an epoch. In order to successfully train a model, we need multiple epochs.

This is what 1 epoch looks like

for (let i = 0; i < dataset.traindata.length; i++) {
    nn.train(dataset.traindata[i].image, dataset.traindata[i].label);
}

We can then loop multiple epochs

let epochs = 10;
for (let e = 0; e < epochs; e++) {
    for (let i = 0; i < dataset.traindata.length; i++) {
        nn.train(dataset.traindata[i].image, dataset.traindata[i].label);
    }
    console.log("Completed epoch " + e + " with a loss of " + nn.loss);
}

With 10 epochs, depending on the model you created & your CPU's performance, it might take a few minutes. 10 epochs might not even be enough to train a very accurate model.

Around 100 epochs, your model might reach an acceptable level of accuracy.

The number of epochs & training time all depend on a lot of factors about the model. Experiment with the creation of a neural network, try adding one more layer, changing activation functions, or changing the learning rate, and see what happens.

Since it might take a while to train, it is a good idea to save a model in order to keep training progress.
Here is how you can do so using fs, (do not forget to install it with npm i fs).

Import fs

const fs = require('fs');

Saving our model to myModel.json

let json = JSON.stringify(nn.toJSON());
fs.writeFileSync('myModel.json', json, 'utf-8');

Loading back our model from myModel.json

let data = fs.readFileSync('myModel.json', 'utf-8');
let model = JSON.parse(data);
const nn = Dann.createFromJSON(model);

So there you have it! How to train an MNIST model with Dannjs.

We are hosting a MNIST model contest, the most accurate model wins!
You can submit your model here & try it out live.
Results are posted in our discord server here is the invite link