Machine Learning in Deno: Single Layer Perceptrons

Ever wished you could solve your regression and classification problems in JavaScript/TypeScript without having to rely on Neural Network libraries like Tensorflow.js?

Presenting La Classy, a single-layer perceptron library for Deno.

Classy provides a bunch of solvers (gradient descent, OLS, etc.), loss functions, activation functions, learning rate schedulers, and optimizers for gradient descent such as Adam and RMSProp. You can try various combinations of these depending on the problem.

Here's a quick example of how to classify the popular Iris dataset using Classy in Deno:

Import required libraries

First, we import the following libraries. You'll see use for them in later sections:

deno.land

import { parse } from "https://deno.land/std@0.217.0/csv/mod.ts";
import {
  ClassificationReport,
  Matrix,
  useSplit,
  CategoricalEncoder,
} from "https://deno.land/x/vectorizer@v0.5.0";
import {
  SagSolver,
  softmaxActivation,
  rmsPropOptimizer,
  crossEntropy,
} from "https://deno.land/x/classylala@v1.1.0/mod.ts";

You can also load them from JSR.

JSR

import { parse } from "jsr:@std/csv";
import {
  ClassificationReport,
  Matrix,
  useSplit,
  CategoricalEncoder,
} from "jsr:@lala/appraisal@0.5.0";
import {
  SagSolver,
  softmaxActivation,
  rmsPropOptimizer,
  crossEntropy,
} from "jsr:@lala/classy@1.1.0";

Load the Dataset

First we load our dataset. You can download the iris.csv file from here.

const data = parse(Deno.readTextFileSync("iris.csv"));

We then skip the first row of our dataset (the header).

data.shift();

Prepare input variables

Now that we have our data ready, we can separate the predictor and target variables from our dataset.

The first four columns of our dataset are the predictor variables and the fifth column indicates the iris species, our target variable.

const x = data.map((fl, i) => fl.slice(0, 4).map(Number));

Classy requires inputs to follow the following type:

type MaybeMatrix = {
    data: Float64Array,
    shape: [number, number]
}

However, our data is currently in the form of a 2D array. We can use the Matrix class from @lala/appraisal since it satisfies the type.

const X = new Matrix(x, {shape: [data.length], dType: "f64"})

In the meantime, let's also prepare the classes. We take just the fourth column from our dataset.

const y_pre = data.map((fl) => fl[4]);

Our y_pre will now be a 1D array with strings that denote the classes of each sample. We need to perform one-hot encoding on this data to use it for machine learning.

For one-hot encoding, we can use CategoricalEncoder from @lala/appraisal.

const encoder = new CategoricalEncoder()
encoder.fit(y_pre)

const y = encoder.transform(y_pre, "f64")

Now that we have both the predictors and the targets, let's take a small portion of the dataset out for testing accuracy.

const [[x_train, y_train], [x_test, y_test]] = useSplit(
  { ratio: [7, 3], shuffle: true },
  X,
  y
);
x_train.slice(0, 10)

We can now begin training our model.

Initialize the Model

In this example, we are using Stochastic Average Gradients with RMSProp as our optimizer.

const solver = new SagSolver({
  loss: crossEntropy(),
  activation: softmaxActivation(),
  optimizer: rmsPropOptimizer(4, 3),
});

• crossEntropy is the standard loss function for multiclass problems.

• softmax is an activation function used for multiclass problems.

Try using hinge as the loss function with linearActivation later.

With our solver ready, we can begin training the model.

Train the model

We can train our model using the train() method.

solver.train(x_train, y_train, {
  learning_rate: 0.01,
  epochs: 300,
  n_batches: 20,
  patience: 10
});

Let's break down each argument:

• The first two arguments are the training input and output variables.

• The learning_rate hyperparameter determines how large each step should be when updating parameters during training. If you are not using adamOptimizer or rmsPropOptimizer, set this to a very small value.

• The epochs determine how many iterations the model should train for. This is the maximum number of iterations even if it doesn't converge.

• n_batches is the number of minibatches. A large number will result in longer epochs.

• patience determines how many epochs the model should continue learning if the loss starts increasing. If there is no improvement in loss within 10 epochs, the model will roll back 10 epochs and stop training.

Testing Metrics

Now that our model is trained, we can test its accuracy on the data we kept aside for testing.

const res = solver.predict(x_test)

We now have a Matrix with joint probabilities of classes for each sample. We can use the CategoricalEncoder again to convert them into one-hot representations.

// this method mutates the variable
CategoricalEncoder.fromSoftmax(res)

We can now convert the categorical variables into class labels.

// Predicted Values
const y_pred = encoder.untransform(res)
// Actual Values
const y_act = encoder.untransform(y_test)

The ClassificationReport class accepts predicted and actual values as parameters to create a classification report and provide metrics.

const report = new ClassificationReport(y_act, y_pred)

console.log(report)

At the time of writing this article, I gave the code a run and this was my classification report.

Class   Precision   F1Score Recall  Support
Iris-setosa 1   1   1   15
Iris-versicolor 1   1   1   13
Iris-virginica  0.85    0.92    1   17
Accuracy            0.98    45

You can try different combinations of loss functions and activation functions, try different hyperparameters, and try to solve different supervised learning problems using Classy.

Check out more examples at deno-ml!

I will make another article with a regression problem later. Have a great day and good luck on making your machine learn!