Understanding Deep Learning By Building Custom Neural Network With PyTorch

To understand how deep learning models work, it's crucial to know how to create your own basic neural network. It doesn't have to be complicated; starting with a simple neural network is an essential first step in understanding how they function. In this article, you'll learn about how deep learning models function by creating your very first neural network.

Prerequsites

To follow this article's tutorial, you'll need to:

• Have a grasp of Python programming and understand how to work with Python classes.
• Understand how to work with tensors, as they are the fundamental building blocks of deep learning models.

Deep learning primer

Deep learning and neural networks are often confused with one another; while deep learning and neural networks are closely related, they are not synonymous terms.

A neural network is a computational model inspired by the structure and functioning of the human brain's neural networks. It consists of interconnected nodes (neurons) arranged in layers, where each neuron receives input, processes it, and passes the output to the next layer. Neural networks can be used for various tasks, such as classification, regression, and pattern recognition.

Deep learning is a subfield of machine learning that involves training neural networks with large amounts of data to learn hierarchical representations of the input data. Deep learning algorithms use multiple layers (hence the term "deep") of neural networks to extract features from raw data automatically.

Deep learning models are comprised of layers of interconnected neurons, each serving a specific function:

• The Input Layer: This layer accepts the feature inputs.
• Hidden Layers: These layers process the inputs through multiple neurons and layers.
• The Output Layer: This layer generates the final prediction or classification.

The way these layers are structured defines the type of neural network. which could be such as feedforward neural networks, convolutional neural networks (CNN), or recurrent neural networks (RNN).

Writing your first custom neural network.

Now that you have an understanding of what deep learning and neural networks really are, let's dive into creating a custom neural network with PyTorch.

Creating a custom neural network is building a new module upon the PyTorch neural network torch.nn module. Here, you're essentially creating a new class that inherits from torch.nn.Module, and defining an __init__method to initialise the module's parameters, as well as a forward method that performs the custom neural network computation. With this, we can create our own custom module. Your object-oriented programming (OOP) knowledge in Python is important here.

Before proceeding to write your custom model, it is important to become familiar with the essential PyTorch modules. These modules can be utilised to create any custom neural network you envision.

Lets get started

If you'll be working from your local computer, run the following command to install PyTorch.

pip install torch

Alternatively, if you're using Google Colab, PyTorch is pre-installed, so you can start working without any need for installation.

Here's a step-by-step guide demonstrating the process of creating a custom module in PyTorch and training it on a dataset we've generated.

Step 1: Import libraries

# Import libraries
import torch  # Import the PyTorch library
import torch.nn as nn  # Import the neural network module from PyTorch
import torch.optim as optim  # Import the optimization module from PyTorch

Step 2: Prepare dataset

Let's create a dataset with tensors:

# Prepare custom dataset

# Define tensors A and B with specified values and data type
A = torch.tensor([[10,20],[30,40]], dtype=torch.float32)
B = torch.tensor([[50,60],[70,80]], dtype=torch.float32)

# Concatenate tensors A and B along dimension 0 to create input features
X = torch.cat((A, B), 0)  # features

# Create tensor Y representing the labels
Y = torch.tensor([0,0,1,1])  # label

Step 3: Create the module

Here we will create a new class that inherits from the nn.Module class. The neural network architecture is defined in the __init__ method.

# Define neural network architecture

class custom_neural_network(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        """
        Initialize the custom neural network.

        Args:
            input_size (int): Size of the input features.
            hidden_size (int): Size of the hidden layer.
            output_size (int): Size of the output.
        """
        super(custom_neural_network, self).__init__()
        # Define layers
        self.linear1 = nn.Linear(input_size, hidden_size)  # Input layer to hidden layer
        self.linear2 = nn.Linear(hidden_size, output_size)  # Hidden layer to output layer

    def forward(self, input):
        """
        Forward pass method for the neural network.

        Args:
            input (torch.Tensor): Input tensor to the neural network.

        Returns:
            torch.Tensor: Output tensor from the neural network.
        """
        lin = self.linear1(input)  # Linear transformation from input to hidden layer
        output = nn.functional.relu(lin)  # Applying ReLU activation function for introducing non-linearity
        pred = self.linear2(output)  # Linear transformation from hidden to output layer
        return pred

The network architecture comprises two linear layers (linear1 and linear2), which serve as the input-to-hidden and hidden-to-output layers, respectively.

The forward method is responsible for executing the forward pass of the neural network. It orchestrates the flow of data through the layers, starting with linear transformations.

Crucially, between these linear operations, the ReLU activation function is applied. This introduces non-linearity to the network, enabling it to learn complex relationships within the data.
It's important to note that without an activation function like ReLU, the network would behave akin to a linear regression model, severely limiting its ability to capture and represent intricate patterns in the data.

Step 4: Choosing Loss Function and Optimizer

The loss function assesses how accurately the model's predictions align with the actual target values. It's a mathematical function that takes into account both the model's predictions and the true labels, yielding a single scalar value representing the degree of error. The primary aim is to determine the model parameters that minimise this error. Common loss functions in PyTorch include:

• Mean Squared Error (MSE): Typically used in regression tasks.
• Cross-Entropy Loss: Commonly employed in classification tasks.

To minimise the loss, various optimisation techniques are employed to adjust the model's weights. These techniques include:

• Gradient Descent: Modifies the weights in a manner that decreases the loss.
• Stochastic Gradient Descent (SGD): A variant that calculates the gradient using either a single example or a small batch of examples.
• Adam: A popular optimisation method that combines the advantages of other techniques.

# Define the loss criterion using Cross Entropy Loss
criterion = nn.CrossEntropyLoss()

# Define the optimizer using the Adam optimizer
# It optimizes the parameters of the model using the gradients computed during backpropagation
# lr=0.001 sets the learning rate to 0.001

# Define the input, hidden, and output sizes for the neural network
input_size = 2
hidden_size = 4
output_size = 2

# Instantiate a model with the specified input, hidden, and output sizes
model = custom_neural_network(input_size, hidden_size, output_size)

# Set up the optimizer to update the parameters of the model during training
optimizer = optim.Adam(model.parameters(), lr=0.001)

Step 5: Training the model

Training the neural network to learn from the data involves multiple loops of iterations. Here's how to train the model:

epochs = 500

# Loop through each epoch for training
for epoch in range(epochs):
    # Zero the gradients to prevent accumulation from previous iterations
    optimizer.zero_grad()

    # Forward pass: compute predicted outputs by passing inputs to the model
    outputs = model(X)

    # Compute the loss using the specified criterion
    loss = criterion(outputs, Y)

    # Backward pass: compute gradient of the loss with respect to model parameters
    loss.backward()

    # Update the weights of the model
    optimizer.step()

    # Print loss every 50 epochs
    if (epoch + 1) % 50 == 0:
        print(f'Epoch {epoch+1}/{epochs}, Loss: {loss.item()}')

Step 6: Evaluating the model

In addition to this iterative training process, you can assess the model's performance on fresh data or examine its outputs for specific inputs.

# Use torch.no_grad() to disable gradient calculation during inference
with torch.no_grad():
    # Pass input data X through the model to obtain predictions
    outputs = model(X)
    # Extract the predicted class indices by taking the maximum value along the second dimension
    _, predicted = torch.max(outputs, 1)
    # Print the predicted class indices
    print('Predicted:', predicted)

torch.no_grad() is used to disable gradient calculation since we are performing inference, not training. The input data X is passed through the model to obtain predictions stored in outputs. The predicted class indices are extracted by taking the maximum value along the second dimension of the outputs tensor. Finally, the predicted class indices are printed.

Puting it all together

Building a custom neural network with PyTorch is a powerful and flexible process that allows you to tailor your model to the specific needs of your task. In addition to defining the model architecture, activation functions, and implementing the forward pass, you have the freedom to customise various aspects of the network, such as regularisation techniques, initialization methods, and advanced layer configurations.

After instantiating the model, selecting an optimizer, and specifying a loss function, you embark on the exciting journey of training the model on your dataset. Throughout this process, you have the opportunity to monitor the training progress, evaluate the model's performance, and fine-tune hyperparameters to achieve the best results.

Thank you for your dedication in following the tutorial to its completion. Your curiosity and commitment are key ingredients in mastering the art of deep learning. Should you encounter any challenges or have ideas for improvement, please don't hesitate to reach out. Your feedback is invaluable in our collective pursuit of knowledge and innovation.

Full code