How to Build a Deep learning model with Keras and ResNet-50

Introduction

Artificial intelligence and machine learning have revolutionized the field of computing. It has led to the development of data-driven models. A data-driven model learns from past/historical data and then recognizes patterns in the data. The model gains knowledge that it uses to predict future outcomes. A machine learning model can learn from data quickly and improve its prediction accuracy over time.

Many scientists strive to build more sophisticated models that give the most accurate future outcomes. The need for these models has led to the introduction of deep learning models.

Deep learning models use the artificial neural network in building sophisticated models. The deep learning models can solve complex tasks such as driverless cars, computer vision, natural language processing, and image classification.

Deep learning models are built using different types of artificial neural networks such as Recurrent Neural Networks, Convolutional Neural Networks, Deep Neural Networks, Long Short Term Memory Networks, Generative Adversarial Networks and Deep Belief Networks which are applied to perform specific tasks.

In this tutorial, you will learn how to build the deep learning model with ResNet-50 Convolutional Neural Network. ResNet-50 is a pre-trained Convolutional Neural Network for image classification and computer vision. ResNet-50 is a residual network with 50 layers stacked on top of each other to form the final neural network. The 50 layers comprise 48 Convolution layers, 1 Average Pool layer, and 1 Max Pooling layer.

ResNet-50 is trained on a large image dataset from the ImageNet database. You can easily import the pre-trained ResNet-50 from Keras and apply it to build a custom image classification model. In this tutorial, you will import the ResNet-50 convolutional neural network from Keras. You will then apply it to build a flower image classification model. Using the pre-trained neural network is faster. It saves time building the entire Convolutional Neural Network from scratch.

Prerequisites

To get started with this tutorial, ensure you are familiar with the following:

• Ensure you know Python programming.
• Ensure you use Google Colab notebook to run the Python code. Google Colab is powerful since it uses Google's cloud GPUs to run the deep learning model. Once you have the Google Colab notebook, let's start setting up our project.

Project Setup

To build the deep learning model, we will require TensorFlow and Keras. We will run Tensorflow in the backend when building the flower classification model. We will then import Keras from TensorFlow and use it to initialize our the deep learning model. We will also import the ResNet-50 network from Keras API. We import these libraries as follows:

import tensorflow as tf
from tensorflow import keras

Let's also import the Keras layers from Keras. You will add custom input and output Keras layers to fine-tune the ResNet-50 network:

from tensorflow.keras import layers

Downloading the dataset

Before building the deep learning model, you will download the dataset and save it to Google Colab. To download the dataset, apply the following code in Google Code:

flowers_url = "https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz"

The code will download the dataset in a compressed format. You then have to extract the dataset and save it into a data directory in Google Colab:

import pathlib
flowers_data = tf.keras.utils.get_file('flower_photos', origin=flowers_url, untar=True)
flowers_data = pathlib.Path(flowers_data)

The code will save the uncompressed data in Google Colab in a directory named flowers_data. Let's print flowers_data to see the data in Google Colab:

print(flowers_data)

The code above outputs the following:

/root/.keras/datasets/flower_photos

Displaying the images

The loaded dataset consists of five categories of flower images: sunflower, tulips, dandelion, roses, and daisy. To display some of the sunflower images, use the code below:

all_sunflowers = list(flowers_data.glob('sunflowers/*'))

To print and display one of the roses images in Google Colab, use this code:

import PIL
print(all_sunflowers[1])
PIL.Image.open(str(all_sunflowers[1]))

The code displays the following image:

The next step is to pre-process our image dataset into a format that the deep learning model can easily understand and learn.

Pre-processing the image dataset

To start, you will set the image height and width as follows:

height,width=180,180

From the code above, we have the image dimensions as 180 by 180 pixels. The next step is to set the batch size that the deep learning model will use during each training cycle:

training_batch_size=32

The next step is to split the downloaded images dataset into training and validation sets. To get the training set, run the following code in Google Colab:

train_set = tf.keras.preprocessing.image_dataset_from_directory(
  flowers_data,
 validation_split=0.2,
 subset="training",
 seed=123,
 image_size=(height,width),
 batch_size=training_batch_size)

From the code snippet above, we use validation_split=0.2 to split the original dataset. It means that 80% of the original image dataset will train the deep learning model while 20% will be for model validation.

To see all the flower categories found in the training set, use the following code:

image_cat = train_set.class_names
print(image_cat)

The code above provides the following output:

['daisy', 'dandelion', 'roses', 'sunflowers', 'tulips']

To get the validation set, run the following code in Google Colab:

validation_set = tf.keras.preprocessing.image_dataset_from_directory(
  flowers_data,
 validation_split=0.2,
 subset="validation",
 seed=123,
 image_size=(height, width),
 batch_size=training_batch_size)

After fininishing the dataset pre-processing, the next step is building the deep learning model.

Building the deep learning model

To start, you will initialize the TensorFlow Sequential model. It will allow us to add a custom input and output Keras layer on the pre-trained ResNet-50 network. The custom layers will fine-tune the ResNet-50 network to perform image classification.

dnn_model = Sequential()

After initializing the sequential model, the next step is to import the pre-trained ResNet-50 network. You will import this pre-trained network using the following code snippet:

imported_model= tf.keras.applications.ResNet50(include_top=False,
 input_shape=(180,180,3),
 pooling='avg',classes=5,
 weights='imagenet')
for layer in imported_model.layers:
        layer.trainable=False

The code snippet will load the pre-trained ResNet-50 network. We have also set the following parameters:

• include_top=False: It ensures we can add a custom input and output Keras layer on the pre-trained ResNet-50 network
• pooling='avg: It will use the average pooling technique when building the deep learning model.
• classes=5: It shows the images dataset has five flower categories.
• weights='imagenet: The ResNet-50 network will use the imagenet weights for classification.
• layer.trainable=False: It ensures that the ResNet-50 parameters and weights are not trained again to speed up the training process.

Fine-tuning the imported pre-trained ResNet-50 network

You will add a custom input and output Keras layer to fine-tune the imported pre-trained ResNet-50 network. The custom input and output Keras layer will enable our final deep learning model to understand our data and produce the desired results. Let's first import the Keras layers as follows:

from tensorflow.python.keras.layers import Dense, Flatten

We have imported the Dense and Flatten layers that will fine-tune and customize the ResNet-50 network. You will add the custom layers using the following code snippet:

dnn_model.add(imported_model)
dnn_model.add(Flatten())
dnn_model.add(Dense(512, activation='relu'))
dnn_model.add(Dense(5, activation='softmax'))

The code will add the imported_model as the first layer. It will also add the Flatten layer and a Dense layer with 512 neurons. It finally adds an output Dense layer with five neurons. The output layer has five neurons because the image dataset has five flower categories. This layer also uses the softmax activation function because the image dataset has more than two flower categories.

Getting the model summary

To get the model summary, use this code:

dnn_model.summary()

The code gives the following output:

The output shows we are building Tensflow's Sequential model. It also shows all the trainable parameters and all the non-trainable parameters imported from ResNet-50.

Compiling the deep learning model

In compiling the deep learning model, you will set the model optimizer, loss, and metrics. For this model:

1. You will import and use Adam as the model optimizer. It will improve the performance and accuracy of the deep learning model in the training phase. It also handles and debugs the errors the deep learning model may have in the training phase.

2. The model loss parameter calculates all the errors the deep learning models encounter during the training phase. You will use sparse_categorical_crossentropy as the model loss parameter value.

3. The model metrics parameter evaluates the performance of the deep learning model. It gets the accuracy score of the model after each training cycle (epoch). You will use accuracy as the model metrics parameter value.

To compile the deep learning model, use this code:

from tensorflow.keras.optimizers import Adam
dnn_model.compile(optimizer=Adam(lr=0.001),loss='sparse_categorical_crossentropy',metrics=['accuracy'])

The final step is to fit the compiled deep learning model into the training and validation set. The dataset will train and evaluate the model performance. Over time, the deep learning model will learn from the dataset and gain knowledge which it will then use to make predictions.

Fitting the deep learning model

To fit the compiled deep learning model into the training and validation set, use this code:

history = dnn_model.fit(
  train_set,
 validation_data=validation_set,
 epochs=10
)

The model will learn from the images dataset over ten training cycles (epochs) and produce the following output:

From the output above, the accuracy score of the deep learning model increases over time as the number of epochs increases. After ten epochs, the deep learning model is ready, and you can use it to make predictions.

Making predictions using the trained deep neural network

You will use the final deep learning model to predict a flower image. It will classify a flower image into one of the five flower categories.

You first import the OpenCV library. It will pre-process the selected flower image to ensure it has the same dimensions as the images that trained the deep learning model.

import cv2
image=cv2.imread(str(all_sunflowers[1]))
image_resized= cv2.resize(image, (height, width))
image=np.expand_dims(image_resized,axis=0)
print(image.shape)

After the pre-processing, the selected sunflower image will have a dimension of 180 by 180 pixels:

(1, 180, 180, 3)

You can now apply the following code for the model to predict this image:

model_pred=dnn_model.predict(sample_image)

To print the predicted category, apply this code:

predicted_class=image_cat[np.argmax(model_pred)]
print("The predicted category is", predicted_class)

The code snippet displays the following output:

The predicted category is sunflowers

From this output, the deep learning model has classified the category of the flower image as sunflowers. It is an accurate prediction and shows you have successfully implemented the deep learning model.

Conclusion

Following this tutorial, you have learned how to build a deep learning model using Keras and ResNet-50. You leveraged the power of the pre-trained ResNet-50 network and fine-tuned it to solve the image classification task. You have also learned how deep learning models build sophisticated models and applications.

This tutorial teaches how to build a simple image classification model. You can try to run the code in Google Colab on your own and get the same model. You can also access all the Python code for this image classification model here.

To gain more exposure to deep learning, start with this tutorial and then improve your knowledge. With practice, you will be building more sophisticated deep learning models that can solve complex tasks. Happy learning!

If you like this tutorial, let's connect on Twitter and LinkedIn. Thanks for Reading and Happy Learning!