Let's have a quick glance on Deep Learning

Introduction

Machine Learning (ML) is a subcategory of Artificial Intelligence (AI) where perceiving, synthesizing, and inferring information is performed by machines instead of humans. ML includes several techniques as follow:

1. Regression
2. Classification
3. Clustering
4. Dimensionality Reduction
5. Ensemble Methods
6. Transfer Learning
7. Reinforcement Learning
8. Natural Language Processing
9. Word Embeddings
10. Neural Networks and Deep Learning

Neural Networks and Deep Learning are the key technologies behind voice control devices or driverless cars which enable them to detect and understand orders, traffic signs or distinguish pedestrians from electrical posts. Recently, Deep Learning has been receiving lots of attention in different fields of humans’ life. In Deep Learning technique, a computer learns how to classify tasks directly from images, texts, or voices through an iterative process. This iterative process requires a large set of labeled data and a neural network architecture with multiple layers.
The level of accuracy obtained from Deep Learning made it as highly demanded technique in Machine Learning which is critical for safety/security applications like autonomous cars or fraud detection. Advancements in deep learning have improved to a level where it outperforms humans in some tasks, especially object classification in images. Deep learning is based on:
Large amount of data, and
Computing power
Recently, with advancement in both areas (i.e., big data and high-performance GPUs) deep learning is being implemented vastly in AI and machine learning rather than classic techniques. Figure 1. Shows performance of deep learning versus classical techniques in presence of data abundance.

Figure 1. Performance of deep learning in presence of abundance of training data[1].

How Deep Learning Works

Deep Learning is mostly based on neural network concept and that’s the reason it is also called deep neural network. The architecture includes input layer, intermediate layers, and output layer. The input layer corresponds to features that are extracted from a dataset and output layer is determined based on the number of classes that exist in the dataset. However, there are intermediate (hidden layers) between input and output layers. The hidden layers could varies between tens to hundreds of layers depending on the complexity of problem and existing features. Figure 2 represents the deep neural network architecture and shows the interaction between nodes in hidden layers.

Figure 2. Deep learning based on neural network architecture[2].

One the most popular types of deep learning method is Convolutional Neural Network (CNN). In this technique, learned features will be convolved with input data and 2D convolutional layers which makes it suitable for processing 2D data (i.e., images). In addition, manual feature extraction is not required in CNNs and user does not need to identify features used to classify images. CNNs is designed to extract features directly from images and the relevant features are not pretrained. These features will be learned while the network trains on collection of images. The automated feature extraction makes deep learning models highly accurate for computer vision classification. Figure 3 represents the general concept of CNNs in image classification. In this concept, multiple filters are used for each training image at different resolution, and the output of each convolved image serves as the input to the next layer.

Figure 3. Example of a network with many convolutional layers[2].

CNNs learn to detect different features of an image using tens or hundreds of hidden layers. Every hidden layer increases the complexity of the learned image features. For example, the first hidden layer could learn how to detect edges, and the last learns how to detect more complex shapes specifically catered to the shape of the object we are trying to recognize[2].
References
[1]. Zappone, Alessio & Di Renzo, Marco & Debbah, mérouane. (2019). Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?. IEEE Transactions on Communications. PP. 1-1. 10.1109/TCOMM.2019.2924010.
[2]. “What is the deep learning: 3 things you need to know”. What Is Deep Learning? | How It Works, Techniques & Applications - MATLAB & Simulink (mathworks.com)