What is Deep Learning?

About 25-30 years ago, people didn't know what the internet was. Now, we can't imagine a day without it!

However, I’ll be giving you a quick run on what is deep learning.

Neural networks and deep learning was invented back at 60s. But they got recognition in the 80s. People started talking about them, conducted a lot of researches and so on. They thought it will change the whole world but then the hype kind of died.

Why is that? The technology to facilitate neural networks were not on the right margin. For deep learning and neural networking you need two things: enormous amount of data and strong processing power; which was not available back at that time.

Let's have look at three years: 1956, 1980, 2017.

Do you know how did storage look back in 1956?

Well, this is a hard drive, and that's 5 megabytes right there. It's on a forklift, about the size of a small room. It was being moved to another location on a plane. In 1956. A company had to pay 2,500$ of that time's money to rent that hard drive for one month, not buy it, just rent it.

In 1980, the situation had improved a little bit. Still very expensive for only 10 megabytes (which is like one photo these days)

And in 2017, we've got a 256 gigabyte SSD card for $150, which can fit on your finger.

So from 1956 to 1980, storage capacity doubled, and then it increased about 25,600 times by 2017. The time periods aren't very different, but there was a huge leap in technology. This shows that the growth isn't linear; it's exponential.

Now, for the processing: here's a chart on an algorithmic scale. If we plot hard drive costs per gigabyte, it quickly approaches zero. Now, you can get free storage on Dropbox and Google Drive.

Currently, scientists are exploring the use of DNA for storage, although it's quite expensive. It costs $7,000 to synthesize 2MB of data and another $2,000 to read it.

But have you noticed that this situation is quite similar to the early days of hard drives and planes? This is also going to improve rapidly due to the exponential growth curve. Ten or twenty years from now, everyone will be using DNA storage.

This information is from Nature. As you can see, you can store all the world's data in just 1 kilogram of DNA storage. Alternatively, you can store about 1 billion terabytes of data in just 1 gram of DNA storage.

This example shows how fast we're progressing, which is why deep learning is gaining momentum now.

The same growth applies to processing capacity, which is also increasing at an exponential rate. This is known as Moore's Law, and you've probably heard of it.

Right now, computers have surpassed the thinking ability of a rat! They have already exceeded the thinking capacity of a human brain, and by 2040 or 2045, they will surpass the combined thinking power of all humans. So, basically we're entering the era of computers that are incredibly powerful and can process things much faster than we can imagine and this is what exactly facilitating the deep learning.

Now all of this makes us question: What exactly is deep learning? What is neural networking? what is it? What is going on here?

This gentleman, Geoffrey Hinton, is known as the godfather of deep learning. He conducted research on deep learning in the 1980s and has done a lot of work in the field. He has published many research papers on deep learning. Currently, he works at Google, so much of what we will discuss comes from Geoffrey Hinton. He won the 2024 Nobel Prize in Physics

[He has many great YouTube videos where he explains things clearly, so I highly recommend checking them out!]
Image description

The idea behind deep learning is to look at the human brain. It tries to mimic the brain (neuroscience stuffs).We don't know everything about the human brain, but with the little we do know, we try to mimic it.** Why?** Because the human is one of the most powerful learning tool on this planet The way brain learns, adapts skills and apply them - we want our computers to copy that.

Here we have some neurons. These neurons are spread onto glass and observed under a microscope with some coloring. You can see what they look like. They have a body, branches, and tails. You can also see a nucleus in the middle. That's what a neuron looks like. In the human brain, there are approximately a hundred billion individual neurons in total and they are connected with each other.

So to give you a picture of this, this is what it looks like, this is an actual dissection of the human brain.
This is just to show how vast the network of neurons is. There are billions and billions of neurons all connected in your brain. We're not talking about five hundred, a thousand, or even a million. We're talking about billions of neurons who takes care of memorizing, balancing etc. And yes, that's what we're going to try to recreate.

So how do we recreate this in a computer? We create an artificial structure called an artificial neural network.

We have nodes or neurons, which are used for input values. These are the values you know about a certain situation. For example, if you're modeling something and want to make predictions, you'll need some input to begin with. This is called the input layer.

Then you have the output, which is the value you want to predict. This is called the output layer.

And in between, we have a hidden layer. In your brain, information comes through your senses like eyes and ears. It doesn't go straight to the result; it passes through many neurons first. This is why we use hidden layers in modeling the brain before reaching the output. This is the whole concept behind it: we are going to model the brain, so we need these hidden layers before the output.

Neurons are connected to the hidden layer, and the hidden layer is connected to the output value.

And then we connect everything, just like in the human brain. Connect everything, interconnect everything. That's how the input values process through all these hidden layers, just like in the human brain, and then we get an output value.

This is what deep learning is all about at a very abstract level.

I will be uploading two blogs on two deep learning models.

1. Artificial Neural Networks for a Business Problem
2. Convolutional Neural Networks for a Computer Vision task

My blogs are pretty long, so I'll upload them on Hashnode and then share another blog here with links to those two posts. Thanks for reading!