Skip to content

re: Is Deep Learning a Dead End? VIEW POST


You seem to have some fundamental misunderstanding about DL and how it relates to natural neurons.

Neurons were just the inspiration for ANNs, we still don't really know how they work, we do know it's not like an artificial "neuron"1. The human brain is also humongous. A quote that stuck with me from a uni professor: "there are more neurons in the brain than there are stars in the universe". So ANNs differ from brains both in a qualitative and a quantitative way. I'd also argue that ANNs are discrete whereas NNNs are continuous (up to quanta).

Essentially, neural networks are just a way to fit (complex) formulas to training data using gradient descent. Linear regression could be considered the simplest one, albeit it is lacking an activation function, and it draws a straight line through datapoints. I don't think anybody in the field is expecting strong AI from ANNs at this point in time. Maybe it can be combined into something bigger in the future. A better contemporary comparison would be between ANNs and just the visual cortex, though my partner who works in neuroscience would probably get argumentative at that comparison as well.

Finally, I think you are underestimating the learning effort of natural brains. Not only have they been subjected to millions of years of evolution (something we've tried to replicate by evolving neural networks!), we actually do get a fuckton of data in our infancy. Every "frame" of input can be seen as a training sample. Presuming the brain "works at 90 fps", that's over 200 million samples in the first month of life.

You do raise an interesting point about mirroring behaviour, the use of "memes" (by their original definition), in essence. In ANNs, AFAIK the main way to pass knowledge between networks is with pre-training.

A few minor things:

  • recursion in ANNs does exist! Check out recursive neural networks.
  • machine learning isn't enabled by DL, it's the superset of DP and other techniques, such as binary classification trees / random forest, Monte Carlo simulation, evolutionary algorithms, SVMs, k-means, kNN, etc... ;-)
code of conduct - report abuse