Why deep learning?

Deep learning, a form of machine learning, has been the predominant computational force behind advancements in AI.

Deep learning is eating:

1) software (Andrej Karpathy)

2) machine learning (Reza Zadeh)

3) artificial intelligence! (Andrew Ng)

But what is deep learning? And how did it come to be?

Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction.
— Yann LeCunn, Yoshua Bengio, Geoff Hinton

In other words, deep learning is a technique through which computers learn by example inputs, and represent this learning in layers of functions that represent the data and its hierarchical features. The below GIFs show how neural networks, the foundational construct in DL, actually work!

 
nn1
nn3
 

Ironically, deep learning has been a computational technique that has been known for years. It is actually inspired by real neurons in the brain, and knowledge of these techniques has existed since the '80s.

In 2012, a Univ. of Toronto professor named Geoff Hinton led a team to victory in the hugely important ImageNet competition, and sparked a renaissance in the field. He showed that deep learning had the potential, with newfound computer power, to evaluate much more complex inputs than previously thought.

 

The history of deep learning as an impactful field is thus quite short. Already, however, questions abound about the future of deep learning and whether it can continue to lead the field forward.

Max Planck once said “Science progresses one funeral at a time.” The future depends on some graduate student who is deeply suspicious of everything I have said.
— Geoffrey Hinton, University of Toronto

In 2018, an NYU professor named Gary Marcus published a paper titled "Deep Learning: A Critical Appraisal". 

As one might expect, this critical appraisal led to heavy debate about whether deep learning really could enable greater advances in artificial intelligence beyond the local maxima we are at.