- You might be wondering why machine learning
took so long to catch on.
After all, it was in 1959 when Arthur Samuel
created his revolutionary checkers program.
At the time it seemed like machine learning
had the wind at it's back.
It was ready to become the dominant form
of artificial intelligence.
Yet what actually happened is that
machine learning took a backseat to other innovations.
Such as symbolic systems.
It wasn't until the late 1980's and early 1990's
when researchers started thinking again
about machine learning.
The rise and fall, and rise again of machine learning
is both sad and interesting.
It shows how just a few researchers were instrumental
in building out early AI.
Back in 1958 a Cornell professor named Frank Rosenblatt
created an early version of an artificial neural network.
Except instead of using nodes and neurons
he used the term Perceptrons for his network.
He argued that if you tied together
enough of these Perceptrons,
you could create a complex form of machine intelligence.
Rosenblatt thought that these Perceptrons
were the most promising path to AI.
He built a machine called the Mark 1 Perceptron.
It tied together thousands of these Perceptrons
into an early neural network.
It had small cameras and was designed to learn
how to tell the difference between two images.
Unfortunately it took thousands of tries
and even then the Mark 1 still had a hard time
distinguishing even basic images.
While Rosenblatt was working on his Mark 1,
an MIT professor named Marvin Minsky
was pushing hard for the symbolic systems approach.
In 1969, Minsky co-authored a book
with Seymour Papert, called "Perceptrons".
In it, he argued decisively against Rosenblatt's approach
to artificial neural networks.
A few years after the book was published,
Rosenblatt died in a boating accident.
Without Rosenblatt to defend Perceptrons,
much of the funding for his approach dried up.
Minsky later dedicated the work to his one time rival,
but it was too late.
Perceptrons and artificial neural networks
languished for nearly a decade.
One of the biggest challenges with Perceptrons
is that Rosenblatt's Mark 1 didn't include a hidden layer.
This is a key part of how artificial neural networks
learn and adapt.
This was partly because the Mark 1 was a hardware device.
So the single layer was hard coded
to classify only a few patterns.
In the mid 1980's, a Carnegie Mellon professor
named Geoff Hinton created a new version
of an artificial neural network,
based on Rosenblatt's Perceptron.
Except his version include several hidden layers.
This allowed his artificial neural network
to work on much more complicated patterns.
Still these early artificial neural networks struggled.
They were slow, and they had to go through the problem
several times before they could learn and improve.
It wasn't until the 1990's that Hinton started working
in a new field called Deep learning.
They called it deep learning
because there are many hidden layers.
These additional layers created a large gap
between the input and the output.
This added capacity gave the artificial neural network
a lot more room to match patterns.
There were also new ways to help this network learn,
newer techniques such as backpropagation
could make sure that all the nodes
spread their knowledge more quickly.
Backpropagation goes back through the network
and helps strengthen many of the neural connections.
That helps these larger networks learn new things.
These deep learning networks also clustered
their neurons to help identify patterns.
Clustering allowed the network to create categories
and then sort the new information
into these categories.
Let's say you wanted to use a deep learning network
to identify pictures of cats.
You could give the network a picture of anything
and it would tell whether there's a cat in that picture.
To train the network,
you'd feed a few million pictures
with cats into the network.
It would on it's own start to think
about the pictures and categories.
That way it could quickly break down
each new picture as it enters the network.
In some ways, clustering is pattern matching for patterns.
It can find certain categories of patterns
that help the network identify the image.
It might notice that most animals have fur
or most landscapes have a blue sky.
Then each time it looked at a new image
the network could discard anything
it didn't recognize as an animal or a landscape.
Then the network could completely focus
on identifying the remaining photographs
that might include a mountain or even a cat.