- [Instructor] The big data revolution has happened
because data has grown, but it hasn't happened
in a gradual process like a little sprout coming up out
of the ground.
I mean, that would be an example
of what's called linear growth.
For each step forward in time,
you move the same step upward
to get this nice, straight line.
That's easy to deal with.
Rather, data growth has been exponential.
Here I take the exact same line but I put it way down here
because here's an exponential curve.
This is actually two to the power of X,
and you don't even get to see how far it goes over to 10
because, at that point, it's got a value of over 1,000.
1,024 to be exact.
That's a whole lot more than the 10 we had
with linear growth.
This is huge, but you know what?
The data revolution is growing even faster than this.
We can shrink that one down and we can draw a curve
that shows the X to the X.
So, we take the number on the bottom,
the four, the five, the 10, and raise that to its own power.
We do that, we get a curve like this.
You don't even get to see how far it goes
because by the time it gets to 10, it's at 10 billion.
This is the kind of growth that we're dealing
with in the data world, and my personal impression is that,
at this exact moment, we're right here
at this inflection point where things are taking off.
And so, when we're talking about data growth,
it's less of a carefully nurtured sprout
popping up through the ground
and more of a barely-controlled explosion.
Really, it's an exciting time,
and there are a couple of major contributors
to this explosive growth.
The first major candidate
for this explosive growth is social media,
and the second one is the Internet of Things.
Now, there are many others,
but these really play a starring role
in the extraordinary growth of the big data world.
Let's talk about how this works.
First off, there is the feedback.
We have the uroboros, the snake that's feeding on itself,
and this is like social media
because social media begets more social media.
I mean, think about it.
A person puts post online, that post gets liked.
Each like is an additional piece of data
with a fair amount of metadata that goes with it.
Somebody puts an image online.
That image gets tagged and it gets shared
with anybody who's in it, and it gets put into forms
that use the same tags, or somebody has a follower,
they put something online and those followers share
that content and multiply it, the reach of the post.
Also, of course, the fact that more
and more people are online
and more and more people have social media profiles,
and so there's this incredible growth.
It's not just that it's doubling,
it's the X to the X kind of growth.
It just goes absolutely through the roof with social media.
Now, another one is the Internet of Things, that's IoT,
and the classical examples of Internet of Things
include things like smart homes.
So, I've got a smart thermostat.
It knows when we're home, it talks to our phones.
It can tell what's going on.
And maybe you've got lights and a security system
and maybe you got a smart lock,
and maybe you've got all sorts of other systems
that are connected with each other.
They're communicating constantly.
And outside of the home, you can have a smart grid,
where the city knows about the traffic levels,
it knows about how much electricity is going
between the generators and the various buildings and houses.
It knows what's going on with the water system.
There is so much information being exchanged here
through sensors and networks that it, again,
has the explosive growth.
Another one that falls into this category
is self-driving cars, which gather an extraordinary amount
of data from the sensors they have all around them
and they communicate to each other,
and the time will soon come that they communicate directly
with the road and with the traffic signals.
There are so many different ways to gather all this data,
and it gets even more complicated
because the quality of the data has changed over time.
Now, by that I don't mean good data versus bad data,
I mean that we start with text.
You know, text comes first.
When you send text messages,
you started with your little flip phone
and you send your little, tiny text message,
which actually was text.
Text can be measured in kilobytes.
But soon, for instance, you learn how to do audio
and you can send a voice message
or a sensor for your home can give you the audio
of what's happening.
That's usually measured in megabytes.
And then, eventually, you get to video.
Say, for instance, you have a doorbell.
It's now giving you HD video over your phone
of who's there at the door.
Video can be measuring gigabytes.
And so, we're not only increasing the number of things
that provide the data, but we're going up the ladder
to kinds, or qualities, of data that are much, much larger
and also might move from the structured
to the large and unstructured format, again,
from text to audio to video,
and what those correspond to are the three V's of big data:
volume and velocity and variety.
And so, the social media revolution,
which is still going on very actively,
and the Internet of Things,
which, again, is still just at the beginning,
have contributed massively to the explosive growth of data
that constitutes both the challenge
and really makes the fertile ground
for the promise of big data.