- [Instructor] One of my great accomplishments
is that a few years ago my family and I
traveled around the world using just carry-on bags.
Now, we had to be very, very selective
about what we took with us,
and occasionally we had to buy stuff when we were there.
But a few weeks ago, we went on a short trip,
and our car ended up looking more like this.
It's not that we don't know how to pack anymore,
it's just that our trip required a lot more things.
That's the same thing about data.
Big data, of course, is big.
Often because of the extraordinary volume of the data,
you can't fit it all in a single file,
or a single drive, or a single computer.
Nor can you process it that way.
Instead, you frequently have to turn
to distributed computing and processing.
Now, several years ago,
Apache Hadoop was the first major solution
to distributed storage,
and it really ushered in the big data age.
And, for that reason,
Hadoop became synonymous with big data.
Major conferences were named the Hadoop Conference
and people used it as a way of symbolizing big data.
Now what it did is it allowed you to spread the same dataset
across many, many different computers.
It could take care of the redundancy,
it could take care of the fault problems,
it could do an amazing number of things.
But, it was built for local storage,
and it wasn't built for processing.
And so Hadoop, while it initiated the big data age
through the distributed storage
and certain kinds of distributed processing,
it really was a foundation and not the ultimate solution.
So, for instance,
right now we're in the age of cloud services.
This is where you have cloud-based
or online Internet-based storage and processing.
So, for instance, you put your data in AWS,
that's Amazon Web Services,
or Google Cloud, or Microsoft Azure,
any of a number of providers.
They will both store the data
and give you a mechanism to analyze the data.
And, if you create an app that runs on big data,
you can have what's called a multi-platform container.
So something like Docker, which is a Linux container engine,
that allows you to create the app that,
like the containers on a container ship,
you load 'em in what you want there, it's super flexible,
load it on whatever boat, get it out of there, it moves.
Containers allow you to create these big data applications
and run them on basically any kind of architecture.
It's one of the amazing developments in the progression
of distributed development, distributed processing,
and finding a way of getting the insights from big data
out to your customers.
So we've progressed from Hadoop, to cloud, to containers,
and who knows what's going to come next?
Which, of course, is why I want to take the next video
to talk about ways of preparing for an uncertain
and unknown future in the big data world.