- [Instructor] If you've been in the tech world
or even just watching for more than, say, six months,
then one thing is clear.
It's a rollercoaster.
It's got constant ups and downs and twists and turns.
It's never the same,
and as a result it's also always exciting.
I've mentioned elsewhere how the changes in the data world,
as anywhere, build on one another like climbing stairs.
Data science led to big data,
which led to machine learning,
which led to artificial intelligence as we now know it,
and so on up and up, step by step.
And one implication of that is
that anything I could tell you
about specific technologies or practices
like Docker containers or Kubernetes
will almost inevitably be obsolete within just a few years.
But this is not a cry of hopelessness.
Rather, it's a call to action.
The idea here is that there will always,
always be new technology.
There will be new hardware and new software
that can be used for collecting data, processing data,
storing and analyzing data.
And it becomes really a critical test
to not just know what those new, better,
wonderful, shinier things are,
but to integrate those innovations
with your current systems.
And that's where some of the current developments
like multicloud, where you use multiple cloud providers
simultaneously for storing data and for processing data,
and container systems, which make it so easy
to develop an app on your big data system
and run it almost anywhere, those allow you to integrate,
and they allow you to include these new technologies,
as well as some of your legacy systems
or the things that have proven to be very effective.
Similarly, there will always be new methods.
For instance, there have been extraordinary developments,
really groundbreaking work
with deep learning neural networks.
They've accomplished things
that I personally never would have thought were possible.
And so have other new developments
in the data science world in general.
But remember, new methods will always come.
As amazing as deep learning is,
it's not going to be the answer always for everything,
at least not in its current variation.
And don't forget that old methods,
things as simple as spreadsheets
and simple as Bash programming
in the terminal still have value
and still allow you to get insight out of your data,
even when you have all of these amazing new technologies.
The idea is this.
You need to be flexible.
You need to remember that you don't have this one system
that's always going to work for everything,
but you need to be willing to try new things out,
and you need to be willing to try a different approach.
Then you need to be able to integrate.
Don't throw out everything and start over again
because something looks newer and better.
There's always a cost with making changes.
And doing things gradually
and testing as you go is critical,
but integrating, which has become easier to do
with some of the new developments
where you include these new technologies,
these new methods along with the ones
that you already have established
and the ones you know how to work,
that's going to be key to continual success
as opposed to the risk of crashing and burning completely.
But no matter what you do, remember to focus on your goals.
We don't do data science,
we don't work with big data just for the sake of it.
It's there to help us get useful insights
to actually know what to do to solve a particular problem.
And the new hardware, the new software,
the new methods are nothing more than tools
to help you reach those goals.
So focus on your goals
and let those be the guiding factor in your decisions
about whether to include new technologies,
how to integrate them into your workflow,
and how you can get value out of them.
Now, one really important thing
in case it's not completely obvious is that
because of the changing environment,
there is a critical need
for continual training and support,
not just for the tech people,
but for everybody in your organization to better understand
how they can use data and big data
to inform their own decisions
and how they can get the value out of it.
And, for instance, managers need to know
what kinds of decisions can be answered,
what kind of support they need to give to the people
who will be doing the technical work on a daily basis,
and how salespeople can integrate that
into the work that they're doing.
The training and support is necessary
any time you have a competitive
and rapidly evolving environment,
which is absolutely the case with big data.
Remember, it's the constant building up,
step by step,
that's going to help both you and your organization
not just to stay in step with everybody
as you go from big data to machine learning to AI,
but really to get to that great next step,
to get that breakthrough, to get that insight,
and possibly to get that competitive advantage
that motivated you to embrace big data in the first place.