- [Instructor] I've described several different ways
of testing software, unit, integration,
and even functional testing
can be automated using a framework.
But how does automation work?
Test automation is separate software
from the application that is being written and tested.
Test automation has multiple purposes.
First, it controls test execution
including order of execution and filtering of tests
based on a criteria.
The goal is to avoid repetitive tasks
that are inherent with manual tests execution.
With that context, a testing framework,
also known as a testing automation framework,
is an execution environment for automated tests.
A testing framework is useful for a number of reasons.
A testing framework performs helpful tasks,
starting with a consistent definition
of how assertions are defined,
which promotes maintainability.
They control and interact with the tested application.
A testing framework executes tests,
or more importantly, it reports on the results.
This includes individual results.
So you know whether or not to test passes
and aggregate results
with the utopian goal of 100% of the tests passing.
The use of a testing framework
provides a developer with several things.
The first are some base assumptions
starting with what responsibilities are delegated
and how?
Practically, the focus of the developers
should be on writing tests.
While it may be tempting,
it's better to develop custom functionality
rather than maintaining supporting software.
Test frameworks provide fundamental concepts
for structural consistency.
The first is around tests
including what a test actually does
and how it's put together.
The other is for the results
and how the successes and failures
are collected for analysis and how they're reported.
Just like code quality,
consistency in testing improves maintainability.
A number of tools are provided with test frameworks.
The first are for the setup of the testing environment,
both around individual tests and groups of tests.
The next are for control of the application,
such as selectively starting and stopping parts.
Some frameworks provide interfaces for test data,
including being able to specify test ranges
or external sources.
Finally, control over execution of tests is provided.
This can include the execution of a subset of tests
or setting thresholds of acceptable failure or duration.
There are many benefits to using a testing framework.
By using a purpose built tool,
there will be a lower cost of maintenance.
If changes are needed,
then developers just need to update the tests,
not the framework itself.
A testing framework can be reused
across multiple applications,
which provides a multiplier effect.
Of course, delegating the responsibility
of maintaining the framework
means developers can focus on writing tests
and great code.
Part of the consistency frameworks provide
is a structure around tests.
There are four distinct phases,
which I'll list and then detail.
They are setup, execution, validation, and cleanup.
Different frameworks have slightly different names
for the phases, but the intent is the same.
Let's take a closer look at each.
The first is setup when the application state
is prepared before testing.
It can be as simple as just including a module.
Setup is also when dependencies are replaced.
This can also be a time when test data is prepared.
The second phase is execution,
where target behavior is performed.
Upon execution, the output is captured
including the response from the API or other sources.
Honestly, this sounds more complicated than it really is.
Practically speaking,
it's just the call to the target method or function.
The third phase is validation,
where the results are compared with the actual
to ensure correctness.
This is when assertions are used to compare the result
with whatever was expected.
The test framework collects the results of validation
and uses them for reporting successes and failures.
The final phase is cleanup,
where the application state is restored
after the tests have been executed.
It's important to cleanup in order to allow other tests
to run without interference.
Of course, cleanup is only needed
if there's something that needs to be restored.
Here's an example of the test phases
using a validator for reservations.
In setup, an invalid reservation is created
such as a bad email address.
In execution, the validator runs
using the invalid reservation as input.
In validation, an assertion would be made
that the reservation would be rejected.
To summarize, a testing framework
facilitates test automation in a number of ways.
They run tests both as a group and selectively
and aggregates and reports the result.
The testing framework provides consistent structure
and tooling that can be used across multiple applications.
Finally, a testing framework
delegates the responsibility of maintenance
so developers can focus their efforts on writing tests,
not creating testing software.
You might be wondering,
what are some examples of common Node.js testing frameworks?
There's Mocha from mochajs.org.
Jasmine from jasmine.github.io.
And Jest from jestjs.io.
Chapter three starts with a comparison survey
of testing frameworks,
then demonstrates how they can be used.
We've been talking about how to automate tests
using a framework,
but how are the actual tests specified?