Purpose
and importance of test plans in software testing
Test plan is the project plan for
the testing work to
be done. It is not a test design specification, a collection of test cases or a set of test procedures; in fact, most
of our test plans do not address that level of detail. Many people have different
definitions for test plans.
Why it is required to write test plans? We have three
main reasons to write the test plans:
First, by writing a test plan it guides our thinking. Writing a test plan
forces us to confront the challenges that await us
And focus our thinking on important topics.
By using a template for writing
test plans helps us remember the important challenges. You can use the IEEE 829
test plan template shown in this chapter, use someone else’s template, or
create your own template over time.
Second, the test planning process and the plan itself serve as
the means of communication with other members of the project team, testers, peers, managers and other
stakeholders. This communication allows the test plan to influence the project
team and the project team to influence the test plan, especially in the areas
of organization-wide testing policies and motivations; test scope, objectives
and critical areas to test; project and product risks, resource considerations
and constraints; and the testability of the item under test. We can complete
this communication by circulating one or two test plan drafts and through
review meetings. Such a draft will include many notes such as the following:
Third, the test plan helps us to manage change. During early
phases of the project, as we gather more information, we revise our plans At
times it is better to write multiple test plans in some situations. For
example, when we manage both integration and system test
levels, those
two test execution periods occur at different points in time and have different
objectives. For some systems projects, a hardware test plan and a software test
plan will address different techniques and tools as well as different
audiences. However, there are chances that these test plans can get overlapped,
hence, a master test plan should be made that addresses the common elements of
both the test plans can reduce the amount of redundant documentation.
IEEE 829 STANDARD TEST PLAN
TEMPLATE
Test plan identifier
Test deliverables
Introduction
Test tasks
Test items
Environmental needs
Features to be tested
Responsibilities
Features not to be tested
Staffing and training needs
Approach Schedule
Item pass/fail criteria
Risks and contingencies
Suspension and resumption criteria Approvals
Things
to keep in mind while planning tests
A good test plan is always kept short and focused. At
a high level, you need to consider the purpose served by the testing work.
Hence, it is really very important to keep the following things in mind while
planning tests:
§ What is in scope and what is out of scope for this
testing effort?
§ What are the test objectives?
§ What are the important project and product risks?
(details on risks will discuss in Section 5.5).
§ What constraints affect testing (e.g., budget
limitations, hard deadlines, etc.)?
§ What is most critical for this product and project?
§ Which aspects of the product are more (or less)
testable?
§ What should be the overall test execution schedule and
how should we decide the order in which to run specific tests? (Product and
planning risks, discussed later in this chapter, will influence the answers to
these questions.)
§ How to split the testing work into various levels
(e.g., component, integration, system and acceptance).
§ If that decision has already been made, you need to
decide how to best fit your testing work in the level you are responsible for
with the testing work done in those other test levels.
§ During the analysis and design of tests, you’ll want
to reduce gaps and overlap between levels and, during test execution, you’ll
want to coordinate between the levels. Such details dealing with inter-level
coordination are often addressed in the master test plan.
§ In addition to integrating and coordinating between
test levels, you should also plan to integrate and coordinate all the testing
work to be done with the rest of the project. For example, what items must be
acquired for the testing?
§ When will the programmers complete work on the system
under test?
§ What operations support is required for the test
environment?
§ What kind of information must be delivered to the
maintenance team at the end of testing?
§ How many resources are required to carry out the work.
Now, think about what would be true about the project
when the project was ready to start executing tests. What would be true about
the project when the project was ready to declare test execution done? At what
point can you safely start a particular test level or phase, test suite or test
target? When can you finish it? The factors to consider in such decisions are
often called ‘entry criteria’ and ‘exit criteria.’ For such criteria, typical
factors are:
§ Acquisition and supply: the availability of staff, tools, systems and other
materials required.
§ Test items: the state that the items to be tested must be in to
start and to finish testing.
§ Defects: the number known to be present, the arrival rate, the
number predicted to remain, and the number resolved.
§ Tests: the number run, passed, failed, blocked, skipped, and
so forth.
§ Coverage: the portions of the test basis, the software code or
both that have been tested and which have not.
§ Quality: the status of the important quality characteristics
for the system.
§ Money: the cost of finding the next defect in the current
level of testing compared to the cost of finding it in the next level of
testing (or in production).
§ Risk: the undesirable outcomes that could result from
shipping too early (such as latent defects or untested areas) – or too late
(such as loss of market share).
When writing exit criteria, we try to remember that a
successful project is a balance of quality, budget, schedule and feature
considerations. This is even more important when applying exit criteria at the
end of the project.
