Software Testing

Software Testing | Gel Abad

Software
Testing

1

Software Testing | Gel Abad

Table of Contents

Fundamentals of Testing................................................................................................................................
Software Testing and its importance ........................................................................................................ 4
Testing Principles ...................................................................................................................................... 5
Fundamentals of Testing Process ............................................................................................................. 6
Psychology of Testing................................................................................................................................ 8

Testing throughout the software life cycle ..................................................................................................
Software development models ................................................................................................................ 9
Software Testing Levels .......................................................................................................................... 11
Software Test Types................................................................................................................................ 12
Maintenance Testing............................................................................................................................... 13

Static Techniques ...........................................................................................................................................
Reviews and the Test Process ................................................................................................................ 14
Review Process........................................................................................................................................ 16
Static Analysis By Tools ........................................................................................................................... 18

Test design techniques...................................................................................................................................
Identifying test conditions and designing test cases ............................................................................. 19
Categories of test design techniques...................................................................................................... 19
Specification-based or black-box techniques ......................................................................................... 20
Structure-based or white-box techniques .............................................................................................. 24
Experience-based techniques ................................................................................................................ 24
Choosing a test technique....................................................................................................................... 24

2

Software Testing | Gel Abad

Test management...........................................................................................................................................
Test plan ................................................................................................................................................. 25
Test Estimates ......................................................................................................................................... 27
Test strategies ........................................................................................................................................ 30
Test progress monitoring and control .................................................................................................... 31
Configuration management.................................................................................................................... 32
Risk and testing ....................................................................................................................................... 33
Incident management ............................................................................................................................ 34

Tool support for testing .................................................................................................................................
Types of test tool .................................................................................................................................... 35
Effective use of tools: Potential benefits and risks ................................................................................ 36
Introducing a tool into an organization .................................................................................................. 36

References ................................................................................................................................................. 37

3

Software Testing | Gel Abad

Fundamentals of Testing

Testing
It is a procedure for critical evaluation. A means of determining the quality, truth presence of

something.

Software Testing
Software testing, also known as Application Testing is a method of assessing the functionality of

a program such as usability, performance, security and compatibility. It is also the process of validating
and verifying that the software meets the business and technical requirements and identifying technical
bugs.

Image 1.0: Iron triangle of Project Management, is a model of the constraints of project management.

Testing is a vital process of any kind of software methodology because it ensures that the
application is safe, secure and meets the requirements set by the client. Fixing technical bug can be
expensive because it takes time and resources which then affects the overall quality of the system
(Image1.0). Which is why it is important that these bugs are found and fixed as soon as possible. The
earlier these defects are found, the cheaper and easier it will be to fix. However, testing does not only
find errors, but also ensures that the program meets the standards, such as IEEE1 and the client or
stakeholders.

1 IEEE (Institute of Electrical and Electronics Engineers) is an organization that develops global standards.

4

Software Testing | Gel Abad
Principles of Testing

1. Bug finding
- Testing reveals the defects/ errors in the application.

2. Testing software exhaustively is impossible
- Testers need optimum time to test the application.

3. Testing a software early
- Start the testing as early as possible.

4. Defect Clustering
- Most defects are related to a small number of modules.

5. Pesticide Paradox
- As the system evolves, the previously reported defects will have been fixed. Every time a
new functionality is added, regression testing is required to make sure it didn’t affect any
other part of the system.

6. Testing is context dependent
- The most appropriate type of testing depends on the nature of the application.

7. Absence of Errors Fallacy
- Even if the test result shows that there are no defects, it doesn’t mean that the software
is ready to be shipped.

5

Software Testing | Gel Abad

Fundamental Testing Process

Image 2.0: The process of Fundamental Testing

1. Planning and Control
- Identify the objectives of testing
- Determine the test approach
- Implement a test strategy2
- Determine the required resources such as people and PC.
- Schedule test analysis, test implementation, execution and evaluation

2. Analysis and Design
- Review the test basis
- Identify test conditions
- Design the test

2 Test Strategy describes the testing portion of the software development cycle. It is one of the most powerful
factor of the accuracy of the test plans.( Common test strategies: Analytical/Risk Analysis, Methodical/Checklist)

6

Software Testing | Gel Abad

- Evaluate testability of the requirements
- Design the test environment set-up and identify the required tools.

3. Implementation and Execution
Implementation:
- This is the part of the process wherein the test conditions are used as test cases.
- Usage of testware3 (scripts for automation)

Execution:
- Execute test cases following the test procedures
- Re-execute the tests that failed previously.
- Log the outcome of the test execution. Also known as test log4.

4. Evaluating Exit Criteria and Reporting
- Check the test log against the exit criteria specified in test planning
- Assess if more test are needed
- Check whether there are changes to be made in exit criteria
- Write a test summary report for stakeholders

5. Test Closure Activities
- Check which planned deliverables are actually delivered. (ensures that all incident
reports have been resolved)
- Finalize the scripts, test environment for later reuse.
- Handover the testware to the maintenance organization
- Evaluate how the testing went.

3 Testware is the umbrella term for all utilities and application software that serve in combination for testing a
software package but not necessarily contribute to operational purposes.

4 Test log includes the following: test cases, tester, and status (pass/fail)

7

Software Testing | Gel Abad

Psychology of Testing

Mindset of Tester vs Developer
Developers are developing applications based on the specification given by the client. They are

also working to solve the problems encountered during the development process where as testers, looks
for the defects/failures in the software. Thus, building application requires a different mindset from
testing however, both are vital to ensure the success of the Software Development.

Balance between self-testing and independent testing
Developers can also become testers (vise-versa). In fact, developers test their own code before

handing it to others. However, finding the developers on mistake is not an easy task because, sometimes,
programmers tend to protect their own work. That is why, architects, business analyst and programmers
depend on others to test their work. The act of giving the application the testing specialists allows an
independent test of the system.

There are several levels of independence in software testing, which are listed here from the lowest level
of independence to the highest:
1. Tests by the person who wrote the item.
2. Tests by another person within the same team, like another programmer.
3. Tests by the person from some different group such as an independent test team.
4. Tests by a person from a different organization or company, such as outsourced testing or
certification by an external body.

8

Software Testing | Gel Abad

Testing throughout the software life cycle

Software development models
Software Development Life Cycle (SDLC) is a structured sequence of stages in software

engineering to develop the intended software product. The selection of model has a very high impact on
the testing. It will define the what, where and when of our planned testing, influence regression testing,
and largely determines which test techniques to use.
Types of SDLC:

 Waterfall
Simplest model of software development paradigm. Functions in a linear manner.

Image 3.0: Process of waterfall model

9

Software Testing | Gel Abad
 V-Model

At every stage, test plans and test cases are created to verify and validate the product according
to the requirement of that stage.

Image 4.0: Process of V-model
 Agile

Iterative approach is taken and working software builds is delivered after each iteration. Each
build is incremental in terms of features; the final build holds all the features required by the
customer.

Image 4.0: Process of Agile Method

10

Software Testing | Gel Abad
Software Testing Levels

Testing Levels identifies the missing areas, prevents overlap and repetition between
development life cycle phases

Types of Testing Level:
1. Unit testing

– done by developers to check if their code is working fine.
2. Component Testing

– Testing the modules of the application that are developed separately.
3. Integration Testing

– is done when two modules are integrated in order to test the functionality after
integration.
4. System Integration Testing

– Testing all the related systems that should maintain data integrity and can operate in
coordination in the same environment.
5. System Testing

– Tests the compatibility of the application with the system
6. Acceptance Testing

– Ensures that the requirements are met.
7. Beta Testing

– It is done just before the launch of the product.

11

Software Testing | Gel Abad
Software Test Types

Test Types are focused on a particular test objective which could be the performance, reliability,
usability, structure of the system or changes in the system.

Kinds of Software Test Types:
 Functional Testing
It is the testing of the functions of the system. Functional tests tend to answer the
questions like “Can the user do this?” or “does this particular feature work?” This is
typically described in requirements specification.
 Non-Functional Testing
It tests the quality of the system. It refers to the aspect of the software that may not be
related to a specific function. Like: reliability testing, usability testing, learnability,
efficiency, satisfaction and portability.
 Structural testing
The testing of the structure of the system. Often referred as the white box because the
interest only lies on what is happening inside the application. In structural testing, testers
are required to have the knowledge of the internal implementation of the code.
 Change Related Testing
Some of the changes related testing is confirmation testing (ex: confirming that the
defects have been fixed) and regression testing (for unintended changes).

12

Software Testing | Gel Abad
Maintenance Testing

Maintenance Testing is testing that is provided in the phase wherein a system is deployed. During
the time the system and its environment are corrected, changed or extended. It is performed to identify
problems, diagnose the current status of the system.
Two parts of Maintenance:

1. Testing the changes that has been made because of the correction in the system, additional
features were added or the system is extended.

2. Regression testing is done to prove that the rest of the system had not been affected by the
maintenance work

13

Software Testing | Gel Abad

Static Test Techniques

Improves the quality of software products by assisting engineers to recognize and fix their own
defects early in the software development process

Static Testing
Is the manual testing of software products or with a set of tools, but not executed. Meaning,

computers are not necessary because the testing is done without executing the program. Like: reviewing,
walk through and inspection.

Two parts of Static Testing:
 Review
- Used to find and eliminate errors in documents such as requirements, design, test cases.
 Static analysis
- Code written by developers are analyzed for structural defects that may lead to error.

Review and the Test Process

Image 5.0: Different levels of formality by review type
Review process can vary widely in their level of formality. Some times of review are informal while
some are formal. Reviews can also have a variety of objectives. The way a review is conducted will depend
on what its specific objective is.

14

Software Testing | Gel Abad

Typical review objectives include:

 Finding defects
 Gaining understanding
 Generating discussion
 Decision making by consensus

Informal Review

Applied many times during the early stages of the life cycle of the document. The goal is to
improve the quality of the document. Informal reviews are not documented (may be documented but it
is not required). There is no formal process underpinning the review.

Informal Review may be implemented by pair programming wherein, one programmer reviews
the code of the other pair programmer or by a technical lead reviewing design and code.

Formal Review
It is a well-structured and regulated type of review.

Types of Review

 Walkthrough
- Lead by the authors
- Not a formal Process
- Useful for the people if they are not from the software discipline, who are not
used to or cannot easily understand software development process

 Technical Review
- Lead by the trained moderator
- Less formal review
- Often performed as peer review without management participation
- Defects are found by the expert who focus on the content of the document.

 Inspection
- Led by the trained moderators
- Most formal type of review
- Involves peer to examine the product
- The defects found are documented in a logging list or issue log
- Formal follow-up is carried out by the moderator applying exit criteria

15

Software Testing | Gel Abad

Review Process

1. Planning
- Begins with a request for review by the author to the moderator/ inspection
leader.
- Moderator takes care of the scheduling. Performs the entry check 5and also
defines the formal exit criteria. 6
- The entry criteria for any documents to go for the reviews are:
o Does not reveal a large number of major defects
o Documents to be reviewed should be with line numbers
o Should be cleaned up by running automated checks that apply
o Author should feel confident about the quality of the document
so that he can join the review team with that document.
- Author decides which part of the document is to be reviewed. Maximum size is
between 10 to 20 pages.

2. Kick-off
- Optional step in a review procedure.
- Gives a short introduction on the objectives of the review and the documents.
- Relationship between the document under review and the other documents are
also explained especially if the numbers of related documents are high.

3. Preparation
- Reviewers checks the document individually using the related documents,
procedures, rules and checklist provided
- Each participants identifies the defects, questions and comments according to
their understanding of the document and role.
- All issues are recorded using a logging form
- Checking rate7 is the success factor for a thorough preparation.

5 Entry check is done to ensure that the reviewer’s time is not wasted on a document that is not ready
6 Exit criteria are the average number of critical/ major defects found per page. If the number of defects found per
page is more than a certain level then the document must be reviewed again.
7 The success factor for a thorough preparation is the number of checker per hour. Usually the checking rate is in
the range of 5 to 10 pages per hour.

16

Software Testing | Gel Abad

4. Review Meeting
- Logging phase
o The issues and the defects that have been identified during the
preparation step are logged page by page. Every defects and its severity
should be logged. Three severity classes are:
 Critical – the defects will cause downstream damage
 Major – the defects could cause a downstream damage
 Minor – the defects are highly unlikely to cause the
downstream damage.
- Discussion Phase
o The issue that needs a discussion is logged and then handled in the
discussion phase.
o The moderator takes care of the people issues and prevents discussion
from getting too personal.
o The outcome of the discussions is documented for the future reference.

- Decision Phase
o At the end of the meeting, a decision on the document under review
has to be made by the participants, sometimes based on formal exit
criteria8.

5. Rework
- If the number of defects found per page exceeds the certain level then the
document has to be reworked.
- Not every defect that is found leads to rework
- The author is the judge whether the defect has to be fixed

6. Follow-up
- The moderator check to make sure that the author has taken action on all
known defects.
- The moderator takes care of the distribution and collection of feedback incase
the participants decided to update the document.
- It is the responsibility of the moderator to ensure that the information is
correct and stored for future analysis.

8 Exit criteria are the average number of critical/ major defects found per page. If the number of defects found per
page is more than a certain level then the document must be reviewed again.

17

Software Testing | Gel Abad

Static Analysis by tools

Static Analysis

Performed on requirement design or code without actually executing the software. The goal of
static analysis is to find the defects whether or not they may cause failure. Finds defects rather than
failure.

Static Analysis Tools

- It is typically used by the developers before and sometimes during component and integration
testing.

- It is also used by the designers during software modeling
- Compiler can be considered as a static analysis tool because it builds a symbol table, points out

incorrect usage and checks for non-compliance to coding language conventions or syntax.

Static Analysis tools are as follows:

 Coding standards
- Consists of a set of programming rules, naming conventions and layout
specifications.
- If the company uses a well-known coding standard, there will probably be
checking tools available that support that standard.

 Coding Metrics
- Measurement of depth of nesting and number of lines of code.
- Are done to see if the design or code is becoming bigger, more complex and
more difficult to understand and maintain.
- Helps us decide between several designs alternatives.
- An example of structural measure is Cyclomatic complexity Metric wherein, it is
based on the number of decisions in the program.

 Code Structure
- Tells us about the effort required to write the code, to understand the code
while making changes or to test the code using particular tools or technique.
- Things to consider in code structure are the following
o Control flow structure
o Data flow Structure
o Data structure

18

Software Testing | Gel Abad

Test Design Techniques

Test Design

Test Design is the act of creating and writing test suites for testing a software. Test analysis and
identifying test conditions gives us a generic idea for testing. However, test cases needs to be very
specific. Exact and detailed input are important. Refer to IEEE 829 Standard for Test Documentation for
detailed Test Case Documentation Specification.

IEEE 829

It is a standard for documenting the testing of software. It applies to any stage in the testing of
developing software and each stage in the software’s development typically is documented using the
same application of the standard.

Identifying test conditions and designing test cases

 The basis for tests is called the “test basis”. It could be a requirement, technical specification,
the code itself or business process.

 A test condition is simply something that users could test.
 A good way to understand requirements better is to try to define tests to meet those

requirements.
 In identifying test conditions, it starts from a wider component leading to a more specific

category and select which ones to take forward to develop in more detail and combine into test
cases. This is called “Test possibilities”.
 It is a good idea to try and think of twice as many test conditions as needed and throw away the
less important ones. That way, there will be a much better set of test conditions

Categories of test design techniques

 Static Testing Technique
- Static testing is the testing of the software work products manually, or with a set of
tools, but they are not executed.

 Dynamic Testing Technique
- Dynamic Testing technique needs a computer for testing.
- It is done during validation process
- Ex: Unit testing, Integration testing

19

Software Testing | Gel Abad

Specification-based or black-box technique

 The testing concentrates on what the software does and not how it does it.
 It is appropriate at all levels of testing.
 Called “black-box” or input/output driven testing technique because the software is views as a

black-box with inputs and outputs

Types of Specification-based / Black-box technique

1. Equivalence Partitioning
- Divides a set of test conditions into groups/partition that can be considered the
same.
- Tests only one condition in each partition because it is assumed that all the
condition in one partition will be treated the same way by the software.
- If a condition in a partition works, it is implied that all the condition in that
partition will work.(Vise-versa)
- Also known as Equivalence Classes

Example: A software that calculates the interest due
Software Specification:

Rate of Interest Balance in Account

3% $0 - $100
5% $100 - $1000
7% $1000 and above

Actual testing: (error message for invalid)

Status Interest Balance
Rate
Valid $0 - $100.00
Valid 3% $100.01 - $999.99
Valid 5% $1000.00++
Invalid 7% -$0.01
Invalid - A (non-numeric)
-

In this case, if the system does not correctly handle a negative balance the
defects can be found. This approach is more effective and efficient.

20

Software Testing | Gel Abad

2. Boundary Value Analysis
- Based on testing at the boundaries between partitions
- We have both valid boundaries and invalid boundaries
- To apply a boundary value analysis, the minimum and maximum boundary
values from the valid partition together with the first or last value respectively
in each of the invalid partitions adjacent to the valid partition

Example: A software that calculates the interest due. (Refer to the example in
Equivalence partitioning)

Boundary Values
-$0.01
$0.00
$100.00
$100.01
$999.99
$1000.00

Ways to test an open Boundary:

- Check the specification to see if a maximum has been stated somewhere else
for a balance amount
- Investigate other related systems. Ex: the field holds that account balance
figure may be only 6 figures plus 2 decimal. This would give a maximum
account balance of $999 999.99
- Intuitive/Experience Based Approach can also be used if the two methods does
not work. (Try inputting various large values)

3. Decision Table
- A good way to deal with combination of things
- Also referred as “Cause-Effect” table
- Provides a systematic way of stating complex business rules which is useful for
developers as well as for testers

How to use decision tables for test designing

- Identify a suitable function or subsystem which reacts according to a
combination of inputs or event

- It is better to deal with large numbers of conditions by dividing into
subsets and deal with one subset at a time

- Put the combinations in table listing of true and false.

21

Software Testing | Gel Abad

Example: A New Customer wants to open a credit card account.

If you are a new customer and you want to open a credit card account then there are
three conditions first you will get a 15% discount on all your purchases today, second if
you are an existing customer and you hold a loyalty card, you get a 10% discount and third
if you have a coupon, you can get 20% off today (but it can’t be used with the ‘new
customer’ discount).

Image 6.0: decision table for credit card example. “X” means this combination should not occur.

o Rule 3 is an assumption. Since coupon has a greater discount, it
is assumed that customer will choose 20% rather than 15%. Note:
these two discount cannot be stacked since it cannot be used with
new customer discount
o The discounts in Rule 5 are stackable since both of them should
apply.

4. State Transition Testing

- Any system wherein a same input can result in different output depending on
what has happened before (finite state system)

- Often shown as a state diagram
- It is used where some aspect of the system can be in a finite number of different

states and the transitions from one state to another are determined by the rules
of machine.
- Advantage of state transition technique: An important part of the system can be
modeled with greater depth of detail whereas the less important, the model can
use a single state to signify what would be a series of different state.

Four Basic Parts of state transition model:

o The states that the software may occupy (open/closed or
funded/insufficient funds);

22

o The transitions from one state to Software Testing | Gel Abad
another (not all transitions are
allowed); Any given state, one event
can cause only one action
o The events that cause a transition
(closing a file or withdrawing if the same
money); event, from a different state

o The actions that result from a cause a different
transition (an error message or and a different
being given your cash).

Example:

If a user request to withdraw $100 from a bank ATM, the user may be given
cash. Later he may make exactly the same request but it may refuse to give
you the money because of your insufficient balance.

- This is because the state of the user’s bank account has changed.

Entering a Personal Identity Number (PIN) to a bank account. The states are
shown as circles, the transitions as lines with arrows and the events as the
text near the transitions

Figure 7.0: State diagram for PIN Entry

Deriving Test Cases:
- First test case: correct PIN is entered the first time
- Second test case: correct PIN is entered the second time.

23

Software Testing | Gel Abad

- Third test case: correct PIN is entered the third time.
- Fourth test case: incorrect PIN is entered each time (eats card)

Structure-based or white-box techniques

 The tester concentrates on how the software does it. Example: exercising loops
 Testers require knowledge of how the software is implemented.
 Developers use white-box techniques in component testing and component integration

testing.
 Also known as glass-box.

Experience-based Testing Techniques

 The knowledge, skills and background of the people are important to the test conditions and test
cases.

 Experience of both technical and business people is required because they bring different
perspectives to the test analysis and design process.

 Go together with specification-based and structure-based techniques.
 Useful under extreme time pressure.

Choosing a Test Technique

Internal factors that influence the decisions about which technique to use are:

 Models used in developing the system
 Testers knowledge and their experience
 Similar type of defects
 Test objective
 Documentation
 Life Cycle model used

External factors that influence the decisions about which technique to use are:

 Risk Assessment
- The greater the risk, the greater the need for more thorough and more formal testing

 Customer and Contractual Requirements
 Type of System used

- The type of system will influence the choice of techniques.
 Regulatory Requirements
 Time and Budget of the Project

24

Software Testing | Gel Abad

Test Management

Test plan

 Test Plan
A document that describes the scope, approach, resources and schedule of intended

test activities. It is the basis for formally testing any software in a project.

Types of Test Plans:

- Master Test Plan – single high-level test plan for a project that unifies all other test plans
- Testing Type Specific Test Plans – plans for major types of testing.
- Testing Level Specific Test Plans – plans for each level of testing

Importance of Test Planning:

- It guides our thinking.
- It serves as the means of communication with other members of the project team.
- It helps the team to manage change.

Things to consider in when planning tests:

- Scope
- Objectives
- Product Risk
- Constraints affecting tests
- Testable aspect of the product
- Schedule of test execution
- Integrate and coordinate all the test work to be done.
- Test Environment
- Required Resources

25

Software Testing | Gel Abad

IEEE 829 Standard Test Plan Template

 Test Plan Identifier
- Provides a unique identifier for the document
 Test Plan Deliverables
- Test plan
- Test Cases
- Test scripts
- Defect/Enhancement Logs
- Test Reports
 Introduction
- Overview of the test plan
 Test tasks
 Test Items
- List the test items/software and their versions
 Environmental needs
- Specify the properties of test environment and the testing tools
 Features to be tested
- List the features of the software to be tested.
 Responsibilities
- List the responsibilities of each team/individual.
 Features not to be tested
- List the features of the software which will NOT be tested.
 Staffing and Training need
- Specify staffing needs by role and required skills
 Approach Schedule
- Specify the testing level, testing types and the test methods
 Item pass/fail criteria
- Specify the criteria to determine whether each test item has passed or failed
 Risk and Contingencies
- List the identified risks and specify the contingency plan for each risk.
 Suspension and Resumption
- Specify criteria to be used to suspend the testing activity and the tests which must be

redone when testing is resumed.

26

Software Testing | Gel Abad

Test Estimates and Strategies

Test Estimation
A management activity which approximates how long a task would take to complete.

Few points need to be considered before estimating testing activities:

 Check if all requirements are finalize or not.
 If it not then how frequently they are going to be changed.
 All responsibilities and dependencies are clear.
 Check if required infrastructure is ready for testing or not.
 Check if before estimating task is all assumptions and risks are documented.

Basis of Calculation of Test Estimation Technique:

 Past Data/Past experience
 Available documents/Knowledge
 Assumptions
 Calculated risks

Types of Software Testing Estimation Technique:

 Delphi Technique
- Collects the data from participants who are experts.
- Each task is assigned to each team member
- Multiple rounds of surveys are conduct
- In each round, task are gathered and feedback is
provided

 Work Breakdown Structure (WBS) task scheduling,
- Breaks down the project into individual components cost estimation
in a hierarchal structure
- Project is divided into modules and are divided into necessary tasks
sub-modules
- Sub-modules are divided into functionality.
- Each Functionality can be divided into sub
functionality.

27

Software Testing | Gel Abad

 Three Point Estimation E
- Based on statistical data
- Tasks are broken down into subtasks. estimation of the task
- Three types of estimation are done on this sub pieces SD
o Optimistic Estimate (Best case scenario) = a
o Most Likely Estimate = m estimation is
o Pessimistic Estimate (worst case scenario) = b correct

Formula to Find Value for Estimate (E):

Standard Deviation:

 Functional Point Method
- Estimates size, duration and cost for the tasks
- Functional Points are divided to:
o Complex
– Compromise of multiple components that interact with each other
o Medium
– System with limited number of components
o Simple
– composed of small components

- Base on the complex of software functions, test manager has to give WEIGHTAGE to
each functional point. Example:

28

Software Testing | Gel Abad
- After classifying the complexity of the function points. The next step is to estimate the

duration to test them.
Example:

- The total effort to complete the task is around 498 man-hours.
Test Estimation Best Practices:

 Add some buffer time
 Account Resource Planning in estimation (Ex: long leave of a member in the team)
 Use Past Experiences as Reference

29

Software Testing | Gel Abad
Test Strategies

The choice of test strategy is one of the most powerful factor in the success of the test effort
and the accuracy of the test plans and estimates.

Types of Test Strategies:
 Analytical
- Risk Based Strategy – involves performing a risk analysis using project documents
- Requirement Based Strategy – analysis of the requirement specification forms the basis
for planning, estimating and designing of tests.
 Model-Based
- Creation or selection of some formal or informal model for critical system behaviors.
 Methodical
- Systematized approach that has been developed in-house
- Assembled from various concepts developed in-house and gathered from outside
 Process – or standard Compliant
- Reliance upon an externally developed approach of testing Like IEEE 829, Extreme
Programming.
 Dynamic
- Concentrates on finding as many defects as possible during test execution.
 Consultative or Directed
- Involves reliance on a group of non-testers to guide/ perform the testing effort.
 Regression-Averse
- Usually automated.
- Testing is almost entirely focused on testing functions that already have been released.

30

Software Testing | Gel Abad

Test progress monitoring and control

Test Monitoring

 Process of evaluating and providing feedback of the currently in progress testing phase.
 Allows opportunities to guide and improve the testing and the project
 Measures the status of the testing, coverage, and test items against the exit criteria.
 Gather data for use in estimating future efforts

Test monitoring activity includes:
 Providing feed back to the team and the other required stakeholders about the progress of the
testing efforts.
 Broadcasting the results of testing to associated members.
 Finding and tracking the test metrics.
 Planning and estimation and deciding the future course of action based on the metrics
calculated.

Metrics that can be used for test monitoring:
 Test coverage metric
 Test execution metrics (Number of test cases pass, fail, blocked, on hold)
 Defect metrics
 Requirement traceability metrics
 Miscellaneous metrics like level of confidence of testers, dates milestones, cost, schedule and
turnaround time.
 Extent of completion of test environment preparation
 Status of the testing (analysis, design and implementation)

Test Control

 Activity of guiding and taking corrective action based on some metrics/information to improve
the efficiency and quality

Sample Action for Test Control:

 Prioritize the testing efforts
 Revisiting the test schedules and dates
 Re-organizing the test environment
 Re-prioritizing the test cases and conditions

31

Software Testing | Gel Abad
Configuration Management

 Determines the items that make up the software (Source code, test scripts, third-party software,
hardware, data and both development and test documentation)

 Ascertain that the “items” are managed carefully, thoroughly and attentively.
 Has a number of important implications for testing. (Allows testers to manage their testware)
 Supports the build process which is important for delivery of a test release into the test

Environment
 Allows the team to keep the record of what is being tested to the underlying files and

components that make it up.
Ideally, when testers receive an organized, version-controlled test release from a change-managed source
code repository, it is along with a test item transmittal report or release notes [IEE 829]

32

Software Testing | Gel Abad

Risk and Testing

Risks
 Possible problems that might endanger the objectives of the project stakeholders
 A Potential Problem
 Something that has not happened yet and it may never happen.

Categories of Risks:

 Product Risk
- Possibility that the software might fail to satisfy some reasonable expectation of the
client.

The product risks that can put the product or software in danger are:

o If the software skips some key function that the customers specified, the users
required or the stakeholders were promised.

o If the software is unreliable and frequently fails to work.
o If software fail in ways that cause financial or other damage to a user or the company

that user works for.
o If the software has problems related to a particular quality characteristic, which

might not be functionality, but rather security, reliability, usability, maintainability
or performance.

 Project Risk

The project risk that can endanger the project are: Remember to
likelihood of occurrence of
o Risk such as the late delivery of the test items the risk and the impact of
to the test team or availability issues with the the risk.
test environment.
Early risk analysis, are often
o There are also indirect risks such as excessive educated guesses. At key
delays in repairing defects found in testing or project milestones it’s
problems with getting professional system important to
administration support for the test
environment. on
the risk analysis

33

Software Testing | Gel Abad
For any risk, project risk or product risk we have four typical actions that we can take:

 Mitigate: Take steps in advance to reduce the possibility and impact of the risk.
 Contingency: Have a plan in place to reduce the possibility of the risk to become an outcome.
 Transfer: Convince some other member of the team or project stakeholder to reduce the

probability or accept the impact of the risk.
 Ignore: Ignore the risk, which is usually a good option only when there is little that can be done

or when the possibility and impact of that risk are low in the project.

Risk based testing
 Uses risk to prioritize and emphasize the appropriate tests during test execution.
 Starts early in the project
 Involves both mitigation and contingency

Guidelines on how to perform risk based testing:
1. Make a prioritized list of risks.
2. Perform testing that explores each risk.
3. As risks evaporate and new ones emerge, adjust your test effort to stay focused on the current
crop.

Incident Management
 When actual result is different from the expected result, then it is called as: incident, bugs,
defects, issues
 Other causes of incidents include misconfiguration or failure of the test environment, corrupted
test data, bad test, invalid expected results and tester mistakes

34

Software Testing | Gel Abad

Tool support for testing

Types of Test Tools: Use computers to do things
that they are really good at.
Classification of different types of test tools according to the test
process activities: Tool support is very useful
for
1. Tool support for management of testing and tests:
 Test management tools A tool that measures some
 Requirements management tools aspect of software may have
 Incident management tools few unexpected side-effects
 Configuration management tools also on that software. Like:

 is an alteration

2. Tool support for static testing: in the frequency of run-time
 Review process support tools
 Static analysis tools (D) computational errors
 Modelling tools (D)
observed when delays are
3. Tool support for test specification:
 Test design tools introduced into concurrent
 Test data preparation tools
programs
4. Tool support for test execution and logging:
 Test execution tools is a bug that
 Test harness/ Unit test framework tools (D) disappears or alters its
 Test comparators behavior when one
 Coverage measurement tools (D) attempts to probe or isolate
 Security tools it.

Ex: If the code is run with the
5. Tools support for performance and monitoring: debugger, then the bug
 Dynamic analysis tools (D) disappears; it only re-
 Performance testing, Load testing and stress-testing appears when the debugger
is turned off.
tools
 Monitoring tools

35

Software Testing | Gel Abad

Effective use of tools: Potential benefits and risks

Advantages
 Reduction of repetitive work
 Greater consistency and repeatability
 Objective Assessment
 Ease of access to information about tests or testing

Disadvantages

 Unrealistic expectations from the tool
 People often make mistakes by underestimating time, cost and effort for the initial

introduction of a tool
 People frequently miscalculate the time and effort needed to achieve significant and

continuing benefits from the tool
 People depend on the tool a lot.

Introducing a tool into an organization

 The organization must be ready to introduce, use and be involved in adapting the tool through
its test processes.

 During evaluation phase of a test tool, identify whether it performs effectively within the
software under test and within the current infrastructure and the potential changes needed.

 Start on a pilot project once the assessment is done for learning more detail about the tool,
confirming that the assumptions are justified and mastering its use before a large
implementation or not

 For deploying a tool efficiently within an organization, success factors can be followed like :
o Rolling out the tool to the rest of the organization incrementally.
o Adapting and improving processes to fit in with the use of the tool.
o Providing training and coaching/mentoring for new users.
o Defining usage guidelines.
o Implementing a way to gather usage information from the actual use of a given tool.
o Monitoring tool use and benefits.
o Providing support for the test team for a given tool.
o Gathering lessons learned from all teams.

36

Software Testing | Gel Abad

References:

Janssen, C. (N.d.). Software Testing. Retrieved June 26, 2015, from
http://www.techopedia.com/definition/17681/software-testing\
Ghahrai, A. (2008, November 30). Seven Principles of Software Testing. Retrieved June 26, 2015, from
http://www.testingexcellence.com/seven-principles-of-software-testing/
Tutorials Point. (n.d) Retrieved June 26, 2015, from
http://www.tutorialspoint.com/software_engineering/software_development_life_cycle.htm
ISTQB EXAM CERTIFICATION. (n.d.) Retrieved June 26, 2015, from http://istqbexamcertification.com/
Software Testing Review Process. (2011, June 3). Retrieved June 29, 2015,
from http://aboutqtp10.blogspot.com/2011/06/software-testing-reviews-test-process.html
Static Testing. (n.d.). Retrieved June 29, 2015, from
http://www.tutorialspoint.com/software_testing_dictionary/static_testing.htm
IEEE 829. (n.d.). Retrieved June 29, 2015, from http://www.webopedia.com/TERM/I/IEEE_829.html
Graham, D., & Veen, E. (n.d.). Identifying Test Conditions. Retrieved June 29, 2015, from
http://learnthebasicsofsoftwaretesting.blogspot.com/2013/05/blog-post.html
TEST PLAN Fundamentals. (2011, April 26). Retrieved June 30, 2015, from
http://softwaretestingfundamentals.com/test-plan/
Introducing a Tool into an Organization. (2012, November 26). Retrieved June 30, 2015, from
http://istqbfoundation.blogspot.com/2012/11/63-introducing-tool-into-organization-k1.html
What is Test Monitoring and Test Control? (n.d.). Retrieved June 30, 2015, from
http://www.softwaretestinghelp.com/test-monitoring-and-test-control/
Software Test Estimation: An Expert View. (n.d.). Retrieved June 30, 2015, from
http://www.guru99.com/an-expert-view-on-test-estimation.html
Software Estimation Techniques - Common Test Estimation Techniques used in SDLC. (n.d.). Retrieved
June 30, 2015, from http://www.softwaretestingclass.com/software-estimation-techniques/

37