Proactive UAT™ Makes Users Competent, Confident, Committed

Proactive UAT™ Makes Users
Competent, Confident, Committed

Robin F. Goldsmith, JD

GO PRO MANAGEMENT, INC.

SYSTEM ACQUISITION & DEVELOPMENT

BUSINESS ENGINEERING QUALITY/TESTING
PRODUCTIVITY

T RA INI NG 22 CYNTHIA ROAD

NEEDHAM, MA 02494-1412
[email protected]
WWW.GOPROMANAGEMENT.COM

(781) 444-5753 VOICE/FAX

©2005 GO PRO MANAGEMENT, INC. - 1 Proactive UAT™ Makes Users Competent, Confident, Committed

Most Acceptance Testing Is Reactive

z At the end, based on system as written

z Technical view dominates with little user
role/awareness of how and what to test

z A lot of work, often with little payback

– Doesn’t find many of the errors
– Adversarial, arguments, blame
– Too late to fix errors anyhow

Lacking competence leads to lacking
confidence leads to lacking commitment

Proactive Testing™ provides the basis for more
confident and competent user commitment to UAT

©2005 GO PRO MANAGEMENT, INC. - 2 Proactive UAT™ Makes Users Competent, Confident, Committed

Objectives

z Identify reasons users often are reluctant to
participate in User Acceptance Testing (UAT)

z Distinguish conventional testing’s often
ineffective approaches to UAT from more
successful Proactive User Acceptance Testing™

z Describe professional testers’ role in helping
users define Acceptance Criteria that create
competence, confidence, and commitment

©2005 GO PRO MANAGEMENT, INC. - 3 Proactive UAT™ Makes Users Competent, Confident, Committed

“V” Model Shows Levels of Testing

Business (User) Requirements Acceptance Test
High-Level Design System Test

Low-Level Design Integration Test
Code Unit Test

©2005 GO PRO MANAGEMENT, INC. - 4 Proactive UAT™ Makes Users Competent, Confident, Committed

Reasons for User Resistance

z Lack of confidence in what and how to test
z Testing against system, not business requirements
z Insufficient time or resources (may signal other

issues)
z “Not my job” syndrome [may be correct]
z Don’t find much anyhow
z Fear of blame for found or unfound defects
z Belief that feedback will be ignored
z Intimidation and misguidance from developer-

and/or tester-centric dominance

©2005 GO PRO MANAGEMENT, INC. - 5 Proactive UAT™ Makes Users Competent, Confident, Committed

Traditional Development and Testing
Gurus Tend to Miss the Boat on UAT

:Consider UAT a test to confirm product’s
system requirements--often expect a
rubber stamp

:Some testing books/courses say

: UAT should be only positive/valid proof-of-
concept tests to demonstrate normal
functionality

: UAT should be a repeated subset of the
System Test, run by users

: UAT simply needs one test for each
functional requirement or use case scenario

Yet, organizations continually rely on UAT to catch many missed problems

©2005 GO PRO MANAGEMENT, INC. - 6 Proactive UAT™ Makes Users Competent, Confident, Committed

Why Do Users Need to Do Thorough

User Acceptance Testing (UAT)?

Self-defense:

Acceptance Testing is the Users often are like
user’s chance and duty to goalkeepers defending
confirm the system works alone without protection
properly, from the user’s from fullbacks and
standpoint, before users halfbacks
and the organization rely
upon it.

Can’t we just trust the developers to do their jobs right?

©2005 GO PRO MANAGEMENT, INC. - 7 Proactive UAT™ Makes Users Competent, Confident, Committed

Proactive TestingTM Life Cycle

FEASIBILITY Feasibility SYSTEMS
ANALYSIS Report ANALYSIS

See: Business SYSTEM
www.sdmagazine.com Requirements DESIGN
/articles/2002/0207
/0208 /0209 /0210 Requirements- High-Level
Based Tests Low-Level
Design
Acceptance
Criteria Technical
Test Plans

OPERATIONS & Acceptance Formal Review DEVELOP-
MAINT. Test Plan MENT
Black/White Box
[Life Cycle reit.] IMPLEMEN- Unit Tests Code (Debug,
TATION Informal Review)
Integration Tests
Acceptance System, Special
Test
Tests

Independent
(QA) Tests

©2005 GO PRO MANAGEMENT, INC. - 8 Proactive UAT™ Makes Users Competent, Confident, Committed

Key Proactive Testing™ Concepts

Find WIIFMs
z Intertwine testing with each development deliverable
z Plan before acting at any point/level, independent paths

– Acceptance testing first; users shouldn’t do technical tests
– Prepare test plans/designs during Design, promote reuse
– Prioritize by level, from full choices, to avoid wasted effort

z Let testing drive development

– Feedback test plans/designs to guide correct coding
– Selectively structure for early tests that prevent rework
– Measure objectively to guide decisions, more CAT-Scans

©2005 GO PRO MANAGEMENT, INC. - 9 Proactive UAT™ Makes Users Competent, Confident, Committed

Proactive Testing™ Strategy/Roles:
More Thorough Tests of Cleaner Code

¾ Enable developers to code ¾ Professional testers

more correctly by using testing ¾ Specialize in test methods

to improve requirements and ¾ Tend to like testing, detail

designs ¾ Double check, any/all test

¾ Enable developers to catch levels, technical view, but
more of their own (fewer) with greater user sensitivity

remaining errors by more ¾ Users double check cleaner
explicit systematic testing from software from business
perspective as they actually
business and technical
will use system
perspectives

©2005 GO PRO MANAGEMENT, INC. - 10 Proactive UAT™ Makes Users Competent, Confident, Committed

Approaches to Acceptance Testing

Reactive--Run what’s given to you Proactive--Plan what to run

z Here it is, try it out 9 Test each requirement

– Assumes understanding of 9 Minimum for acceptability

how system works and is to 9 Assures requirements are
be used testable (sufficient?

– Reasonable for changes to 9 (Variant) Run Use Cases
known system
9 Define acceptance criteria
z (Variant) Test that it works
the way it’s written 9 Independently of
requirements, sets priorities
– Check choices, links, fields
9 A test of the requirements
– Match to user instructions
¾ Execute the plan

©2005 GO PRO MANAGEMENT, INC. - 11 Proactive UAT™ Makes Users Competent, Confident, Committed

ÂTwo Types of Requirements:

Business/User System/Software
z Technical/product operational
z User view & language,
conceptual; exists within the view, language

business environment z Human-defined system/

z Serves business objectives product design of one of the

z What business results must possible ways, How, to
be delivered to solve a deliver the business results

business need (problem, z What external functions each

opportunity, or challenge) and piece of the product must
provide value when delivered accomplish for it to work as
designed (Functional
Many possible ways to accomplish Specifications)

©2005 GO PRO MANAGEMENT, INC. - 12 Proactive UAT™ Makes Users Competent, Confident, Committed

Even Requirements “Experts” Think
the Difference is Detail

Business Requirements
(High-Level, Vague)

System/ Reqs.
Software (Detailed)

©2005 GO PRO MANAGEMENT, INC. - 13 Proactive UAT™ Makes Users Competent, Confident, Committed

When Business/User Requirements

Are Detailed First, Creep Is Reduced
User Acceptance Test Technical Tests

Business Requirements System/Software Reqs.
(High-Level) (High-Level)

Business Reqs. System/ Reqs.
(Detailed) Software (Detailed)

©2005 GO PRO MANAGEMENT, INC. - 14 Proactive UAT™ Makes Users Competent, Confident, Committed

What Tests Are Needed to Be Confident
This Requirement Is Met?

Separately identify sales tax
calculated as 5% of purchase price
and include it in the total amount due

©2005 GO PRO MANAGEMENT, INC. - 15 Proactive UAT™ Makes Users Competent, Confident, Committed

Define the Use Case Scenario(s)
that Test This Requirement?

©2005 GO PRO MANAGEMENT, INC. - 16 Proactive UAT™ Makes Users Competent, Confident, Committed

Requirements-Based Test Issues:

Must Be Business Requirements

z Tests are only as good as z Use Cases define usage, not

the requirements definition- user, requirements of a product

irrelevant for missing/wrong – Users are unlikely to write Use

z It can be hard to tell what Cases; they’ll use existing ones,
the requirements are but then not user’s view

– Narratives are confusing – Don’t address business rules or
quality factors
– Itemized format helps
z Seldom written from a thorough
– Needs detail, much more testing perspective, happy path
but not the multiple (esp.
z Need at least 2 tests per negative) test conditions
requirement, usually more

©2005 GO PRO MANAGEMENT, INC. - 17 Proactive UAT™ Makes Users Competent, Confident, Committed

Acceptance Criteria--Define Without
Referring to the Requirements

z What the users/customers/stakeholders (may be
multiple perspectives) must have demonstrated
to be confident the delivered system works—
before they stake their jobs on system working

z Determination will be made whether or not it is
conscious, planned, or explicit

z True empowerment builds cooperation

– Ability to accept or reject delivered system

Identifies overlooked and incorrect requirements

©2005 GO PRO MANAGEMENT, INC. - 18 Proactive UAT™ Makes Users Competent, Confident, Committed

1. What Functions the System Must
Perform

¾ From user’s perspective

¾ Real world business processes, not programs

¾ Original source through end-use

¾ Common and likely to be problematical

¾ May include structure, e.g., checking every
menu choice, every field, all outputs

¾ Technical role is facilitator, gatherer, organizer
(normal, not just new/problems)

©2005 GO PRO MANAGEMENT, INC. - 19 Proactive UAT™ Makes Users Competent, Confident, Committed

2. How Much Must Be Shown to
Give Confidence It Works

¾ How many (and which)

¾ Transactions, accounts
¾ Users, locations

¾ Days, business cycles

¾ How frequently, how extensively should they be
demonstrated

¾ What environments should they be in

©2005 GO PRO MANAGEMENT, INC. - 20 Proactive UAT™ Makes Users Competent, Confident, Committed

3. How Well, How System Quality
Will Be Assessed

¾ Performance measures

¾ Response times, turnaround, throughput
¾ Staffing requirements and productivity
¾ Accuracy, errors and responses to them

¾ Subjective judgments

¾ Usability
¾ Suitability
¾ Whose opinion counts, guidelines

©2005 GO PRO MANAGEMENT, INC. - 21 Proactive UAT™ Makes Users Competent, Confident, Committed

4. By Whom, Who Should Perform
Tests for Confidence We Can Use It

¾ Actual end-users themselves

¾ Typical, randomly selected

¾ Mix of locations, skills, roles

¾ “Super,” “power,” or “good” users

¾ User stand-ins, e.g., business analysts, liaisons,
temporary helpers, full-time acceptance testers

¾ Non-users, e.g., QA or testing group,
documentation writer, developers

©2005 GO PRO MANAGEMENT, INC. - 22 Proactive UAT™ Makes Users Competent, Confident, Committed

5. In What Manner, Test Format for
Confidence the System Works

¾ Stand-alone--test inputs and outputs
¾ Parallel--production inputs, test outputs

¾ Most complete definition of “right” results
¾ Old, new, reconcile is three times the work

¾ Pilot--subset, production inputs and outputs

¾ No definition of expected results
¾ Bridging adds complexity and risk to test

¾ Big Bang--production inputs and outputs

©2005 GO PRO MANAGEMENT, INC. - 23 Proactive UAT™ Makes Users Competent, Confident, Committed

Acceptance Criteria Example 1 of 2

FUNCTIONS
1 Create an order for one taxable item.
2 Create an order for one non-taxable item.
3 Create an order that includes two or more taxable items
and two or more nontaxable items.
4 For an order with two or more taxable items and two or more
nontaxable items, cancel one of the taxable items and one of
the nontaxable items.
5 Cancel an entire order that included two or more taxable items
and two or more nontaxable items.
6 Produce credit vouchers for returned taxable and nontaxable items.
7 Modify sales prices of already-entered taxable and nontaxable items.
8 Create orders for items costing less than $1.00 and more than $999.
9 Change the sales tax rate to 5.25 percent.

©2005 GO PRO MANAGEMENT, INC. - 24 Proactive UAT™ Makes Users Competent, Confident, Committed

Acceptance Criteria Example 2 of 2

HOW MUCH
10 Create orders that cover one page and two pages.
11 Create an order with total sales tax of more than $1000.
HOW WELL
12 Have an untrained user enter orders.
13 Create an order for 25 different types of items.
BY WHOM
14 At least two real users, one with and one without training.
IN WHAT MANNER
15 Stand-alone.

©2005 GO PRO MANAGEMENT, INC. - 25 Proactive UAT™ Makes Users Competent, Confident, Committed

Summary

¾ Perceived lack of competence and confidence often
makes users reluctant to participate in User
Acceptance Testing (UAT)

¾ Conventional testing’s common testing-centric views
of UAT can render UAT an ineffective
disempowering rubber-stamp time-waster

¾ Proactive User Acceptance Testing™ empowers
users by defining Acceptance Criteria that are
important to the users and create competence,
confidence, and commitment

©2005 GO PRO MANAGEMENT, INC. - 26 Proactive UAT™ Makes Users Competent, Confident, Committed

Go Pro Management, Inc. Seminars--Relation to Life Cycle

Systems QA Improving the REAL Software Process
Managing System Projects with Credibility

System Measurement IT ROI Test Process Management

Feasibility Proactive User Acceptance Testing

Analysis Systems Proactive Risk-Based Testing

Analysis System Develop-

Defining and Managing Design ment Implement-

User Requirements Reusable Test Designs ation Operations
Maintenance
Re-Engineering:
Opportunities for IS Test Estimation

Testing Early in the Life Cycle Winning Approaches to

21 Ways to Test Requirements Structured Software Testing

Managing Software Acquisition and Outsourcing:
> Purchasing Software and Services
> Controlling an Existing Vendor’s Performance

Making You a Leader

©2005 GO PRO MANAGEMENT, INC. - 27 Proactive UAT™ Makes Users Competent, Confident, Committed

Robin F. Goldsmith, JD

[email protected] (781) 444-5753

www.gopromanagement.com

• President of Go Pro Management, Inc. consultancy since 1982, working directly with and training
professionals in business engineering, requirements analysis, software acquisition, project
management, quality and testing.

• Previously a developer, systems programmer/DBA/QA, and project leader with the City of
Cleveland, leading financial institutions, and a “Big 5” consulting firm.

• Degrees: Kenyon College, A.B.; Pennsylvania State University, M.S. in Psychology; Suffolk
University, J.D.; Boston University, LL.M. in Tax Law.

• Published author and frequent speaker at leading professional conferences.
• Formerly International Vice President of the Association for Systems Management and

Executive Editor of the Journal of Systems Management.
• Founding Chairman of the New England Center for Organizational Effectiveness.
• Member of the Boston SPIN and SEPG’95 Planning and Program Committees.
• Chair of BOSCON 2000 and 2001, ASQ Boston Section‘s Annual Quality Conferences.
• Member ASQ Software Division Methods Committee.
• Admitted to the Massachusetts Bar and licensed to practice law in Massachusetts.
• Author of book: Discovering REAL Business Requirements for Software Project Success

©2005 GO PRO MANAGEMENT, INC. - 28 Proactive UAT™ Makes Users Competent, Confident, Committed