EqPart

Black Box Testing Methods

In black box testing we don't have access to the internal mechanism of a component, so we are limited to providing input data and inferring correctness based on its externally observable behavior. For any but the most trivial program, the number of potential input data items is often astronomical, making exhaustive testing impractical. So we need strategies for inventing tests that have a high likelihood of revealing a defect. Two common techniques are Equivalence Class Partitioning and Boundary Value Analysis

Equivalence Class Partitioning

Suppose that we were going to test a program that reads an integer and prints the absolute value. Exhaustive testing would require testing with every possible integer value. For anything other than trivial programs like this one, the number of test cases can quickly become astronomical. In addition, many of the tests would not be "good" tests in the sense that they don't have a high likelihood of revealing a defect.

One strategy for inventing tests that will give us the most "bang for the buck" involves identifying a small set of representative input values that invoke as many different input conditions as possible.

Equivalence Partitioning divides the input domain of a program into classes of data from which test cases can be derived. The theory says we only need to create one test case for each "class" of input data. The intent is to define a representative or "power" test case that uncovers a class of errors, thereby reducing the total number of test cases that must be developed.

An equivalence class is a set or range of input domain values that can reasonably be expected to cause “similar” responses from the application under test (AUT).

Example:
If the absolute value function produces the correct result for input of -36, is there any reason we would expect it to fail for -37? No. These two values are candidates to be in the same equivalence class.
On the other hand, -36 and +37 would react differently.
| -36 | = 36, while | 37 | = 37.
In one case, the absolute value is the negative of the input, while in the other case, the output is the same as the input. These two values should definitely be in different equivalence classes.

Thus, for the absolute value function, we define three equivalence classes: positive numbers, negative numbers, and zero.

Rules for Creating Equivalence Classes

The following guidelines are applied to assist in defining equivalence classes:

if an input condition specifies a range, one valid and two invalid equivalence classes are defined: within the range is valid, too large or too small are invalid.
Example: Age is a number from 1 - 99.
Valid equivalence class: [1 - 99]
Invalid equivalence class: [less than one]
Invalid equivalence class: [greater than 99]
if an input condition requires a member of an enumerated set where all values are treated the same, one valid class for values in the set and one invalid equivalence class are defined.
Example: San Luis Obispo = {SLO, San Luis, Slo Town}
Valid equivalence class: [SLO, San Luis, Slo Town]
Invalid equivalence class: [Pismo]
if an input condition requires a member of an enumerated set where each is treated differently, one valid class for each value in the set and one invalid equivalence class are defined.
Example: Vehicle = {car, truck, motorcycle}
Valid equivalence class: [car]
Valid equivalence class: [truck]
Valid equivalence class: [motorcycle]
Invalid equivalence class: [skateboard]
if an input condition is boolean, two valid classes are defined.
Example: overnight shipping requested
Valid equivalence class: [yes]
Valid equivalence class: [no]

Be careful, some conditions may sound like a boolean, but are actually a range,
for example: "first character must be a letter."
The problem domain defines some characteristic that distinguishes somes inputs from others.
Example: a square is a 4-sided polygon where all sides have equal length.
Valid equivalence class: [5,5,5,5]
Invalid equivalence class: [5,5,6,6]

To some extent defining equivalence classes is an art, so use your knowledge of the problem domain and your intuition. If you suspect that elements in an equivalence class are not handled in an identical manner by the AUT, split the equivalence class into smaller classes.

Choosing Test Input Values

Simply choose a “representative” value from each equivalence class. Any value ought to be as good as any other.

Examples:

Valid equivalence class: [1 - 99]	Choose 50
Invalid equivalence class: [Integer.MIN_VALUE, 0]	Choose -20
Valid equivalence class: [SLO, San Luis, Slo Town]	Choose SLO

Deriving Test Cases

Since the results of the partitioning can yield a large number of classes, we want to be clever about deriving test cases in order to minimize the number of tests.

Assign a unique identifier to each equivalence class.
Until all valid equivalence classes have been covered by at least one test case, write a new test case covering as many of the valid equivalence classes as possible. Note: The disadvantage is if the test fails, you don't immediately know which class is guilty; more diagnosis is required. But you do know the candidates to inspect. And in general, minimizing the number of tests will save resources in the long run.
Until all invalid equivalence classes have been covered, write a test case that covers one, and only one, of the uncovered invalid equivalence classes. Note: These require separate tests because they are expected to fail, and if you put two together, one failing could disguise the other that isn't.
For each test case, annotate it with the equivalence class identifiers that it covers.

Special Concerns

Strongly typed languages eliminate the need for the consideration of some invalid inputs. For example, with Java enumerated types, "skateboard" would not be allowed as a value in rule 3 above.
You may find a specification doesn't define expected output for invalid equivalence classes. Usually this indicates that the an additional requirement needs to be added to the SRS.
If the input states a precondition, then we don't need to test inputs that violate the precondition.

Example

Specification:
Input is length of three sides of a triangle: a, b, c
Output true if each side is a positive number less or equal to 20
and the triangle is Isosceles ( 2 sides are equal ), false otherwise.
Precondition: the sides form a triangle.

Rule	Valid equivalence class	Invalid equivalence class
#1	V1: a,b,c are all [1 - 20]	X1: a > 20 X2: b > 20 X3: c > 20 X4: a <= 0 X5: b <= 0 X6: c <= 0
#5	V2: a == b, a != c V3: a == c, a != b V4: b == c, a != b	X7: a == b == c X8: a != b != c

Test Cases

#	Test Data	Expected Outcome	Classes Covered
1	5,5,2	T	V1, V2
2	5,3,5	T	V3
3	4,7,7	T	V4
4	5,5,5	F	X7
5	3,4,5	F	X8
6	0,2,2	F	X4
7	2,0,2	F	X5
8	2,2,0	F	X6
9	22,3,3	F	X1
10	3,22,3	F	X2
11	3,3,22	F	X3

Complete Example

Create test cases using equivalence partitioning for the following problem.

Specification:
Rabbit Mail advertises overnight delivery anywhere in California and two-day delivery in the continental U.S. The delivery fee is fifty cents per ounce for letters in California (75 cents outside of CA), and sixty cents per ounce for parcels (one dollar outside of CA). The maximum weight they deliver is 16 ounces for a letter and ten pounds for a parcel.
The program should read the shipping class, weight (in ounces), and zipcode for the destination and output the fee. If the item weighs too much output "Item too heavy".

Rule	Valid equivalence class	Invalid equivalence class
#1	V1: letter, [1 - 16]	X1: letter, weight < 1 X2: letter, weight > 16
#1	V2: parcel, [1 - 160]	X3: parcel, weight < 1 X4: parcel, weight > 160
#2	V3: zipcode in CA V4: zipcode outside CA, but within continental US	X5: zipcode outside continental US

Test Cases

#	Test Data	Expected Outcome	Classes Covered
1	letter, 10, 93401	$5.00	V1, V3
2	letter, 10, 21001	$7.50	V1, V4
3	parcel, 10, 93401	$6.00	V2
4	parcel, 10, 21001	$10.00	V2, V4
5	letter, -1, 93401	unknown	X1
6	letter, 20, 93401	Item too heavy	X2
7	parcel, -1, 93401	unknown	X3
8	parcel, 200, 93401	Item too heavy	X4
9	letter, 10, 99950	unknown	X5

Notes:
1. "unknown" means not specified. Suggest clarify SRS.
2. You could argue that X1 and X3 are redundant.
3. 99950 is Ketchikan, AK
4. Do you see any potential improvements for this test suite?

Boundary Value Analysis

Complements equivalence partitioning. Once equivalence classes have been determined, design tests for the limits (boundaries) of each class.

Rules for Creating Boundary Value Cases

1. Input is a range a - b	select six test values: just above a at a just below a just above b at b just below b
2. Input specifies n values, with min a and max b	provide values: a, b, a-1, a+1, b-1, b+1

Notes:

Also apply 1 and 2 to output conditions.
Also apply to internal data structures.

Example: Applying BVA to the Rabbit Mail problem would yield these additional Test Cases:

#	Test Data	Expected Outcome	description
1	letter, 1, 93401	$.50	weight lower bound
2	letter, 0, 93401	unknown	one below LB
3	letter, 2, 93401	$1.00	one above LB
4	parcel, 1, 93401	$1.00	weight lower bound
5	parcel, 0, 93401	unknown	one below LB
6	parcel, 2, 93401	$2.00	one above LB
7	parcel, 160, 21001	$160.00	weight upper bound
8	parcel, 159, 21001	$159.00	one below UB
9	parcel, 161, 21001	Item too heavy	one above UB
10	letter, 16, 93401	$8.00	weight upper bound
11	letter, 15, 93401	$7.50	one below UB
12	letter, 17, 93401	Item too heavy	one above UB

CPE 309 Home