Test Cases from a Specification: An Example

 

John D. McGregor

 

For the past two columns I have been talking about developing test cases from the specification for a class. I have discussed several choices for representing specification information and the types of tests that can be derived from the various types of specification information. I also described techniques for reusing test cases when there is an inheritance relation between two classes. This month I will provide a detailed example. Much of the code comes from a fairly simple example that we use in one of my companyís courses. I will use two classes from the program.

 

While I will show examples of constructing test cases in this column, one thing I definitely will not do is address the issue of how much testing is enough. Different company cultures, different domain communities and individual testersí experiences all influence how much testing is conducted. I will simply talk about specific tests and if you think there should be more, add them and if you think what I have is too much then remove some!

The Classes Under Test

The example I am going to use is an instance of the Brickles game that appeared on many early home computers. The game has been implemented in Java and C++ and we use the example in several of our training courses at Software Architects. I will use the Java version but it really makes little difference, for tests based on the specification, which language is used. There are a few attributes of a method that may be more precisely specified in one language than another and these provide valuable input into testing.

 

This first class under test represents the velocity of a moving object in the game. The velocity of a moving object has two attributes, the speed of the object and the direction in which the object is moving. Since the graphics are represented in a cartesian coordinate system, the velocity is decomposed into components along the x and y dimensions. The value of each component depends upon the magnitude of the speed and the direction. The complete source code for the class can be obtained from http://www.cs.clemson.edu/~johnmc so that you can consider the types of errors that are in the class and whether tests constructed from the classís specification actually detect them.

 

Listing 1: A Velocity Class

//This class represents the velocity of an object in a video game.

//For this reason, all values are integer

class Velocity{

 

//Pre: None

//Post: A new object exists with a velocity of 0.

public Velocity(){}

 

//Pre: newSpeed>=0

//Post: A new Velocity object exists with

speed == newSpeed and

direction == newDirection

public Velocity(int newSpeed,int newDirection){}

 

//Pre: An instance of Velocity exists

//Post: The x component of the velocity has the same

magnitude but the direction has been reflected

about the y axis.

public void reverseX(){}

 

//Pre: An instance of Velocity exists

//Post: The y component of the velocity has the same

magnitude but the direction has been reflected

about the x axis.

public void reverseY(){}

 

//Pre: An instance of Velocity exists

//Post: The speed attribute has been set to a new value

public void setSpeed(int newSpeed){}

 

//Pre: An instance of Velocity exists

//Post: The direction attribute has been set to its new

value

public void setDirection(int newDirection){}

 

//Pre: An instance of Velocity exists

//Post: The value of the direction attribute is returned

public int getDirection(){}

 

//Pre: An instance of Velocity exists

//Post: The value of the x-component of the velocity

attribute is returned

public int getSpeedX(){}

 

//Pre: An instance of Velocity exists

//Post: The value of the y component of the velocity

attribute is returned

public int getSpeedY(){}

 

//Pre: An instance of Velocity exists

//Post: The x component of the speed is computed as the

product of the speed and the cosine of the

direction. The y component is similarly computed.

void decomposeSpeed(){}

}

 

Note that the pre-conditions for this Java class are carefully written to define when an object comes into existence. This is particularly important in languages such as Smalltalk and Java. In Java much use is made of class methods. For a class method to work, no object needs to have been explicitly created. So, for example, there would probably be a Math.cos() message, in decomposeSpeed, is a message to the Math class rather than a message to an object that must have been initialized by the programmer. The decomposeSpeed method itself has a pre-condition that requires that the programmer has created a Velocity object prior to sending this message.

 

Test Cases

There are several essential features that a test case must possess. First, the test case must define exactly how the pre-conditions will be established before the actual test is conducted. Second, the test case must define a clear sequence of actions, and input data, that constitute the test sequence. Finally, the test case must define an expected result.

 

Our tests must be verifiable. That is, it must be possible to observe the results of the test and determine whether the expected result was achieved. For object-oriented systems this includes being able to examine the state of the object before and after a test is conducted. This last point is a debatable one. A "black-box" testing approach seems to say that only what is externally observable is used to verify the results of a black-box test. But outside of what? A method? An object? I usually want to look at the internal object state after exercising methods because such a small percentage of the overall functionality of an object can be observed via return values. In the PACT approach we typically circumvent information hiding and directly verify directly with the attributes until the accessor methods have been sufficiently well tested to be trustworthy.

 

Expected Results

Having a definitive expected result seems to be an obvious component of a test case, but it is one that is often overlooked and sometimes very difficult to obtain. On projects that involve operations on large databases, the expected results of a specific search often are expensive to determine but still quite necessary. In this environment, the sequencing of tests becomes very important since we probably donít want to reload a fresh database after each specific test. The expected results in this environment must consider the effects of the previous tests.

 

For the Velocity class, the expected results from several of the operations can be prototyped on a spreadsheet such as shown in Figure 1. One thing to be careful of is the conversion from degrees to radians for the trig functions.

 

Figure 1: Test cases calculated in a spreadsheet

 

Interface Specification

The attributes specified for the Velocity class are fairly simple to analyze. The speed is specified to be a simple integer and so is the direction. The table appears in Table 1.

 

Table 1: Specification Analysis for Velocity

Data Attribute

Object Type

Boundary Values

Equivalence Classes

speed

int

0

0, negative values, positive values

direction

int

0, multiples of 90 degrees

four quadrants

 

 

Now letís address the methods in the order they would need to be operational. That means, constructors first.

 

The first constructor presents no challenge from the standpoint of its specification. The default constructor takes no arguments so a single test case that simply invokes the constructor is sufficient. The expected result is by definition, but the constructed object must be checked for compliance with that expected value.

 

The second constructor provides an opportunity to construct a number of different objects that can be used in the same set of subsequent tests. Table 2, the data permutations table, presents the different combinations of data that can be used to create test cases. All of these constructor tests can be verified by examining the values of the internal attributes, speed and direction, or by using the getDirection and getSpeed methods if we have reason to trust their results (more on that later).

 

Table 2: Data Permutations

Test Case #

Value of speed

Value of direction

1

0

0

2

0

90

3

10

0

4

10

90

5

10

180

6

10

270

7

10

360

8

10

45

9

10

135

10

10

225

11

10

315

12

10

405

 

Several notes about the test cases.

 

Modifier methods

Now consider the twin modifier methods, reverseX and reverseY. For reverseX() the intent is to change the x component of the velocity to simulate the physical law of angle of incident equaling angle of reflection. The analysis must factor in the model used by Java. Because Y values increase from the top of the window toward the bottom, the quadrants are labeled as seen in Figure 2. ReverseX() is intended to reflect the velocity about the y axis so we should consider the cases that are likely to behave differently. In the case of periodic behavior (this type of operation is almost always implemented by a trigonometric function). Within a given quadrant I would consider the two axes that bound the quadrant as boundary values, the angle half way between these two axes I would choose as a popular value that will occur many times and a value half way between that value and each of the axes. So I would use, for example, 0, 90, 45, 37 and 68. The last two being chosen basically at random but within the criteria. Since these values lie in the negative y half plane, I would also choose a similar set in the positive y half plane and I would choose them from the negative x half plane just to investigate any possible interaction there.

 

These tests do not constrain the magnitude of the speed attribute at all so we can add a few more constructor tests to the table above to set up objects for these tests. Tests 13 through 16, shown in Table 3, are additional object construction scenarios. If each of these object constructor test cases are followed with a reverseX() message, all the required test cases have been executed. To exercise the reverseY() method, we follow the same procedure as for reverseX(). The constructor tests, shown in test cases 17 through 20 in Table 3, plus the tests from Table 2 are each followed by a reverseY() method.

 

The setSpeed(int) and setDirection(int) methods directly modify the two attributes of the Velocity object. There are no specified limits on the parameters for these two methods so no limit on the test values. We can reuse the analysis that has already been done by using the default constructor test case and by using the parameter values for the other constructor tests as parameters to the setSpeed(int) and setDirection(int) methods. The result of the Velocity() - setSpeed(int) - setDirection(int)" test case should result in an object in exactly the same state as the constructor test case that uses the same data. The only tests that remain to be constructed are the extrema. Using a very large value for direction such as 745 degrees and a negative twin, -745 provides this coverage.

 

 

Figure 2: Analysis of Cases

 

Table 3: Data Permutations

Test Case #

Value of speed

Value of direction

13

20

37

14

20

68

15

20

212

16

20

233

17

20

127

18

20

158

19

20

302

20

20

323

21

20

745

22

20

-745

 

Accessor methods

I have saved the accessor methods for last even though they are the second thing I test, after the constructors. In the PACT approach a baseline test suite is created that tests all of the accessor methods. After these tests are run and verified by direct access to the attributes then other tests can be built and verified using these methods to provide access to the objectís attributes. The getSpeedX(), getSpeedY() and getDirection() methods provide access to three attributes of the class. From a black-box perspective the tester is unaware of whether these methods directly access variables or compute the values that are returned.

 

A sufficient set of test cases for these methods can be constructed from the previous analysis. For the baseline test suite, each attribute is retrieved directly from the variable that holds the attributeís value (if such exists) and compared to the value returned by the accessor method.

Protocol Specification

The protocol specification for an object defines the sequences of messages that will be considered legitimate. These are not flows through the system because the sequence of messages received by an object may well be interspersed with messages to other objects. The state of the object; however, implicitly records these sequences in that the state at any given moment is the result of the sequence of messages received to that point in time.

 

For example, since the Velocity class is part of the Brickles game, we can expect that the reverseX() and reverseY() messages will alternate as the puck bounces first off a horizontal surface and then a vertical one. Testing several sequences of this type will identify any interactions between the two methods.

 

For a more useful example, I want to switch and consider another class, the BricklesGame class. Listing 2 shows the interface for the class and Figure 3 presents a dynamic model for the class. The dynamic model specifies the sequence of messages that are legitimate. Test cases derived from this protocol specification are in the form of graph traversals. We can omit any test case that simply uses a single transition since those will have already been constructed using the other techniques discussed earlier. Each traversal of this state machine will begin with a constructor followed by a sequence of pause/resume messages that transition from InPlay to Paused and back again. Note that the transition labeled OK is in response to the user pressing the OK dialog button.

 

One technique for systematically covering the specification is the n-way switch cover defined by Chow[1] for telecommunication protocol testing. In this technique test cases are developed based on following a transition by N additional transitions beyond the initial one. A one-way transition provides a reasonable level of coverage but does not uncover those faults that are cumulative and will not surface until several repetitions of the same pattern of transitions has been executed. For example, if the switch between paused and running was implemented by an integer counter, a possible error would be to increment the counter on both a pause and a resume. Eventually the integer value would hit MAX_INT and a failure would result of but not until many cumulative iterations of pause() and resume().

 

As always we have the question of how much testing should we do outside the explicit specification. The example here, test case #8, is testing a sequence in which a resume() message is received prior to receiving a pause() message. It is fairly easy to see that the result of such a case should simply leave the game running. This test case would catch an implementation in which the implementer used a boolean and then reversed its value at each call to either pause() or resume(). The GUI might prevent this sequence from happening in the current application, but reusing the component in another application might lead to an error when this behavior is not controlled. This behavior can be tested perhaps by doing a MouseDown (mouse button pressed) event outside the game window and then doing a MouseUp event inside the window. Through callbacks, this is equivalent to doing a resume with no initial pause.

 

Listing 2: BricklesGame Interface

class BricklesGame{

//Pre: None

//Post: A new object exists with a handle to the user interface.

public BricklesGame(BricklesView);

 

//Pre: Game object exists.

//Post: A new playingfield would be returned but the class is abstract.

public abstract PlayingField newPlayingField(ArcadeGameMatch);

 

//Pre: Game object exists.

//Post: The current match object is returned.

public ArcadeGameMatch getMatch();

 

//Pre: Paddle is movable

//Post: A Puck is placed in play.

public void start() throws OutOfPucksException, OutOfObstaclesException;

 

//Pre: Game is in progress.

//Post: Game is paused.

public void pause();

 

//Pre: Game is paused

//Post: Game is progressing.

public void resume();

 

}

 

Figure 3: Dynamic Model for BricklesGame

 

 

Table 4: Test Cases

 

Object Under Test

Message Sequence

Result

1

BricklesGame(BricklesView)

start()

game is InPlay

2

BricklesGame(BricklesView)

start(), pause()

game is Paused

3

BricklesGame(BricklesView)

start(), pause(), resume()

game is InPlay

4

BricklesGame(BricklesView)

start(), pause(), resume(), pause()

game is Paused

5

BricklesGame(BricklesView)

start(), pause(), resume(), pause(), resume()

game is InPlay

6

BricklesGame(BricklesView)

start(), pause(), resume(), play until out of pucks

Lost message displayed

7

BricklesGame(BricklesView)

start(), pause(), resume(), play until out of obstacles

Won message displayed

8

BricklesGame(BricklesView)

start(), resume()

game is InPlay

 

Summary

I have attempted to convey the analysis used to construct the tests, the order in which the tests are conducted and how the tests and analysis can be reused. I look forward to hearing about additional test cases that you think are important but that were left out of this specification-based suite. Remember that I have only been considering tests based on the specification and, for that reason, I have not included some tests that would be part of a complete test suite.

Reference

Chow, Tsun. Testing Software Design Modeled by Finite-State Machines, Transactions on Software Engineering, SE-4, 1987.