Although the first version of the tool generates all the required test cases shown by the decision table, it is still desirable to check the validation across the boundary values. This will ensure that the specification meets the implementation across the boundaries. This is an important aspect of testing for a safety critical application
5.6. Tool Improvement 47
such as the one in hand. Hence, a new version of our test-case generation tool is developed.
Consider the data dependency of the SUT:
M inV alue < B ∗ (C/100) ∗ D ≤ M axV alue The test cases reflecting the boundary values are:
– Valid values of B, C, D such that B ∗ (C/100) ∗ D = MinValue – Valid values of B, C, D such that B ∗ (C/100) ∗ D = MinValue+1 – Valid values of B, C, D such that B ∗ (C/100) ∗ D = MaxValue – Valid values of B, C, D such that B ∗ (C/100) ∗ D = MaxValue+1
Suppose for a particular A and E combination we get as 30 and 18000 the MinValue and the MaxValue, respectively. The test cases representing the boundaries is shown in Figure 5.8.
Figure 5.8: Controlled Randomized Testing along with Decision Table
30
Combination of B, C and D
18000
Combination of B, C and D
The difference between this implementation and previous implementation is that earlier we had to find B, C, D combination such that it lies within (or outside) a big range but now we have to find B, C, D combination such that it is equal to a particular value and may be such a combination does not exist for that particular value. There can be several optimizations that can be done here. Initially, when we have valid ranges as:
1 ≤ B ≤ 9999999 100 ≤ C ≤ 10000
2 ≤ D ≤ 32
A total of 30689996931 (9999999 (valid values for parameter B) * 9901 * 31) comparisons may be required in the worst case for checking whether a product of the three parameters is equal to a particular value. There are 400 such particular values (100 valid A and E combination and four test cases for each combination)
48 Chapter 5. Data Dependency
to be checked. Next, we apply the first optimization by redefining the range for particular MinValue and MaxValue pair, say 30 and 18000 as previous.
1 ≤ B ≤ 9000 100 ≤ C ≤ 10000
2 ≤ D ≤ 32
Again, a total of 27900000 comparisons may be required in the worst case for checking a particular value.
There is further scope of improvement if we apply some algebra. We have to find a particular value that is the product of three numbers, i.e., B, C/100 and D. If such three numbers exists, then surely they are factors of that particular number.
We further optimize our calculation by following three steps:
– Find the factors of that particular number.
– Intersect the factors with the valid valuations of B, C and D.
– Now make combinations with the resultant after intersection.
For example, assume that particular value is 30. Multiplying it with 100 makes it as 3000. Its factors are:
1, 2, 3, 4, 5, 6, 8, 10, 12, 15, 20, 24, 25, 30, 40, 50, 60, 75, 100, 120 125, 150, 200, 250, 300, 375, 500, 600, 750, 1000, 1500, 3000 Its intersection with parameters results in:
B:
1, 2, 3, 4, 5, 6, 8, 10, 12, 15, 20, 24, 25, 30, 40, 50, 60, 75, 100 120, 125, 150, 200, 250, 300, 375, 500, 600, 750, 1000, 1500, 3000 C:
100, 120, 125, 150, 200, 250, 300, 375, 500, 600, 750, 1000, 1500, 3000 D:
2, 3, 4, 5, 6, 8, 10, 12, 15, 20, 24, 25, 30 resulting in 5824 comparisons in the worst case.
Everything works fine if three integers exist such that their product is equal to a particular value (as shown in Figure 5.8). However, there can be situations where
5.6. Tool Improvement 49
such a product of three integers does not exist (for example if the particular value turns out to be 31). In those cases we have to either increment or decrement that particular value and make comparison again. This is repeated until a combination is found. Whether to increment or decrement the particular value depends on what is the particular value:
– MinValue: Decrease – MinValue+1: Increase – MaxValue: Decrease – MaxValue+1: Increase
These values are incremented or decremented so that the test cases check the values within the partition or outside the partition as before, i.e., a test case at MinValue is expected to return a Failure and all other values less than MinValue is also expected to return Failure. If a combination of the parameters B, C and D do not result in MinValue, we try to find a combination that result in MinValue - 1 and so on.
The algorithm used for this version of the tool is presented in Algorithm 3:
5.6.1 Tools Combination
Tool Version 1 generates test cases such that the combination of parameters B, C and D lies within or outside MinValue and MaxValue for a particular (A, E) combination. Tool version 2 generates test cases such that two combination of parameters B, C and D (= MinValue and MaxValue + 1) lie outside the MinValue and MaxValue for a particular (A, E) combination and two combinations of pa-rameters B, C and D (= MinValue + 1 and MaxValue) lie within the MinValue and MaxValue.
In this way version 2 can be considered as a superset of version 1. It has ob-vious advantages of boundary value analysis, which is a critical attribute to be tested. The only attribute it lacks is randomness. However due to the mentioned advantages of randomness, we can cannot ignore version 1 completely.
Since test case generation is fully automatic, we can combine the two versions of the tool into one, giving rise to Tool version 3. The previous two versions of the tool would generate 205 and 405 test cases (for 100 valid combination of parameter A and E) and this version will generate 605 test cases.
50 Chapter 5. Data Dependency
Algorithm 3 Data Dependency Algorithm1 Input: Variables with their range
Input: Dependency in form of equations
1: procedure Generate Test Cases
2: for all Variables V do
3: ValidRange V ⇒ MinV ≤ ValidRange V ≤ MaxV
4: InValidRange V 6= ValidRange V
5: end for
6: Extract all values of parameters A and E from PRS
7: for all Combination C of A and E do
8: Extract minValue C from PRS
9: Extract maxValue C from PRS
10: FindCombination(minValue C)
17: Write Test Case for combination
18: end if
19: end while
20: FindCombination(minValue C = minValue C + 1)
21: while CombinationNotFound do
27: Write Test Case for combination
28: end if
37: Write Test Case for combination
38: end if
39: end while
40: FindCombination(maxValue C = maxValue + 1)
41: while CombinationNotFound do
42: FindCombination(maxValue C)
5.6. Tool Improvement 51
43: if CombinationNotFound then
44: MaxValue C = MaxValue + 1
45: else
46: CombinationNotFound = false
47: Write Test Case for combination
48: end if
49: end while
50: end for
51: . Adding Test Cases for last five cases in Decision table
52: for all Last 5 columns C of Decision Table do
53: for all Variables V do
54: if V against C == true then
60: Add Test Case with randomly selected values of variables
61: end for
62: end procedure
Figure 5.9: Test Cases: Randomized + Decision table + BVA
30
Combination of B, C and D
18000
Combination of B, C and D
The number of generated test cases can be considered high. However, for com-pletely testing 200 boundaries (100 combination of parameters A and E) and incor-porating the randomness feature, this is the minimum number. Also, regenerating 605 test cases happens within a minute, with a single click of a button and when executed with the help of Spec Explorer, the execution also happens within min-utes. The test cases for the example where MinValue is 30 and MaxValue is 18000 is shown in Figure 5.9.
5.6.2 Tool Functionality Enhancement
We have mentioned before that the values of parameters A and E vary for every configuration. For the configurations that already exists, the tool uses an Excel sheet, prepared by the verification team. However, the verification team has to
52 Chapter 5. Data Dependency
manually extract these data from the PRS of that configuration in the Microsoft Word format, which consumes almost a day and often is error prone. Therefore, we automated this activity in the current version of the tool, as every now and then new configurations pops up.
This gave rise to tool version 3.1, which accepts a new PRS in Word format, parses it and extracts the required information in an Excel sheet. This excel sheet can now be used to generate test cases.