The Boundary Value Analysis is a popular test technique. It is easy to understand and easy to explain to the testers and the non-testers alike. Part of the ease in comprehension is based on the elegant and simple examples that can be used to explain the technique. A boundary value is introduced, and it’s explained that the three values are needed to test the boundaries thoroughly. But this article tries to show that three values are not enough to perform a black box test to prove that a boundary has been implemented correctly.
First of all let’s find out what is meant by Equivalence Partitioning (EP). The basic idea of EP is quite simple. If you expect the same result from two tests, it’s sufficient to run one of them. But this idea is based on two assumptions.
- Every test input can be divided into several classes.
- System processes every possible input value in a class exactly the same way as it process every other input value in the class.
(Note that test outputs also can be considered the same as above)
For an example: A bank system accepts clients aged 55 years old or younger. All other potential clients are not accepted. Ages are entered into the system under test in whole years.
Considering above example, we can divide two possible classes as follows:
- Clients aged 55 and younger will be accepted (we call this class as valid class)
- Clients aged 56 and older will be rejected (we call this class as invalid class)
When using EP for above example we can test the system with following test values.
- 15 to represent valid class
- 75 to represent invalid class
Note that the above values are chosen arbitrarily due to the assumptions we make on EP technique. Also the values 20 and 80, or 40 and 65 could have been chosen.
Let’s find out what is Boundary Value Analysis (BVA). While testing, it was discovered that some defects seemed to cluster around values outline the equivalence classes (boundaries). The boundaries are being incorporated into programming code during programming. In the example above, focusing on the “younger than or equal to 55 years old” boundary, code can say as “IF age <= 55”.
Since this boundary is code, this is where programming error most likely could occur. In Boundary Value Analysis (BVA) test technique this means to test with test values on, just below, and just above the boundary to detect any possible programming fault made in rational operator (<, <=, =, >, >=, <>). The defect could exist regarding the rational operator programming faults in the <= sign. This operator could wrongly be programmed as >, >=, = , <, <> or = instead of <=. Let’s check whether operator fault can be found using three values considering BVA.
| Possible Fault
Instead of <= 55 |
Test Value | Actual Result | Expected Result | Fault detected
per value |
Fault detected
using three values |
| >55 | 54 | Reject | Accept | Yes | Yes |
| 55 | Reject | Accept | Yes | ||
| 56 | Accept | Reject | Yes | ||
| >=55 | 54 | Reject | Accept | Yes | Yes |
| 55 | Accept | Accept | No | ||
| 56 | Accept | Reject | Yes | ||
| < 55 | 54 | Accept | Accept | No | Yes |
| 55 | Reject | Accept | Yes | ||
| 56 | Reject | Reject | No | ||
| =55 | 54 | Reject | Accept | Yes | Yes |
| 55 | Accept | Accept | No | ||
| 56 | Reject | Reject | No | ||
| <>55 | 54 | Accept | Accept | No | Yes |
| 55 | Reject | Accept | Yes | ||
| 56 | Accept | Reject | Yes |
In Summary, the use of BVA comes down as follow:
- Identify equivalence classes.
- Determine Boundary Values.
- Specify test cases using three values for each boundary.
But BVA is often used in black box testing and test professional has no insight into the implementation of the functional design above example also can be coded as “IF age < 56”. If so, let’s find out same three values enough to detect the fault.
| Possible Fault
Instead of < 56 |
Test Value | Actual Result | Expected Result | Fault detected
per value |
Fault detected
using three values |
| >56 | 54 | Reject | Accept | Yes | Yes |
| 55 | Reject | Accept | Yes | ||
| 56 | Reject | Reject | No | ||
| >=56 | 54 | Reject | Accept | Yes | Yes |
| 55 | Reject | Accept | Yes | ||
| 56 | Accept | Reject | Yes | ||
| <= 56 | 54 | Accept | Accept | No | Yes |
| 55 | Accept | Accept | No | ||
| 56 | Accept | Reject | Yes | ||
| =56 | 54 | Reject | Accept | Yes | Yes |
| 55 | Reject | Accept | Yes | ||
| 56 | Accept | Reject | Yes | ||
| <>56 | 54 | Accept | Accept | No | No |
| 55 | Accept | Accept | No | ||
| 56 | Reject | Reject | No |
As shown in the table above, if “<> 56” was (wrongly) programmed instead of the intended “< 56”, the actual results and the expected results are identical for all three BVA test cases. Therefore, the fault will not be detected for the black box testing.
As a remedy, we can follow the following steps to find out another suitable test value to detect the fault.
Identify the equivalence classes.
According to the example,
IF Age <= 55 is valid class. (Age < 56)
IF Age > 55 is invalid class.
Get closest value for the boundary value from each valid and invalid class.
From Valid EP (55)
From Invalid EP (56)
Get the next nearest value for selected values in step#2.
From Valid EP (54)
From Invalid EP(57)
Specify test cases, using four test values per boundary.
Let’s apply identified test values for the example.
| Possible Fault
Instead of < 56 |
Test Value | Actual Result | Expected Result | Fault detected
per value |
Fault detected
using three values |
| >56 | 54 | Reject | Accept | Yes | Yes |
| 55 | Reject | Accept | Yes | ||
| 56 | Reject | Reject | No | ||
| 57 | Accept | Reject | Yes | ||
| >=56 | 54 | Reject | Accept | Yes | Yes |
| 55 | Reject | Accept | Yes | ||
| 56 | Accept | Reject | Yes | ||
| 57 | Accept | Reject | Yes | ||
| <= 56 | 54 | Accept | Accept | No | Yes |
| 55 | Accept | Accept | No | ||
| 56 | Accept | Reject | Yes | ||
| 57 | Reject | Reject | No | ||
| =56 | 54 | Reject | Accept | Yes | Yes |
| 55 | Reject | Accept | Yes | ||
| 56 | Accept | Reject | Yes | ||
| 57 | Reject | Reject | No | ||
| <>56 | 54 | Accept | Accept | No | Yes |
| 55 | Accept | Accept | No | ||
| 56 | Reject | Reject | No | ||
| 57 | Accept | Reject | Yes |
By adding the extra test value (57) the fault (“<>” being programmed instead of “<”) will, even when using Boundary Value analysis as a black box technique, be found.
Summary & Conclusion:
When we perform BB testing using EP and BVA techniques, we are use only the three test values and I have proved that we could have miss some failures. To avoid and increase the probability of error detection, we could have used the four test values, which are taken from each valid EP and invalid EP closest to the boundary value.
Rule of Thumb:
Four test values are chosen:
- Two values from the valid EP, which are closest to the boundary value,
- Two values from the invalid EP, which are closest to the
boundary value,
