Find Reduction Rules for Model-Based Testing
Meta-Information
Origin: Gianpaolo Massaroli-Luigi Velardi / ASTS, Wolfgang Herzner / AIT
Written: November 2014
Purpose: Testing the fulfilment of requirements can make necessary to consider a lot of situations under which requirements need to be fulfilled. These situations correspond to combinations of input parameters values (stimuli).
This pattern helps to reduce the number of generated test cases by avoiding stimuli for input parameters which, according to requirements, don‘t affect the output ones.
The MBA is used for assuring the correctness of these assumed reduction rules.
Context/Pre-Conditions: The requirements themselves and a model which allows to understand and detect the correct flows/relationship among input and output parameters, are available or can be generated.
To consider: Expert knowledge is needed for identifying relationships between input/output parameters in each set of requirements. In addition, knowledge about MBA is needed. In worst case, the benefits of reducing number of test cases are annihilated by the efforts to make and analyse the dependency model.
Structure
Participants and Important Artefacts
A&T Model Engineer: Person who analyses the requ.s and creates the I/O-Requ.s Relation Model that represents them in a formalised way.
Reduction Rules Developer: Person with a significant expertise on the domain who analyses the IO-relation model in order to identify rules that allow to avoid irrelevant stimuli for test cases.
Test Case Generator: Tool that allows generating test scripts starting from high-level system description.
Reduction Rules Analyzer: Tool that allows checking correctness of identifed rules (perhaps under inclusion of model checking).
Test Executer: Tool(s) that is/are used to apply test cases.
.
Actions / Collaborations
(1) The A&T Model Engineer develops the I/O-Requ.s Relation Model by specifying the relations between input stimuli, output parameters and requirements. An example how this can be done is shown in the following.
|
– In order to test requirements Req2 and Req3 in figure, we have to consider all the possible combinations of the inputs (i3, i4 and i5) involved into these requirements both with all the possible values for the other inputs (up to i7) which contribute to the definition of the output o1. – On the contrary, as far as concerns the other inputs (i8, i9, etc.) generating the outputs o2 and o3, if the system has been correctly designed, their behavior will not impact on the generation of the output o1. – This consideration allows discarding specific combinations of these inputs, which can be set to a given value for the generation of tests, saving hence time and costs during the test execution phase. |
 |
(2) With the IO/requ.-mapping model developed in previous step, the reduction rules are identified. An initial procedure how this can be done is as follows:
(2.1) For each requirement R: identify parameter set A that influences it directly, and the parameter set B that influences other requirements in same block as R.
(2.2) Combine value sets for A with all value sets of B.
(2.3) For remaining parameters select default values.
(3) In order to verify the correctness of the Reduction Rules, an equivalent model checking technique is adopted. This technique consists in the following steps:
(3.1) Duplication of the part of the model under analysis with its relative inputs.
(3.2) Restriction to a fixed value for each input of the cloned part which is supposed to be not influencing the output under test (e.g.: i8 in the figure above).
(3.2) Check of the equivalence between the outputs of the two models on varying the free inputs.
An example for this equivalent model checking technique is represented in figure on next slide. It is taken from the Ansaldo STS use case described below, but can be considered as representative for this step.
Discussion
Implications:
– The main benefit of the application of the reduction rules to the testing execution phase is a – potentially significant – reduction of testing effort.
– This benefit may, however, be diminished in cases where each input affects most of the requirements.
– The main drawback of the application of the reduction rules to the testing execution phase is the fact that the hypothesis of input/output independence has to be verified (by means of static analysis) to ensure the correctness of the reduction of the number of tests.
Comment:
– A significant expertise on the domain is required because these rules can be derived also from particular characteristics of the domain itself!
.
Application Example
MBAT Rail Use Case 2 (Rapid Transit Metro System ) – Ansaldo STS / Scenarios 4.1c, 4.2 (Reduction Rules) and 2.2b (Analysis on Model):
Participants / Tools:
A&T Model Engineer: The system functional model is created from the system requirements (Out Of Scope of MBAT project).
Reduction Rules Developer: The Reduction Rules are elaborated before the ATG phase deriving from the specifications themselves and/or from one’s own domain characteristics (Out Of Scope of MBAT project).
Test Case Generator: Test scripts are generated starting from high-level system description (Out Of Scope of MBAT project).
Reduction Rules Analyzer: ALES Formal Specs Verifier (FSV)
Model-checker: FSV
Test Executor: FSV (code lines)
Comment:
– In order to verify the correctness of the Reduction Rules, an equivalent model checking technique is adopted (see action 3).
.
Relations to other Patterns
| Pattern Name | Relation* |
| MBT | This pattern complements (extends) MBT |
| Use Test Run to Slice Model | “Use Test Run to Slice Model” helps to cope with models too large to be completely analysed; similarly, this pattern can help to identify partial models that ease TCG |
* “this pattern” denotes the pattern described here, “that pattern” denotes the related pattern