Target MBT to Failing Test-Case
Written: July 2014
Purpose: Generate additional related test cases in the same model neighborhood due to bug cluster assumption.
Context/Pre-Conditions: “MBT” is applied, perhaps also “MBT with Test Model Analysis”, and a test case caused the SUT to fail.
To consider: Nothing in addition to those already listed for pattern “MBT” and “MBT with Test Model Analysis”.
Participants and Important Artefacts
The same as in “MBT”.
Actions / Collaborations
Only those are described here that operate in addition to those described in “MBT”.
(1) If the SUT fails on a test case or shows suspect behaviour, the test case is abstracted to the test model level (if necessary) and forwarded to the test case generator.
(2) The test case generator generates new test cases in the „neighbourhood“ of that test returned from the verdict inspector. The neighbourhood may for example be defined as the set of paths found by choosing alternative outcomes at branching points in the original path [Peled, in FME 2001]. Alternatively, a small “trace distance” could be considered.
[Peled, in FME 2001] Natasha Sharygina, Doron Peled; A Combined Testing and Verification Approach for Software Reliability.
Proceedings of FME ’01 (International Symposium of Formal Methods for Increasing Software Productivity), pp.611-628
Springer-Verlag London, UK ©2001; ISBN:3-540-41791-5
– According to observations and the coupling theorem, faults are usually not equally distributed over a software, but clustered. Hence, with that method it is likely to find more bugs with rather little effort.
– It is of course possible, that it is an isolated fault. Then, this pattern causes redundant effort, but this additional effort is usually low (of course, execution of test cases may be expensive).
– If another test case fails, the pattern can be applied again. Of course, it should then be avoided to retest an already generated test.
– If test cases are not generated from the test model, check that error trace /failing test is valid in the model.
MBAT Automotive Use Case 8 (Virtual Prototype Airbag ECU) / Scenarios 1 – 4:
Participants / Tools:
User: Verification engineer
T&A Model Engineer: The T&A model is constructed by the verification engineer
Test Objectives: Verification of the model or system under test. If faults are detected the location of the fault shall be automatically highlighted in the T&A model
Test Suite: The generated test suite generally containing long test cases is used as a stimuli for the SUT
SUT: model (Simulink) or system under test (HW)
Test Executor: The tests are executed and failing results are fed back to the TC generation tool, which will reduce the generated TCs in order to trace down the source of failing to a minimum set of model elements
Relations to other Patterns
|MBT||MBT is complemented by this pattern|
|MBT with Test Model Analysis||Frame in which this pattern is usually applied|
* “this pattern” denotes the pattern described here, “that pattern” denotes the related pattern