Target MBT to Failing Test-Case

Meta-Information

Origin: Brian Nielsen / Aalborg University, completed by Wolfgang Herzner / AIT

Written: July 2014

Purpose: Generate additional related test cases in the same model neighborhood due to bug cluster assumption.

Context/Pre-Conditions: “MBT” is applied, perhaps also “MBT with Test Model Analysis”, and a test case caused the SUT to fail.

To consider: Nothing in addition to those already listed for pattern “MBT” and “MBT with Test Model Analysis”.

Structure

Participants and Important Artefacts

The same as in “MBT”.

Actions / Collaborations

Only those are described here that operate in addition to those described in “MBT”.

(1) If the SUT fails on a test case or shows suspect behaviour, the test case is abstracted to the test model level (if necessary) and forwarded to the test case generator.

(2) The test case generator generates new test cases in the „neighbourhood“ of that test returned from the verdict inspector. The neighbourhood may for example be defined as the set of paths found by choosing alternative outcomes at branching points in the original path [Peled, in FME 2001]. Alternatively, a small “trace distance” could be considered.

[Peled, in FME 2001] Natasha Sharygina, Doron Peled; A Combined Testing and Verification Approach for Software Reliability.

Proceedings of FME ’01 (International Symposium of Formal Methods for Increasing Software Productivity), pp.611-628

Discussion

Benefit:

– According to observations and the coupling theorem, faults are usually not equally distributed over a software, but clustered. Hence, with that method it is likely to find more bugs with rather little effort.

Limitation:

– It is of course possible, that it is an isolated fault. Then, this pattern causes redundant effort, but this additional effort is usually low (of course, execution of test cases may be expensive).

Comments:

– If another test case fails, the pattern can be applied again. Of course, it should then be avoided to retest an already generated test.

– If test cases are not generated from the test model, check that error trace /failing test is valid in the model.

Application Examples

MBAT Automotive Use Case 8 (Virtual Prototype Airbag ECU) / Scenarios 1 – 4:

Participants / Tools:

User: Verification engineer

T&A Model Engineer: The T&A model is constructed by the verification engineer

Test Objectives: Verification of the model or system under test. If faults are detected the location of the fault shall be automatically highlighted in the T&A model

Test Suite: The generated test suite generally containing long test cases is used as a stimuli for the SUT

SUT: model (Simulink) or system under test (HW)

Test Executor: The tests are executed and failing results are fed back to the TC generation tool, which will reduce the generated TCs in order to trace down the source of failing to a minimum set of model elements

Relations to other Patterns

Pattern Name	Relation*
MBT	MBT is complemented by this pattern
MBT with Test Model Analysis	Frame in which this pattern is usually applied

* “this pattern” denotes the pattern described here, “that pattern” denotes the related pattern