Origin: Eike Möhlmann / OFFIS
Written: March 2019
Purpose: Estimation of the risk that a given SUT* violates a given catalog of requirements.
To consider: The quality of the estimation depends on the quality of the used simulation environment and the abstract scenarios catalog.
Participants and Important Artefacts
Test Field: Recorded/Measured data from the real world in the deployment context of the SUT.
Abstract Scenarios DB: A catalog of abstract scenarios which shall be qualified to be a proxy for the real world, e.g. left turns on two lane city road intersection.
Set of Synthetic Scenarios: A set of concrete scenarios that has been derived by randomly instantiating abstract scenarios.
Requirements DB: A set of requirements under which the abstract scenario catalog will be compared to the real world scenarios.
Criticality Indicators DB: A set of indicators and causes for requirement violations (like accidents or near-accident situations). These indicators can be used to improve scenario selection and simulation guiding during the (semi-)virtual testing.
Estimated Requirement Satisfaction: Estimated probability that a (random) (semi-)virtual run (of the system-under-test in the virtual world) will satisfy the requirements.
Generate Synthetic Scenarios: Generate a set of synthetic scenarios by randomly (i.i.d.*) instantiating abstract scenarios from the abstract scenario catalog.
Setup of Physical Components: Setup of the physical components that are required for executing the test cases corresponding to the set of synthetic scenarios.
Setup of Virtual Components: Setup of the physical components that are required for executing the test cases corresponding to the set of synthetic scenarios.
Guided Simulation with Requirement Monitoring: Execute the test cases corresponding to the set of synthetic scenarios and observe the satisfaction of the requirements to estimate the probability that the SUT violates the requirements in a random scenario (i.i.d.).
* independently and identically distributed
– The (semi-) virtual testing allows testing of components in isolation with known-to-work reference components as well as reduces the effort for setup and execution of the test.
– The (semi-) virtual testing allows testing of critical scenarios in which human and property are at risk.
Comment: As this is a hypothesis-test the result is influenced by parameters characterizing the statistical power of the test (sensitivity, significance).
The pattern was applied for estimating the risk that the SUT (an autonomous driving function) will collide with pedestrians in following use cases:
ENABLE-S3 UC2 “Intersection Crossing using ACPS”
ENABLE-S3 UC6 “Valet Parking”:
Relations to other Patterns
|Scenario-based V&V Process
|This pattern uses the scenarios catalog developed with that pattern
* “this pattern” denotes the pattern described here, “that pattern” denotes the related pattern