On the effect of test-suite reduction on automatically generated model-based tests

Mats P.E. Heimdahl, Devaraj George

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

Model checking techniques can be successfully employed as a test-case generation technique to generate tests from formal models. The number of tests-cases produced, however, is typically large for complex coverage criteria such as MC/DC. Test-suite reduction can provide us with a smaller set of test-cases that preserve the original coverage-often a dramatically smaller set. Nevertheless, one potential drawback with test-suite reduction is that this might affect the quality of the test-suite in terms of fault finding. Previous empirical studies provide conflicting evidence on this issue. To further investigate the problem and determine its effect when testing implementations derived from formal models of software we performed an experiment using a large case example of a Flight Guidance System, generated reduced test-suites for a variety of structural coverage criteria while preserving coverage, and recorded their fault finding effectiveness. Our results indicate that the size of the specification based test-suites can be dramatically reduced and that the fault detection of the reduced test-suites is adversely affected. In this report we describe our experiment, analyze the results, and discuss the implications for testing based on formal specifications.

Original languageEnglish (US)
Pages (from-to)37-57
Number of pages21
JournalAutomated Software Engineering
Volume14
Issue number1
DOIs
StatePublished - Mar 1 2007

Keywords

  • Automated test generation
  • Fault finding
  • Model checkers
  • Specification-based testing
  • Test reduction

Fingerprint Dive into the research topics of 'On the effect of test-suite reduction on automatically generated model-based tests'. Together they form a unique fingerprint.

Cite this