TY - GEN
T1 - On MC/DC and implementation structure
T2 - 2008 IEEE/AIAA 27th Digital Avionics Systems Conference, DASC 2008
AU - Heimdahl, Mats P.E.
AU - Whalen, Michael W.
AU - Rajan, Ajitha
PY - 2008
Y1 - 2008
N2 - In civil avionics, obtaining D0-178B certification for highly critical airborne software requires that the adequacy of the code testing effort be measured using a structural coverage criterion known as Modified Condition and Decision Coverage (MC/DC). We hypothesized that the effectiveness of the MC/DC metric is highly sensitive to the structure of the implementation and can therefore be problematic as a test adequacy criterion. We tested this hypothesis by evaluating the fault-finding ability of MC/DC-adequate test suites on five industrial systems (flight guidance and display management). For each system, we created two versions of the implementations-implementations with and without expression folding (i.e., inlining). We found that for all five examples, the effectiveness of the test suites was highly sensitive to the structure of the implementation they were designed to cover. MC/DC test suites adequate on an inlined implementation have greater fault finding ability than test suites generated to be MC/DC adequate on the non-inlined version of the same implementation at the 5% significance level. (The inlined test suites outperformed the non-inlined test suites in the range of 10% to 5940%.) This observation confirms our suspicion that MC/DC used as a test adequacy metric is highly sensitive to structural changes in the implementation, and that test suite adequacy measurement using the MC/DC metric will be better served if done over the inlined implementation.
AB - In civil avionics, obtaining D0-178B certification for highly critical airborne software requires that the adequacy of the code testing effort be measured using a structural coverage criterion known as Modified Condition and Decision Coverage (MC/DC). We hypothesized that the effectiveness of the MC/DC metric is highly sensitive to the structure of the implementation and can therefore be problematic as a test adequacy criterion. We tested this hypothesis by evaluating the fault-finding ability of MC/DC-adequate test suites on five industrial systems (flight guidance and display management). For each system, we created two versions of the implementations-implementations with and without expression folding (i.e., inlining). We found that for all five examples, the effectiveness of the test suites was highly sensitive to the structure of the implementation they were designed to cover. MC/DC test suites adequate on an inlined implementation have greater fault finding ability than test suites generated to be MC/DC adequate on the non-inlined version of the same implementation at the 5% significance level. (The inlined test suites outperformed the non-inlined test suites in the range of 10% to 5940%.) This observation confirms our suspicion that MC/DC used as a test adequacy metric is highly sensitive to structural changes in the implementation, and that test suite adequacy measurement using the MC/DC metric will be better served if done over the inlined implementation.
UR - http://www.scopus.com/inward/record.url?scp=67249107592&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=67249107592&partnerID=8YFLogxK
U2 - 10.1109/DASC.2008.4702848
DO - 10.1109/DASC.2008.4702848
M3 - Conference contribution
AN - SCOPUS:67249107592
SN - 9781424422081
T3 - AIAA/IEEE Digital Avionics Systems Conference - Proceedings
SP - 5B31-5B313
BT - 2008 IEEE/AIAA 27th Digital Avionics Systems Conference, DASC 2008
Y2 - 26 October 2008 through 30 October 2008
ER -