Towards Preserving Model Coverage and Structural Code Coverage

3.1. Structural Code-Coverage Criteria

Structural code-coverage criteria are metrics to analyze and quantify the control-flow coverage that is achieved for a given set of test data. Execution traces are used to collect the coverage information. In general, the satisfaction of a structural code-coverage criterion is not the primary test-case generation strategy in functional testing. Instead, structural code-coverage achieved during testing is analyzed as a supplementary measure to decide whether the implemented functionality has been sufficiently tested and does not contain any unintended functionality. However, there are also rare testing scenarios where the satisfaction of a certain code-coverage is the primary directive for test-data generation. For example, in measurement-based timing analysis an estimation of the worst-case execution time (WCET) is derived by systematic measurements [24].

In the following we review the properties of several structural code-coverage criteria.

Line Coverage. Is not a serious code coverage criterion, as without strict coding guidelines there is an ambiguous mapping from source lines to statements. In the extreme case one could write the whole program within one source line. Historically, line coverage was used as an easy hack when tools for analyzing statement coverage were missing. Thus, we do not discuss preservation of line coverage in this work.

Statement Coverage (SC). Requires that every statement of a program is executed at least once. Statement coverage alone is quite weak for functional testing [26] and should best be considered as a minimal requirement. Using our above definitions, we can formally define SC as follows:

(6)

Note that the boundary recognition of basic blocks can be tricky due to hidden control-flow. A statement in a high level language like ANSI C can consist of more than one basi cblock. For example, the ANSI C statement f=(a==3 && b==2); consists of multiple basic blocks due to the short-circuit evaluation order of ANSI C expressions.

Decision coverage (DC). Requires that each decision of a program has been tested at least once with each possible outcome. Decision coverage is also known as branch coverage or edge coverage. Decision coverage implies statement coverage:

(7)

Condition Coverage (CC). Requires that each condition of the program has been tested at least once with each possible outcome. It is important to mention that CC does not imply DC. A formal definition of CC is given in (8)

(8)

Note that our definition requires in case of short-circuit operators that each condition is really executed. This is achieved by the semantics of . However, often definitions are used that do not explicitly consider short-circuit operators (e.g., [27]), thus having in case of short-circuit operators only a "virtual" coverage since they do not guarantee that the short-circuit condition is really executed for the evaluation to TRUE as well as for the evaluation to FALSE.

Condition/Decision Coverage (CDC). Requires that both, condition coverage and decision coverage are achieved.

Modified Condition/Decision Coverage (MCDC). Requires to show that each condition can independently affect the outcome of the decision [12]. Thus, having conditions in a decision, test cases are required to achieve MCDC. Note that MCDC implies DC and CC. A formal definition of MCDC is given in (9) based on the set of input test data . It requires that for each condition of a decision there exists two test vectors such that the predicate symbol holds, which ensures that the two test vectors show different outcomes for as well as but the same outcomes for all other conditions within . This is exactly how MCDC is described above

(9)

(10)

The predicate symbol tests whether one of the test data , is a member of the input dataset and the other one a member of the input data set . If this predicate symbol is TRUE it is guaranteed that the expression evaluates to both, TRUE and FALSE:

(11)

The predicate symbol tests whether the input-data set provides a constant outcome for the evaluation of . Actually, the predicate symbol is used to test whether there exists a test-data subset for a given condition, such that the results of all other conditions remain unchanged. Thus, this predicate symbol is used to ensure that each condition can independently control the output of the decision:

(12)

The above definition of MCDC is the original definition given in the RTCA/DO178b document [12]. However, this definition is rather strict, so that people thought of some less restrictive definitions. For example, it is not possible with the original definition to cover a decision with strongly coupled conditions. (Two conditions are strongly coupled, iff they have the same input data partitioning for their satisfiability valuation, i.e., .) As described in [25], there exist at least three definitions of MCDC:

(i)Unique-Cause MCDC: this is the original definition given in [12].

(ii)Unique-Cause + Masking MCDC: this definition of MCDC is less restrictive as it requires in case of strongly coupled conditions to test only that one of them covers the decision (masking) [15].

(iii)Masking MCDC: this is less restrictive than the two above, as it does not require the Unique-Cause. A condition is masked if its value cannot influence the outcome of a decision due to the overruling values of other conditions. For Masking MCDC it is sufficient to show that each condition can affect the outcome of the decision without being masked. However, Masking MCDC is not required to test whether conditions do independently cover the decision. It focuses more on testing the correct implementation of subexpressions within a Boolean expression.

According to Chilenski the metric Masking MCDC should be the preferred form of MCDC as it provides the same error detection probability but allows for more different test sets and thus the generation of test data more is cost-effective [25].

Within this article we focus on the original definition of MCDC. Extending above formal definition of MCDC to Unique-Cause + Masking MCDC or Masking MCDC is straight-forward. One has to exchange the predicate by another predicate that formalizes the semantics of the alternative MCDC criterion.

Multiple Condition Coverage (MCC). Requires besides DC and CC that each possible combination of outcomes of the conditions of each decision is executed at least once. MCC demands a rather high number of test cases: to achieve full MCC of a decision with conditions tests are necessary. MCC is desired in theory, but MCC tends to be infeasible for industrial code, because there are too many conditions per decision [27]. Thus, in this work we do not address MCC.

Path Coverage (PC). Requires that each path of a program has been tested at least once. Since the number of paths within a program typically grows exponentially with the program size (PC is even stronger than MCC), we do not address PC.

Scoped Path Coverage (SPC). Is a coverage metric recently introduced by the authors. We use this type of code coverage for measurement-based timing analysis [7]. In this article we formalize this coverage metric to reason about necessary properties of a compilation profile for preserving SPC. The basic idea of SPC is to partition the program into program scopes and cover all possible paths within each program scope. Actually, PC is just the special case of using SPC in combination with only one program scope covering the whole program .

The appropriate partitioning of a program into program scopes depends on the concrete testing goal. For example, in case of our research on measurement-based timing analysis [7] we use the partitioning of the program into scopes to achieve a compromise between precision of measurement results (the larger the segments the more precise) and number of necessary measurements.

SPC requires that each path within a program scope is tested at least once. Thus, there must be a test datum that covers all basic blocks along the path. Using our above definitions, we can formally define SPC as follows:

(13)

Note, that the condition " " of (13) ensures that in the pathological case of having a program scope that is completely free of conditions, coverage of the only single path in the program scope is guaranteed.

Whether SPC is feasible in practice, depends on the program complexity itself and also on the application-specific partitioning of a program into program scopes.

Examples of test vectors sufficient for full coverage according to the different coverage metrics are given in Table 1. The ANSI C code example is a decision including three conditions. Note that in C the operators and && influence the control flow of the program due to the short-circuit evaluation in ANSI C. This small example is meant to support the definition of different variants of coverage metric. It is not meant to show the relative costs of the different variants of structural coverage metric.

Test vector			ANSI C expression	Coverage criterion
A	B	C	if(A || (B && C)	SC	DC	CC	MCDC	MCC	PC	SPC
F	F	F	F
F	F	T	F
F	T	F	F
F	T	T	T
T	F	F	T
T	F	T	T
T	T	F	T
T	T	T	T

The condition coverage (CC) needs a relative high number of test vectors. This is because of test vectors that enforce the entering of a program decision do not necessarily enforce the execution of a specific condition within the decision. Multiple condition coverage (MCC) has a relative high cost for testing a single decision. However, when looking at the whole program, then path coverage (PC) is typically much more complex, and depending on the definition of program scopes, scoped path coverage (SPC) requires significantly less test vectors than PC.

4.1. Preserving Statement Coverage (SC)

Equation (15) of Theorem 4.1 provides a coverage preservation criterion for statement coverage. Equation (15) essentially says that for each basic block of the transformed program there exists a basic block of the original program such that reaching with a given test vector implies that also is reached with the same test vector.

Theorem 4.1 (Preservation of SC).

Assuming that a set of test data achieves statement coverage on a given program , then (15) provides a sufficient—and without further knowledge about the program and the test data (there is now knowledge about the test data or the program assumed), also necessary—criterion for guaranteeing preservation of statement coverage on a transformed program .

(15)

Proof.

Preservation of SC: Part 1, showing sufficiency: Since is assumed to achieve SC on , it holds for each that . Since (15) states that it follows that for each we also have . Thus, SC is preserved at .

Part 2, showing necessity by indirect proof: Assuming there exists a basic block of such that for all basic blocks of it holds that , then each contains at least one input that is not in . If consists of exactly those inputs, then is never reached although SC holds in , which implies that SC is not preserved.

4.2. Preserving Condition Coverage (CC)

To define a coverage preservation criterion for CC (Theorem 4.2) we use the auxiliary predicate given in (16).

The predicate is only TRUE if the set of input data includes at least the true-satisfiability valuation or the false-satisfiability valuation of expression , where is either a condition or a decision. The predicate is used for the coverage preservation criterion of CC (and also DC) to test whether the evaluation of any expression of the original program to both, TRUE and FALSE, implies that the test data include at least one element of , needed for the coverage of an expression in the transformed program

(16)

Equation (17) states that for each condition of the transformed program there exists at least one condition of the original program whose coverage implies that evaluates to TRUE and there exists at least one condition of the original program whose coverage implies that evaluates to FALSE.

Theorem 4.2 (Preservation of CC).

Assuming that a set of test data achieves condition coverage on a given program , then (17) provides a sufficient—and without further knowledge about the program and the test data, also necessary—criterion for guaranteeing preservation of condition coverage on a transformed program :

(17)

Proof.

Preservation of CC: Part 1, showing sufficiency: Since is assumed to achieve CC on , it holds for each that and . Since (17) states that for each it holds that

(18)

it follows that for each we also have

(19)

Thus, CC is preserved at .

Part 2, showing necessity by indirect proof: Assuming there exists a condition of program such that for all conditions of program it either holds that

(a)

(b)

then it is possible that

(a) :

(b) :

which in both cases violates the preservation of CC.

Simplification of the CC Preservation Criteria

The goal of defining the coverage preservation criterion is to decide for a set of code transformations whether they could potentially disrupt the structural code coverage achieved on the original program. Typically, when checking the preservation of structural code coverage, one would simplify (17) by just checking whether each condition is kept equal or simply is inverted. This would result in the simpler criterion given in (20)

(20)

Working with the simple constraint of (20) may be sufficient in practice when analyzing the effect of concrete code transformations, since many transformations do not modify the conditions within a decision, but only their grouping into decisions. The simplified criterion is sufficient to allow only such code transformations that do not introduce new conditions with new unique satisfiability by the test data. Further, some transformations just invert a condition, which can be checked also with this simplified criterion.

4.3. Preserving Decision Coverage (DC)

To define a coverage preservation criterion for DC (Theorem 4.3) we use the auxiliary predicate given in (16), which is also used for preserving CC.

Equation (21) of Theorem 4.3 provides a coverage preservation criterion for decision coverage. Equation (21) essentially says that for each decision of the transformed program there exists at least one decision of the original program whose coverage implies that evaluates to TRUE and there exists at least one decision of the original program whose coverage implies that evaluates to FALSE.

Theorem 4.3 (Preservation of DC).

Assuming that a set of test data achieves decision coverage on a given program , then (21) provides a sufficient—and without further knowledge about the program and the test data, also necessary—criterion for guaranteeing preservation of decision coverage on a transformed program

(21)

Proof.

Preservation of DC: Part 1, showing sufficiency: since is assumed to achieve DC on , it holds for each that and . Since (21) states that for each

(1)

(2)

it follows that for each we also have and . Thus, DC is preserved at .

Part 2, showing necessity by indirect proof: assuming there exists a decision such that for all conditions it either holds that

(a)

(b)

then it is possible that

(a) or

(b)

which in both cases violates the preservation of DC.

Guaranteeing Decision Coverage

Guaranteeing the preservation of a structural code coverage criterion that depends on the coverage of decisions of a program is challenging, since there are many ways to re-group conditions into hierarchies of decisions without changing the program semantics.

The criterion given in (21) imposes quite strong restrictions on the performed code transformations, since it requires that for each decision there is an adequate decision of the original program such that decision coverage is preserved. For example, consider the following code transformation:

if   (a==3)   

if   (a==3 && b==2)    if   (b==2)   

c(); c();

inlined style

noninlined style

Such a transformation is quite typical when source-code is transformed into assembly code. Actually, the only decision in the original code is (a==3  &&  b==2). Having decision coverage on the original code, there are numerous code transformations possible that do not preserve decision coverage.

Thus, it would be useful to have another criterion to guarantee decision coverage at the transformed program. Equation (22) provides a sufficient criterion for guaranteeing decision coverage on the transformed program, assuming that condition coverage is fulfilled on the original program

(22)

The new criterion requires a different, but not stronger, structural code coverage at the original code to guarantee decision coverage at the transformed code. This criterion is typically more flexible when generating assembly code (which typically does not have control-flow statements with complex decisions). Further, in case that condition decision coverage (CDC) is fulfilled at the original program, one may chose between the criteria of (21) and (22) to guarantee decision coverage at the transformed program.

4.4. Preserving MCDC

Preserving MCDC coverage on a transformed program is especially challenging, since the code transformation may produce arbitrary groupings of conditions into decisions. Especially the requirement that each condition can independently influence the outcome of its conditions, is rather complex to check.

As the MCDC coverage preservation criterion is rather complex, we derive them in two steps. First, we describe a rather naive criterion that is relatively ease to understand. This criterion is sufficient but not necessary (too strict). Second, we describe a "realistic" (more detailed) criterion that is sufficient and necessary.

A Naive Coverage Preservation Criterion

A sufficient but not necessary coverage preservation criterion for MCDC is given in (23). The predicate symbol is used in the same way as the real criterion: it is used to express that only input data that fulfill MCDC at the original program have to be considered for coverage preservation

(23)

This naive criterion is not necessary since it requires the coverage preservation of the conditions in the transformed program by a single condition from the original program .

Another drawback of this naive criterion is that it is based on a concrete set of test data that are used to achieve MCDC at the original program. To ensure coverage preservation in general, it would be necessary to ensure that the criterion holds for all possible sets of test data that achieve MCDC at the original program, which tends to be intractable in practice.

A Realistic Coverage Preservation Criterion

To define an easier testable (but more complicated) coverage preservation criterion for MCDC (Theorem 4.4) we use the auxiliary predicate given in (24). The predicate is similar to the predicate symbol , with the difference that it performs the control check on all members of two sets of input data. The predicate is used for the coverage preservation of MCDC to test whether the condition of the original program refers to TRUE for one input data set or and refers to FALSE for the other. Besides , also the predicate (10) is used to describe the preservation criterion for MCDC coverage

(24)

The criterion given in equ_preserve_mcdc states that for each condition of a decision of the transformed program there exist two sets of input data and whose members achieve the criterion needed for MCDC coverage. Further, there has to be a condition of the original program such that the is a subset of either the true-satisfiability valuation or the false-satisfiability valuation (tested with the predicate ). the same requirement as .

Theorem 4.4 (Preservation of MCDC).

Assuming that a set of test data achieves MCDC coverage on a given program , then (25) provides a sufficient—and without further knowledge about the program and the test data, also necessary—criterion for guaranteeing preservation of MCDC coverage on a transformed program

(25)

Proof.

Preservation of MCDC: Part 1, showing sufficiency: Since is assumed to achieve MCDC on , it holds for each and for each that there exist at least two test vectors such that . Since as defined in (10) for each condition is the formal definition of MCDC it directly follows that

(26)

is a sufficient criterion to ensure that MCDC is preserved at program .

Part 2, showing necessity by indirect proof: Assuming there exists a decision with a condition such that for all input-data subsets it either holds that

(27)

then it is possible that

(28)

(for all conditions in the original program condition coverage is not fulfilled; this case is already excluded by assumption of having MCDC coverage at )

(29)

(there is no MCDC coverage at the original program ; this case is already excluded by assumption of having MCDC coverage at )

(30)

(the test data do not provide MCDC coverage at the transformed program ) which in each case violates the preservation of MCDC: Case (a) and (b) violate the preservation of MCDC since they are in contradiction with the requirement that MCDC is achieved at the original program. Case (c) states that there exists a condition in the transformed program for which there are no test data to achieve unique cause coverage, which is required for MCDC.

4.5. Preserving Scoped Path Coverage (SPC)

To define a coverage preservation criterion for SPC (Theorem 4.5) we use the auxiliary predicate given in (31).

The predicate is only TRUE if there is at least one condition from the set of conditions whose true-satisfiability valuation is a subset of the input data or there is at least one condition from the set of conditions whose false-satisfiability valuation is a subset of the input data . The predicate is used for the coverage preservation criterion of SPC to test whether for a condition in the transformed program with true/false-satisfiability valuation there exist two conditions in the original program whose true/false coverage are a subset of

(31)

As stated in Theorem 4.5, (32) provides a coverage preservation criterion for SPC. Equation (32) says that for each scoped path of the transformed program there exists a scoped path such that the reachability of the first basic block of implies the reachability of the first basic block of . Further, Equation (32) states that for each condition of that has to be evaluated to TRUE, there exists a condition of a scoped path in the original program that will imply the True evaluation of (by predicate ). Finally, Equation (32) states that for each condition of that has to be evaluated to FALSE, there exists a condition of a scoped path in the original program that will imply the FALSE evaluation of (by predicate ).

Theorem 4.5 (Preservation of SPC).

Assuming that a set of test data achieves scoped path coverage on a given program , then (32) provides a sufficient—and without further knowledge about the program and the test data, also necessary—criterion for guaranteeing preservation of scoped path coverage on a transformed program

(32)

Proof.

Preservation of SPC: Part 1, showing sufficiency: Since is assumed to achieve SPC on , it holds for each and each with that there exists test data with

(33)

Since (32) states that

(34)

it follows that

(35)

As (32) also states that

(36)

it follows that

(37)

Finally, as (32) states that

(38)

it follows that

(39)

Thus, SPC is preserved at .

Part 2, showing necessity by indirect proof: Assuming there exists a scoped program path such that for all scoped program paths it either holds that

(40)

then at least one of the following cases is possible:

(41)

(the first basic block of a scoped program path of the transformed program is not executed with the given test data )

(42)

(one of the conditions of a scoped program path of the transformed program that has to be evaluated to True evaluates to False for all test vectors of )

(43)

(one of the conditions of a scoped program path of the transformed program that has to be evaluated to False evaluates to True for all test vectors of ) which in each case violates the preservation of SPC.

5.1. Transformations That Change Reachability

In this section we will demonstrate how to identify code transformations that can disrupt structural code coverage by changing the reachability of program elements. As a case study, we have chosen condition reordering, a common code optimization that is typically used to enable other code optimizations. As shown in Figure 4, condition reordering changes the order of conditions within a decision.

The result of applying our formalism to the original and the transformed program is shown in Figure 5. As a major result we see that the element is surrounded by a box, which means that this coverage is not preserved during optimization. This means that the second condition at line 1 of the transformed program (a > 5) is not guaranteed to be evaluated to FALSE.

The coverage-relation graph in Figure 5 contains coverage types for statements, conditions, and decisions. From this graph we can conclude which structural code-coverage criteria are preserved by condition reordering. For example, missing the coverage of but not of implies that SC is still preserved. Also DC is preserved. However, coverage criteria that are based on the coverage of conditions are not guaranteed to be preserved. For example, preservation of CC or MCDC is not guaranteed. For simplicity, we do not discuss preservation of SPC, because analyzing SPC preservation would make it necessary to unroll the program to make the different scoped program paths explicit.

5.2. Transformations That Add New Paths

In this section we will demonstrate how to identify code transformations that can disrupt structural code coverage by adding new control-flow paths to the program. As a case study, we have chosen loop inversion, a common code optimization that transforms a while-loop into a do-while-loop, as shown in Figure 6.

Loop inversion creates new control-flow paths. As shown in Figure 7, coverage preservation cannot be established for the exit test of the transformed loop: for the decision and conditions of source line 5 the TRUE/FALSE evaluation is not preserved. Again, this has no impact on the preservation of statement coverage. But coverage preservation of CC, DC, or MCDC is not guaranteed.

5.3. Transformations That Preserve Coverage

In this section we will demonstrate how to identify code transformations that preserve structural code-coverage. As a case study, we have chosen loop reversal, a typical code optimization enable further loop optimizations or vectorization of computations. As shown in Figure 8, loop reversal changes the condition in a program, but as we will see, it still preserves structural code-coverage.

Concluding from Figure 9, coverage preservation is achieved for all conditions, decisions, and basic blocks of the program. Thus, coverage of SC, CC, DC, and MCDC is preserved by loop reversal.

As a general pattern, if for each relevant expression of the transformed program there is a line or arrow to that expression originating from any expression of the same type in the original program, then the code coverage is preserved.

5.4. Modeling Hidden Control Flow

Some programming languages include statements that provide hidden control flow. For example, the logical operators && and of ANSI C/C++ have a short-circuit evaluation semantics, which in fact already is conditional control flow. The short-circuit evaluation of ANSI C/C++ states that the second argument of the operators && and is not evaluated if the final result of the operator is already determined by the first argument.

For example, lets look at the following code:

   if   (a &&  (b  ∣ ∣  c)  

Above code might be translated into the following assembly code, which demonstrates how the short-circuit evaluation of ANSI C/C++ works:

   if   (!a)  goto skip;

   if   ( b)  goto process;

   if   (!c)  goto skip;

process:

skip:

If we do not explicitly address the hidden control-flow with the coverage preservation framework, we risk to loose full structural code-coverage as soon as the program is transformed into a new program with explicit control flow instead of the hidden, implicit control-flow.

5.5. Towards Automatic Calculation of Coverage Profiles

In this section we have demonstrated that the formalism provided in this article is helpful for determining the coverage profile of a code transformation. However, in Section 4 we argue that the calculation of the coverage profile could be done automatically, given the formal coverage preservation criterion and a formal description of the code transformation.

The code transformations can be formalized in an axiomatic semantics [28, 29], that is, by providing preconditions, postconditions, and invariants of the code transformation. Even model checking can be used to analyze code transformations [30]. There exist specific frameworks to model code transformations, for example OPTIMIX [31]. Such frameworks seem to be well-suited as a starting point for future research on automating the calculation the coverage profile of a code transformation.

6.1. A Simple Example

The Switch element of MATLAB/Simulink is used to control the data flow. A simple model using that element is given in Figure 10.

The semantics of the Switch element is best described by the code given in Figure 11 that is generated out of the simple model.

The code shows a single decision with one condition inside. In this concrete example it is valid to define the scope of decision coverage (or MCDC) as the single MATLAB/Simulink element. The generated source code for this element contains the whole decision, thus generating test data for decision coverage (or MCDC) based on the hidden control-flow of the Switch element results in a full decision coverage (or MCDC) at source code level.

Coverage criteria that are not based on the decisions, are easier to preserve, since coverage of conditions or basic blocks can be directly derived from the coverage of a single Matlab/Simulink element. Thus, structural code coverage like SC, CC, or SPC can be directly preserved based on the hidden control-flow.

In the following we show that it is not always that easy to apply decision-based structural code-coverage criteria like DC or MCDC to the Matlab/Simulink language and to ensure their preservation.

6.2. Identifying Code Structures in Models

In Section 6.1 we have seen how to use the implicit control flow of a data-flow element to apply a structural code-coverage metric. Given that a code generator creates explicit control flow, for example, as C code, there is the question whether it is valid to analyze Matlab/Simulink elements in isolation because the code generator can combine the logic of several data-flow elements into a single C statement. Looking only at the Matlab/Simulink elements without considering that the concrete behavior of the code generator, does not allow, for example, to identify the set of conditions that will be grouped into a single decision.

To give an example, we use two logical operators as shown in Figure 12. Basically, the logical or as well as the logical and form a condition. But the choice of how to map these conditions into decisions in the generated code is left to the code generator. For example, in the non-inlined implementation variant of this model we get two decisions:

expr  =  (In2 && In3);  / decision 1 /

Out1 = (In2 && In3);  / decision 2 /

The decisions in this code example are the assignments of a logical expression to a program variable. This is just a compact form instead of using an if/then construct. Now lets look at the inlined implementation variant of this model, which is generated by default by the code generator Real-Time Workshop of Matlab 6.5.1 and Simulink 5.1. In the inlined implementation variant we get a single decision:

Out1  =  (In1 (In2 && In3);

Actually, achieving structural code coverage on the non-inlined implementation variant requires in general a smaller number of test data. For example, Rajan et al. have shown that full MCDC coverage at the non-inlined implementation variant yielded only about 13.6% MCDC coverage at the inlined implementation variant [20].

Without the knowledge about the behavior of the code generator it is not possible to formulate a preservation criterion for decision coverage (DC) or MCDC coverage that is both, sufficient and necessary independently of the exploited implementation variant. Maybe more surprisingly, this problem also arises for condition coverage (CC) because given that = (In2  &&  In3), the chosen implementation variant in this concrete example influences the set (where the condition is reached with an evaluation to True). The problem also arises for statement coverage (SC) because inlining reduces the reachability valuation of (In2  &&  In3).

6.3. The Complexity of Model Coverage

In Section 6.2 we raised the question of how much model elements have to be considered together to identify the scopes of structural code-coverage criteria. Now we argue that this scoping problem can also arise within a single element due to the rather complex code that may be generated for a Matlab/Simulink element.

For example, consider the lookup table used in the model given Figure 13. The lookup table in this example approximates a function by using a vector for some concrete data of the x-axis of a function (Look_Up_Table_XData) and a vector for the corresponding data of the y-axis of the function (Look_Up_Table_YData). There are different ways of how to map with help of the look-up table an arbitrary input value of the approximated function to the corresponding function value. With Matlab 6.5.1 and Simulink 5.1 the code generator Real-Time Workshop offers the following lookup policies: interpolation-extrapolation, interpolation-use end values, use input nearest, use input below, and use input above. Using the lookup policy use input nearest we get the implementation shown in Figure 14 of the lookup table using binary search. This implementation uses the library function BINARYSEARCH_real_T_Near_iL given in Figure 15.

Without plotting the implementations for the other look-up policies, one can see that the Look-Up Table element results in rather complex control flow.

To conclude, specifying structural code-coverage criteria at the model-level that are both, sufficient and necessary, is not possible without knowledge about the behavior of the code generator.