Covert and Phantom Features in Annotations: Do They Impact Variability Analysis?

The annotation-based variability of the C preprocessor (CPP) has a bad reputation regarding comprehensibility and maintainability of software systems, but is widely adopted in practice. To assess the complexity of such systems’ variability, several analysis techniques and metrics have been proposed in scientific communities. While most metrics seem reasonable at first glance, they do not generalize over all possible usages of C preprocessor variability that appear in practice. Consequently, some analyses may neglect the actual complexity of variability in these systems and may not properly reflect the real situation. In this paper, we investigate two types of variation points, namely negating and #else directives, to which we refer to as corner cases, as they are seldom explicitly considered in research. To investigate these directives, we rely on three commonly used metrics: lines of feature code, scattering degree, and tangling degree. We (1) describe how the considered directives impact these
metrics, (2) unveil the resulting differences within 19 systems, and (3) propose how to address the arising issues. The results show that the corner cases appear regularly in variable feature code and can heavily change the results obtained with established metrics. We argue that we need to refine metrics and improve variability analysis techniques to provide more precise results, but we also need to reason about the meaning of corner cases and metrics.