How Explicit Feature Traces Did Not Impact Developers' Memory

Software features are intuitive entities used to abstract and manage the functionalities of a software system, for instance, in product-line engineering and agile software development. Nonetheless, developers rarely make features explicit
in code, which is why they have to perform costly program comprehension and particularly feature location to (re-)gain knowledge about the code. In a previous paper, we conducted an experiment on how explicit feature traces impact developers’ program comprehension by facilitating feature location. We found that annotating features in code improved program comprehension, while decomposing them into classes had a negative impact. Additionally, but not reported in that paper, we were concerned with understanding whether the different traces would impact developers’ memory regarding the code and its features. To this end, we repeatedly asked our participants questions about the code on different levels of detail within time periods of two weeks. Since developers’ memory decays over time, we expected that our participants would provide fewer correct answers over time, with differences depending on the feature traces in their code. Unfortunately, the actual results were inconclusive and up for interpretation, particularly due to challenges in designing an experiment on developers’ memory. In this paper, we discuss our experimental design, the null results, and challenges for improving the methodology of future studies in this direction.