Towards Developer Support for Merging Forked Test Cases

Developers rely on branching and forking mechanisms of modern versioning systems to evolve and maintain their software systems. As a result, systems often exist in the form of various short-living or even long-living (i.e., clone & own development) variants. Such variants may have to be merged with the main system or other variants, for instance, to propagate features or bug fixes. Within such merging processes, test cases are highly interesting, since they allow to improve the test coverage and hopefully the reliability of the system (e.g., by merging missing tests and bug fixes in test code). However, as all source code, test cases may evolve independently between two or more variants, which makes it non-trivial to decide what changes of the test cases are relevant for the merging. For instance, some test cases in one variant may be irrelevant in another variant (e.g., because the feature shall not be propagated) or may subsume existing test cases. In this paper, we propose a technique that allows for a fine-grained comparison of test cases to support developers in deciding whether and how to merge these. Precisely, inspired by code-clone detection, we use abstract syntax trees to decide on the relations between test cases of different variants. We evaluate the applicability of our technique qualitatively on five open-source systems written in Java (e.g., JUnit 5, Guava). Our insights into the merge potential of 50 pull requests with test cases from these systems indicate that our technique can support the comprehension of differences in variants’ test cases, and also highlight future research opportunities.