Assessing and improving the quality of modeling : a series of empirical studies about the UML

TY - THES

T1 - Assessing and improving the quality of modeling : a series of empirical studies about the UML

AU - Lange, C.F.J.

N1 - Proefschrift.

PY - 2007

Y1 - 2007

N2 - Assessing and Improving the Quality of Modeling A Series of Empirical Studies about the UML This thesis addresses the assessment and improvement of the quality of modeling in software engineering. In particular, we focus on the Unified Modeling Language (UML), which is the de facto standard in industrial practice. The language is used for a wide variety of purposes, such as specification, maintenance, comprehension, communication, test case generation, and code generation. The UML has some inherent characteristics, that cause risks to the quality of UML modeling. The characteristics of interest in this thesis are its lack of a formal semantics, its multidiagram architecture, and its large complexity. These characteristics can lead to quality problems with respect to correctness, comprehensibility, consistency, nonredundancy, completeness, or unambiguity. In this thesis we assess the quality of modeling in practice, and we provide and evaluate techniques to improve the quality of modeling. We conducted three large-scale experiments with 365 participants in total. Additionally a series of industrial case studies was conducted. To define the notion of quality that is used throughout this thesis, we present a framework that is based on existing work. The framework decomposes quality of models into the following quality notions: system, semantic, syntactic, pragmatic, social, and communicative quality, and correspondence between the model and the implementation. We use these notions throughout the thesis to denote which aspect of quality is addressed in each of the presented studies. Additionally, we propose a quality model for UML modeling. The purpose of the quality model is to support developers in selecting metrics and rules to analyze the quality of a model with respect to a particular purpose of modeling. We report on a series of industrial case studies. We conducted the case studies to assess the quality of UML models in practice. The results of the case studies reveal the frequency of occurrence of defect types in modeling. This knowledge can be used to focus quality assurance techniques on common quality problems. Through the case studies we discovered defects in UML models. We conducted an experiment to study the effects of several of the discovered defect types. The results of the experiment show that defects often remain undetected by developers. Furthermore, defects cause a variety of different interpretations of the model amongst developers and, hence, lead to misinterpretations and miscommunication. An additional result of the experiment is a classification of defect types with respect to their likelihood of non-detection and misinterpretation. This objective classification can be used to prioritize defects, such that removal effort is assigned to the most severe defects first. As a preventive quality assurance technique we propose modeling conventions, similar to coding conventions for programming. Based on a literature review we provide a classification of modeling conventions. We report on an exploratory experiment that studied the effectiveness of modeling conventions with respect to defect prevention and the extra effort entailed by modeling conventions. The results show a slight improvement with respect to defect occurrences, however, these results are not statistically significant. The results showing an increase in development effort are significant. Based on the experiment we provide recommendations to improve the benefit obtained by using modeling conventions. The recommendations include improved adherence to the modeling conventions, training and experience, and developer motivation. Finally, we propose task-oriented views for UML. Task-oriented views are visualizations of UML models that support developers by providing the information that is necessary for a particular task, such as maintenance, comprehension, or quality analysis. We argue that the existing UML diagram types and existing UML tools do not provide the required information effectively. For example, it is tedious to find the relations between model elements in different diagrams and to relate data such as metrics, bug data, or evolution data to model elements. We discuss a framework that we used as a basis for creating task-oriented views. The proposed views are: MetaView, ContextView, MetricView, UML-City-View, Quality Tree View, and Evolution View. We conducted an experiment to validate the views with respect to comprehension tasks. The results are very promising and show that the correctness of comprehension is improved by 4.5% and that the effort for comprehension is reduced by 20%. The proposed task-oriented views are implemented in the tool MetricView Evolution.

AB - Assessing and Improving the Quality of Modeling A Series of Empirical Studies about the UML This thesis addresses the assessment and improvement of the quality of modeling in software engineering. In particular, we focus on the Unified Modeling Language (UML), which is the de facto standard in industrial practice. The language is used for a wide variety of purposes, such as specification, maintenance, comprehension, communication, test case generation, and code generation. The UML has some inherent characteristics, that cause risks to the quality of UML modeling. The characteristics of interest in this thesis are its lack of a formal semantics, its multidiagram architecture, and its large complexity. These characteristics can lead to quality problems with respect to correctness, comprehensibility, consistency, nonredundancy, completeness, or unambiguity. In this thesis we assess the quality of modeling in practice, and we provide and evaluate techniques to improve the quality of modeling. We conducted three large-scale experiments with 365 participants in total. Additionally a series of industrial case studies was conducted. To define the notion of quality that is used throughout this thesis, we present a framework that is based on existing work. The framework decomposes quality of models into the following quality notions: system, semantic, syntactic, pragmatic, social, and communicative quality, and correspondence between the model and the implementation. We use these notions throughout the thesis to denote which aspect of quality is addressed in each of the presented studies. Additionally, we propose a quality model for UML modeling. The purpose of the quality model is to support developers in selecting metrics and rules to analyze the quality of a model with respect to a particular purpose of modeling. We report on a series of industrial case studies. We conducted the case studies to assess the quality of UML models in practice. The results of the case studies reveal the frequency of occurrence of defect types in modeling. This knowledge can be used to focus quality assurance techniques on common quality problems. Through the case studies we discovered defects in UML models. We conducted an experiment to study the effects of several of the discovered defect types. The results of the experiment show that defects often remain undetected by developers. Furthermore, defects cause a variety of different interpretations of the model amongst developers and, hence, lead to misinterpretations and miscommunication. An additional result of the experiment is a classification of defect types with respect to their likelihood of non-detection and misinterpretation. This objective classification can be used to prioritize defects, such that removal effort is assigned to the most severe defects first. As a preventive quality assurance technique we propose modeling conventions, similar to coding conventions for programming. Based on a literature review we provide a classification of modeling conventions. We report on an exploratory experiment that studied the effectiveness of modeling conventions with respect to defect prevention and the extra effort entailed by modeling conventions. The results show a slight improvement with respect to defect occurrences, however, these results are not statistically significant. The results showing an increase in development effort are significant. Based on the experiment we provide recommendations to improve the benefit obtained by using modeling conventions. The recommendations include improved adherence to the modeling conventions, training and experience, and developer motivation. Finally, we propose task-oriented views for UML. Task-oriented views are visualizations of UML models that support developers by providing the information that is necessary for a particular task, such as maintenance, comprehension, or quality analysis. We argue that the existing UML diagram types and existing UML tools do not provide the required information effectively. For example, it is tedious to find the relations between model elements in different diagrams and to relate data such as metrics, bug data, or evolution data to model elements. We discuss a framework that we used as a basis for creating task-oriented views. The proposed views are: MetaView, ContextView, MetricView, UML-City-View, Quality Tree View, and Evolution View. We conducted an experiment to validate the views with respect to comprehension tasks. The results are very promising and show that the correctness of comprehension is improved by 4.5% and that the effort for comprehension is reduced by 20%. The proposed task-oriented views are implemented in the tool MetricView Evolution.

U2 - 10.6100/IR629604

DO - 10.6100/IR629604

M3 - Phd Thesis 1 (Research TU/e / Graduation TU/e)

SN - 978-90-386-1107-5

T3 - IPA Dissertation series

PB - Technische Universiteit Eindhoven

CY - Eindhoven

ER -