Assessing the quality of model-comparison tools: a method and a benchmark data set

M.G.J. Brand, van den; A.T. Hofkamp; T. Verhoeff; Z. Protic

doi:10.1145/2000410.2000412

Assessing the quality of model-comparison tools: a method and a benchmark data set

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Model comparison is an important aspect of model driven software engineering. In particular, exploring the evolution of a model would be impossible without means for comparing different versions of that model. However, the techniques and tools for model comparison are still being perfected for practical application. Moreover, there exist no systematic methods and no controlled benchmarks that could be used for assessing the quality of tools for model comparison. In this paper, we describe a systematic method for assessing the quality of model-comparison tools, and we present a data set to be used for controlled assessment experiments. Additionally, we use our method, and the specified data, to asses the quality of two model-comparison tools, namely EMFCompare and RCVDiff. The results of the experiments show that, in generic cases, both tools exhibit similar performance, and that both tools are of similar quality, though there are some notable difference in the details. The defined method, the selected dataset, and the results obtained by assessing the two mentioned tools, constitute a benchmark for model-comparison tools.

Original language	English
Title of host publication	Proceedings of the 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) June 30, 2011; held in conjunction with TOOLS 2011 , Zurich, Switzerland
Place of Publication	New York NY
Publisher	Association for Computing Machinery, Inc.
Pages	2-11
ISBN (Print)	978-1-4503-0668-3
DOIs	https://doi.org/10.1145/2000410.2000412
Publication status	Published - 2011
Event	conference; 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) - Duration: 1 Jan 2011 → …

Conference

Conference	conference; 2nd International Workshop on Model Comparison in Practice (IWMCP 2011)
Period	1/01/11 → …
Other	2nd International Workshop on Model Comparison in Practice (IWMCP 2011)

Access to Document

10.1145/2000410.2000412

Cite this

Brand, van den, M. G. J., Hofkamp, A. T., Verhoeff, T., & Protic, Z. (2011). Assessing the quality of model-comparison tools: a method and a benchmark data set. In Proceedings of the 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) June 30, 2011; held in conjunction with TOOLS 2011 , Zurich, Switzerland (pp. 2-11). Association for Computing Machinery, Inc.. https://doi.org/10.1145/2000410.2000412

Brand, van den, M.G.J. ; Hofkamp, A.T. ; Verhoeff, T. et al. / Assessing the quality of model-comparison tools: a method and a benchmark data set. Proceedings of the 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) June 30, 2011; held in conjunction with TOOLS 2011 , Zurich, Switzerland. New York NY : Association for Computing Machinery, Inc., 2011. pp. 2-11

@inproceedings{1255e4e6c38f4406b234d93ff4fa2bac,

title = "Assessing the quality of model-comparison tools: a method and a benchmark data set",

abstract = "Model comparison is an important aspect of model driven software engineering. In particular, exploring the evolution of a model would be impossible without means for comparing different versions of that model. However, the techniques and tools for model comparison are still being perfected for practical application. Moreover, there exist no systematic methods and no controlled benchmarks that could be used for assessing the quality of tools for model comparison. In this paper, we describe a systematic method for assessing the quality of model-comparison tools, and we present a data set to be used for controlled assessment experiments. Additionally, we use our method, and the specified data, to asses the quality of two model-comparison tools, namely EMFCompare and RCVDiff. The results of the experiments show that, in generic cases, both tools exhibit similar performance, and that both tools are of similar quality, though there are some notable difference in the details. The defined method, the selected dataset, and the results obtained by assessing the two mentioned tools, constitute a benchmark for model-comparison tools.",

author = "\{Brand, van den\}, M.G.J. and A.T. Hofkamp and T. Verhoeff and Z. Protic",

year = "2011",

doi = "10.1145/2000410.2000412",

language = "English",

isbn = "978-1-4503-0668-3",

pages = "2--11",

booktitle = "Proceedings of the 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) June 30, 2011; held in conjunction with TOOLS 2011 , Zurich, Switzerland",

publisher = "Association for Computing Machinery, Inc.",

address = "United States",

note = "conference; 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) ; Conference date: 01-01-2011",

}

Brand, van den, MGJ , Hofkamp, AT , Verhoeff, T & Protic, Z 2011, Assessing the quality of model-comparison tools: a method and a benchmark data set. in Proceedings of the 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) June 30, 2011; held in conjunction with TOOLS 2011 , Zurich, Switzerland. Association for Computing Machinery, Inc., New York NY, pp. 2-11, conference; 2nd International Workshop on Model Comparison in Practice (IWMCP 2011), 1/01/11. https://doi.org/10.1145/2000410.2000412

Assessing the quality of model-comparison tools: a method and a benchmark data set. / Brand, van den, M.G.J.; Hofkamp, A.T.; Verhoeff, T. et al.
Proceedings of the 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) June 30, 2011; held in conjunction with TOOLS 2011 , Zurich, Switzerland. New York NY: Association for Computing Machinery, Inc., 2011. p. 2-11.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Assessing the quality of model-comparison tools: a method and a benchmark data set

AU - Brand, van den, M.G.J.

AU - Hofkamp, A.T.

AU - Verhoeff, T.

AU - Protic, Z.

PY - 2011

Y1 - 2011

N2 - Model comparison is an important aspect of model driven software engineering. In particular, exploring the evolution of a model would be impossible without means for comparing different versions of that model. However, the techniques and tools for model comparison are still being perfected for practical application. Moreover, there exist no systematic methods and no controlled benchmarks that could be used for assessing the quality of tools for model comparison. In this paper, we describe a systematic method for assessing the quality of model-comparison tools, and we present a data set to be used for controlled assessment experiments. Additionally, we use our method, and the specified data, to asses the quality of two model-comparison tools, namely EMFCompare and RCVDiff. The results of the experiments show that, in generic cases, both tools exhibit similar performance, and that both tools are of similar quality, though there are some notable difference in the details. The defined method, the selected dataset, and the results obtained by assessing the two mentioned tools, constitute a benchmark for model-comparison tools.

AB - Model comparison is an important aspect of model driven software engineering. In particular, exploring the evolution of a model would be impossible without means for comparing different versions of that model. However, the techniques and tools for model comparison are still being perfected for practical application. Moreover, there exist no systematic methods and no controlled benchmarks that could be used for assessing the quality of tools for model comparison. In this paper, we describe a systematic method for assessing the quality of model-comparison tools, and we present a data set to be used for controlled assessment experiments. Additionally, we use our method, and the specified data, to asses the quality of two model-comparison tools, namely EMFCompare and RCVDiff. The results of the experiments show that, in generic cases, both tools exhibit similar performance, and that both tools are of similar quality, though there are some notable difference in the details. The defined method, the selected dataset, and the results obtained by assessing the two mentioned tools, constitute a benchmark for model-comparison tools.

U2 - 10.1145/2000410.2000412

DO - 10.1145/2000410.2000412

M3 - Conference contribution

SN - 978-1-4503-0668-3

SP - 2

EP - 11

BT - Proceedings of the 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) June 30, 2011; held in conjunction with TOOLS 2011 , Zurich, Switzerland

PB - Association for Computing Machinery, Inc.

CY - New York NY

T2 - conference; 2nd International Workshop on Model Comparison in Practice (IWMCP 2011)

Y2 - 1 January 2011

ER -

Brand, van den MGJ , Hofkamp AT , Verhoeff T, Protic Z. Assessing the quality of model-comparison tools: a method and a benchmark data set. In Proceedings of the 2nd International Workshop on Model Comparison in Practice (IWMCP 2011) June 30, 2011; held in conjunction with TOOLS 2011 , Zurich, Switzerland. New York NY: Association for Computing Machinery, Inc. 2011. p. 2-11 doi: 10.1145/2000410.2000412

Assessing the quality of model-comparison tools: a method and a benchmark data set

Abstract

Conference

Access to Document

Fingerprint

Cite this