Abstract
The growing number of models and other related artefacts in model-driven engineering has recently led to the emergence of approaches and tools for analyzing and managing them on a large scale. The framework SAMOS applies techniques inspired by information retrieval and data mining to analyze large sets of models. As the data size and analysis complexity goes up, however, further scalability is needed. In this paper we extend SAMOS to operate on Apache Spark, a popular engine for distributed Big Data processing, by partitioning the data and parallelizing the comparison and analysis phase. We present preliminary studies using a cluster infrastructure and report the results for two datasets: one with 250 Ecore metamodels where we detail the performance gain with various settings, and a larger one of 7.3k metamodels with nearly one million model elements for further demonstrating scalability.
| Original language | English |
|---|---|
| Title of host publication | MODELSWARD 2018 - Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development |
| Editors | Slimane Hammoudi , Luis Ferreira Pires |
| Publisher | SCITEPRESS-Science and Technology Publications, Lda. |
| Pages | 767-772 |
| Number of pages | 6 |
| ISBN (Electronic) | 978-989-758-283-7 |
| DOIs | |
| Publication status | Published - 1 Jan 2018 |
| Event | 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018 - Funchal, Madeira, Portugal Duration: 22 Jan 2018 → 24 Jan 2018 |
Conference
| Conference | 6th International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018 |
|---|---|
| Country | Portugal |
| City | Funchal, Madeira |
| Period | 22/01/18 → 24/01/18 |
Fingerprint
Keywords
- Apache Spark
- Big Data
- Distributed Computing
- Model Analytics
- Model-Driven Engineering
- Scalability
Cite this
}
Towards distributed model analytics with apache spark. / Babur, Önder; Cleophas, Loek; van den Brand, Mark.
MODELSWARD 2018 - Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development. ed. / Slimane Hammoudi ; Luis Ferreira Pires . SCITEPRESS-Science and Technology Publications, Lda., 2018. p. 767-772.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review
TY - GEN
T1 - Towards distributed model analytics with apache spark
AU - Babur, Önder
AU - Cleophas, Loek
AU - van den Brand, Mark
PY - 2018/1/1
Y1 - 2018/1/1
N2 - The growing number of models and other related artefacts in model-driven engineering has recently led to the emergence of approaches and tools for analyzing and managing them on a large scale. The framework SAMOS applies techniques inspired by information retrieval and data mining to analyze large sets of models. As the data size and analysis complexity goes up, however, further scalability is needed. In this paper we extend SAMOS to operate on Apache Spark, a popular engine for distributed Big Data processing, by partitioning the data and parallelizing the comparison and analysis phase. We present preliminary studies using a cluster infrastructure and report the results for two datasets: one with 250 Ecore metamodels where we detail the performance gain with various settings, and a larger one of 7.3k metamodels with nearly one million model elements for further demonstrating scalability.
AB - The growing number of models and other related artefacts in model-driven engineering has recently led to the emergence of approaches and tools for analyzing and managing them on a large scale. The framework SAMOS applies techniques inspired by information retrieval and data mining to analyze large sets of models. As the data size and analysis complexity goes up, however, further scalability is needed. In this paper we extend SAMOS to operate on Apache Spark, a popular engine for distributed Big Data processing, by partitioning the data and parallelizing the comparison and analysis phase. We present preliminary studies using a cluster infrastructure and report the results for two datasets: one with 250 Ecore metamodels where we detail the performance gain with various settings, and a larger one of 7.3k metamodels with nearly one million model elements for further demonstrating scalability.
KW - Apache Spark
KW - Big Data
KW - Distributed Computing
KW - Model Analytics
KW - Model-Driven Engineering
KW - Scalability
UR - http://www.scopus.com/inward/record.url?scp=85052021429&partnerID=8YFLogxK
U2 - 10.5220/0006735407670772
DO - 10.5220/0006735407670772
M3 - Conference contribution
AN - SCOPUS:85052021429
SP - 767
EP - 772
BT - MODELSWARD 2018 - Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development
A2 - Hammoudi , Slimane
A2 - Ferreira Pires , Luis
PB - SCITEPRESS-Science and Technology Publications, Lda.
ER -