Towards a resource manager for scheduling frameworks

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/CongresprocedureConferentiebijdrageAcademicpeer review

Uittreksel

Due to the diversity in the applications that run in large distributed environments, many different application frameworks have been developed, such as MapReduce for data-intensive batch jobs and Spark for interactive data analytics. After initial deployment, a framework starts executing a large set of jobs that are submitted over time. When multiple such frameworks with time-varying resource demands are consolidated in a large distributed environment, static allocation of resources on a per-framework basis leads to low system utilization and to resource fragmentation. The goal of my PhD research is to improve the system utilization and framework performances in such consolidated environments by using dynamic resource allocation for efficient resource sharing among frameworks. My contribution towards this goal is a design and an implementation of a scalable resource manager that dynamically balances resources across set of multiple diverse frameworks in a large distributed environment based on resource requirements, system utilization or performance levels in the deployed frameworks.
TaalEngels
Titel2016 16th IEEE/ACM International Symposium on Custer, Cloud, and Grid Computing, CCGrid 2016, 16-19 May 2016, Cartagena, Colombia
UitgeverijIEEE Computer Society
Pagina's592-595
ISBN van geprinte versie978-1-5090-2452-0
DOI's
StatusGepubliceerd - 2016
Evenement16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), May 16-19, 2016, Cartagena, Colombia - Cartagena, Colombia
Duur: 16 mei 201619 mei 2016

Congres

Congres16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), May 16-19, 2016, Cartagena, Colombia
Verkorte titelCCGrid 2016
LandColombia
StadCartagena
Periode16/05/1619/05/16

Vingerafdruk

Managers
Scheduling
Electric sparks
Resource allocation

Citeer dit

Kuzmanovska, A., Mak, R. H., & Epema, D. (2016). Towards a resource manager for scheduling frameworks. In 2016 16th IEEE/ACM International Symposium on Custer, Cloud, and Grid Computing, CCGrid 2016, 16-19 May 2016, Cartagena, Colombia (blz. 592-595). IEEE Computer Society. DOI: 10.1109/CCGrid.2016.70
Kuzmanovska, Aleksandra ; Mak, Rudolf H. ; Epema, Dick. / Towards a resource manager for scheduling frameworks. 2016 16th IEEE/ACM International Symposium on Custer, Cloud, and Grid Computing, CCGrid 2016, 16-19 May 2016, Cartagena, Colombia. IEEE Computer Society, 2016. blz. 592-595
@inproceedings{fe04085a33be4f13ae68c5208d0d78da,
title = "Towards a resource manager for scheduling frameworks",
abstract = "Due to the diversity in the applications that run in large distributed environments, many different application frameworks have been developed, such as MapReduce for data-intensive batch jobs and Spark for interactive data analytics. After initial deployment, a framework starts executing a large set of jobs that are submitted over time. When multiple such frameworks with time-varying resource demands are consolidated in a large distributed environment, static allocation of resources on a per-framework basis leads to low system utilization and to resource fragmentation. The goal of my PhD research is to improve the system utilization and framework performances in such consolidated environments by using dynamic resource allocation for efficient resource sharing among frameworks. My contribution towards this goal is a design and an implementation of a scalable resource manager that dynamically balances resources across set of multiple diverse frameworks in a large distributed environment based on resource requirements, system utilization or performance levels in the deployed frameworks.",
author = "Aleksandra Kuzmanovska and Mak, {Rudolf H.} and Dick Epema",
year = "2016",
doi = "10.1109/CCGrid.2016.70",
language = "English",
isbn = "978-1-5090-2452-0",
pages = "592--595",
booktitle = "2016 16th IEEE/ACM International Symposium on Custer, Cloud, and Grid Computing, CCGrid 2016, 16-19 May 2016, Cartagena, Colombia",
publisher = "IEEE Computer Society",
address = "United States",

}

Kuzmanovska, A, Mak, RH & Epema, D 2016, Towards a resource manager for scheduling frameworks. in 2016 16th IEEE/ACM International Symposium on Custer, Cloud, and Grid Computing, CCGrid 2016, 16-19 May 2016, Cartagena, Colombia. IEEE Computer Society, blz. 592-595, Cartagena, Colombia, 16/05/16. DOI: 10.1109/CCGrid.2016.70

Towards a resource manager for scheduling frameworks. / Kuzmanovska, Aleksandra; Mak, Rudolf H.; Epema, Dick.

2016 16th IEEE/ACM International Symposium on Custer, Cloud, and Grid Computing, CCGrid 2016, 16-19 May 2016, Cartagena, Colombia. IEEE Computer Society, 2016. blz. 592-595.

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/CongresprocedureConferentiebijdrageAcademicpeer review

TY - GEN

T1 - Towards a resource manager for scheduling frameworks

AU - Kuzmanovska,Aleksandra

AU - Mak,Rudolf H.

AU - Epema,Dick

PY - 2016

Y1 - 2016

N2 - Due to the diversity in the applications that run in large distributed environments, many different application frameworks have been developed, such as MapReduce for data-intensive batch jobs and Spark for interactive data analytics. After initial deployment, a framework starts executing a large set of jobs that are submitted over time. When multiple such frameworks with time-varying resource demands are consolidated in a large distributed environment, static allocation of resources on a per-framework basis leads to low system utilization and to resource fragmentation. The goal of my PhD research is to improve the system utilization and framework performances in such consolidated environments by using dynamic resource allocation for efficient resource sharing among frameworks. My contribution towards this goal is a design and an implementation of a scalable resource manager that dynamically balances resources across set of multiple diverse frameworks in a large distributed environment based on resource requirements, system utilization or performance levels in the deployed frameworks.

AB - Due to the diversity in the applications that run in large distributed environments, many different application frameworks have been developed, such as MapReduce for data-intensive batch jobs and Spark for interactive data analytics. After initial deployment, a framework starts executing a large set of jobs that are submitted over time. When multiple such frameworks with time-varying resource demands are consolidated in a large distributed environment, static allocation of resources on a per-framework basis leads to low system utilization and to resource fragmentation. The goal of my PhD research is to improve the system utilization and framework performances in such consolidated environments by using dynamic resource allocation for efficient resource sharing among frameworks. My contribution towards this goal is a design and an implementation of a scalable resource manager that dynamically balances resources across set of multiple diverse frameworks in a large distributed environment based on resource requirements, system utilization or performance levels in the deployed frameworks.

U2 - 10.1109/CCGrid.2016.70

DO - 10.1109/CCGrid.2016.70

M3 - Conference contribution

SN - 978-1-5090-2452-0

SP - 592

EP - 595

BT - 2016 16th IEEE/ACM International Symposium on Custer, Cloud, and Grid Computing, CCGrid 2016, 16-19 May 2016, Cartagena, Colombia

PB - IEEE Computer Society

ER -

Kuzmanovska A, Mak RH, Epema D. Towards a resource manager for scheduling frameworks. In 2016 16th IEEE/ACM International Symposium on Custer, Cloud, and Grid Computing, CCGrid 2016, 16-19 May 2016, Cartagena, Colombia. IEEE Computer Society. 2016. blz. 592-595. Beschikbaar vanaf, DOI: 10.1109/CCGrid.2016.70