Towards a resource manager for scheduling frameworks

Aleksandra Kuzmanovska, Rudolf H. Mak, Dick Epema

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)
1 Downloads (Pure)

Abstract

Due to the diversity in the applications that run in large distributed environments, many different application frameworks have been developed, such as MapReduce for data-intensive batch jobs and Spark for interactive data analytics. After initial deployment, a framework starts executing a large set of jobs that are submitted over time. When multiple such frameworks with time-varying resource demands are consolidated in a large distributed environment, static allocation of resources on a per-framework basis leads to low system utilization and to resource fragmentation. The goal of my PhD research is to improve the system utilization and framework performances in such consolidated environments by using dynamic resource allocation for efficient resource sharing among frameworks. My contribution towards this goal is a design and an implementation of a scalable resource manager that dynamically balances resources across set of multiple diverse frameworks in a large distributed environment based on resource requirements, system utilization or performance levels in the deployed frameworks.
Original languageEnglish
Title of host publication2016 16th IEEE/ACM International Symposium on Custer, Cloud, and Grid Computing, CCGrid 2016, 16-19 May 2016, Cartagena, Colombia
PublisherIEEE Computer Society
Pages592-595
ISBN (Print)978-1-5090-2452-0
DOIs
Publication statusPublished - 2016
Event16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), May 16-19, 2016, Cartagena, Colombia - Cartagena, Colombia
Duration: 16 May 201619 May 2016

Conference

Conference16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), May 16-19, 2016, Cartagena, Colombia
Abbreviated titleCCGrid 2016
Country/TerritoryColombia
CityCartagena
Period16/05/1619/05/16

Fingerprint

Dive into the research topics of 'Towards a resource manager for scheduling frameworks'. Together they form a unique fingerprint.

Cite this